You want to fine-tune an LLM. Chances are, you don’t need to.

There’s a trap that almost every developer falls into the first time they explore fine-tuning.

They see an LLM that doesn’t know their product, their terminology, or their brand voice. So they start looking for ways to adapt it. They come across fine-tuning. They dive in.

But nine times out of ten, they never needed it in the first place.

The Real Problem

Large language models—GPT, Claude, Llama—are trained on billions of tokens from the web, books, and public code. They know an enormous amount.

What they don’t know is your product, your internal acronyms, your brand voice, or your private data.

The question isn’t « How do I teach the model all of this? »

The real question is:

Do I want to change what the model knows, or how it behaves?

That single distinction determines everything.

RAG First. Fine-Tuning Only If You Really Need It.

RAG: Changing What the Model Knows

Retrieval-Augmented Generation (RAG) injects relevant documents into the prompt at inference time. The model itself never changes—it simply reads the information you provide.

Compared to fine-tuning, RAG has three massive advantages:

No training required. No GPUs, no ML training pipeline, no waiting.
Real-time updates. Update a PDF today, and the next query immediately benefits. With fine-tuning, you’d have to retrain the model.
Traceability. The model can cite the exact source it used. With fine-tuning, information becomes embedded across billions of parameters, making it impossible to trace.

A concrete example:

Suppose you want an assistant that answers questions using your product documentation, Jira tickets, and customer emails.

Use RAG.

There’s really no debate.

A simple rule of thumb:

If you want to change what the model knows, use RAG.

If you want to change how the model behaves, use fine-tuning.

When Fine-Tuning Actually Makes Sense

There are only a few situations where fine-tuning provides something that RAG simply cannot.

1. You Want to Change the Model’s Behavior

Maybe you want the model to:

Always return a specific JSON schema.
Adopt a concise, factual writing style.
Consistently refuse certain categories of requests.
Follow formatting rules with near-perfect reliability.

Prompt engineering can get you surprisingly far.

Fine-tuning pushes these behaviors into the model itself.

2. You Have Extremely Specialized Vocabulary

Fields like:

Quantitative finance
Law
Medicine
Internal enterprise jargon

General-purpose models will often approximate unfamiliar terminology.

Fine-tuning teaches those concepts much more deeply.

Three Fine-Tuning Techniques

If you’ve determined that fine-tuning is genuinely necessary, you have three options.

Between the most expensive and the cheapest, there’s roughly a 1000× cost difference.

1. Full Fine-Tuning — The Original Approach

You take the pretrained model and continue training it on your own data.

Every single parameter is updated.

For Llama 70B, that’s 70 billion parameters.

A Quick Refresher on Parameters and Gradients

A parameter is simply a number stored inside the model.

Llama 70B contains about 70 billion of them.

Training means adjusting those numbers to reduce prediction error.

The gradient tells each parameter which direction to move.

Gradient computation is the most expensive part of training.

More parameters mean more gradients.

More gradients mean dramatically higher costs.

What It Looks Like for Llama 70B

Typical requirements include:

Around 8 H100 GPUs
Roughly 1 TB of VRAM in practice (parameters, gradients, optimizer state)
Approximately €200,000 of infrastructure, or €30–50/hour in the cloud
Several days of training for a dataset of around 10,000 examples

There’s another problem:

Catastrophic forgetting.

If you push all 70 billion parameters too far toward your own domain, the model can actually lose capabilities it previously had.

It becomes better at your niche—and worse everywhere else.

For years, this was the only option.

Then, in 2021, everything changed.

2. LoRA — Low-Rank Adaptation (Microsoft Research, 2021)

The paper that changed the game:

« LoRA: Low-Rank Adaptation of Large Language Models »
Hu et al., Microsoft Research (2021)

The core idea is remarkably simple.

Freeze the original 70 billion parameters.

Instead of modifying them, add two small matrices—called A and B—alongside selected layers.

Imagine an original layer with dimensions:

4096 × 4096

That’s roughly 16 million parameters.

LoRA instead learns:

A matrix of 4096 × 32
A matrix of 32 × 4096

Multiplying A × B produces another matrix with the same dimensions as the original layer.

That correction is simply added to the layer’s output.

Rather than learning an entirely new model, you’re learning a small correction.

Those matrices contain roughly 250,000 parameters instead of 16 million.

Across Llama 70B, LoRA typically trains only 0.1–0.3% of the model.

That’s around 70 to 200 million parameters, instead of 70 billion.

The Practical Impact

Instead of:

8 H100 GPUs

You now need roughly:

1–2 H100 GPUs

VRAM drops from roughly:

1 TB → 150 GB

Training costs fall from:

Several thousand euros
to roughly €100–500

Performance?

Around 95% of full fine-tuning on common benchmarks.

Why does this work so well?

Because adapting a model to a specific domain doesn’t require changing everything.

The useful information lives in a surprisingly low-dimensional space.

An additional advantage:

No catastrophic forgetting.

The original model stays frozen.

Only the learned correction is applied.

You can even keep multiple LoRA adapters and switch between them depending on the task.

3. QLoRA — Quantized LoRA (University of Washington, 2023)

The next breakthrough came with:

« QLoRA: Efficient Finetuning of Quantized LLMs »
Dettmers et al., University of Washington (2023)

LoRA solved one major problem.

But another remained.

Even though the original parameters are frozen, they still have to stay in GPU memory.

In 16-bit precision, Llama 70B alone occupies roughly 140 GB.

QLoRA asks:

« What if we stored those frozen parameters using only 4 bits instead of 16? »

Think of it like MP3 compression.

You lose a tiny amount of precision, but almost no practical quality.

The result:

The 70-billion-parameter model now occupies roughly 35 GB instead of 140 GB.

Suddenly, an RTX 4090 can fine-tune Llama 70B.

On your own desktop.

What Changes?

Total training cost: roughly €30–100
Performance: 99.3% of full fine-tuning, according to the original paper
Easily accessible through libraries like PEFT (Hugging Face) and Axolotl

The Decision Matrix

Three Things to Remember

1. Fine-Tuning Changes Behavior. RAG Adds Knowledge.

If you need to inject facts, documentation, or company knowledge…

Use RAG.

If you need to modify how the model behaves…

Consider fine-tuning.

2. Three Techniques, Three Price Tags

Technique	Cost	Performance
Full Fine-Tuning	100%	100%
LoRA	~5%	~95%
QLoRA	~1%	~99.3%

3. The Decision Order

Start here:

RAG

↓

If that’s not enough:

QLoRA

↓

Only choose Full Fine-Tuning if you know exactly why you need it.

You Want to Fine-Tune an LLM. You Probably Don’t Need To.