Every AI engagement we start seems to arrive with this question already half-answered, usually by whoever read the most recent blog post. The honest answer is that retrieval and fine-tuning solve different problems, and the choice falls out of two properties of your business rather than any benchmark.
How often does the knowledge change?
Fine-tuning bakes information into model weights. Updating it means another training run, another evaluation pass, and another deployment. If your knowledge base changes weekly — pricing, policy, inventory, documentation — you have signed up for a retraining treadmill.
Retrieval fetches the current answer at query time. Updating it means updating a document. For anything that changes faster than your release cadence, this alone settles the argument.
Do you need to show your work?
A fine-tuned model produces an answer with no provenance. A retrieval system produces an answer attached to the passages it drew from. In regulated contexts, support settings, and anywhere a wrong answer has a cost attached, that citation is not a nice-to-have — it is the thing that makes the system defensible when someone challenges an output.
Where fine-tuning genuinely wins
Fine-tuning is the right tool for shaping behaviour rather than supplying facts: a consistent house tone, a rigid output schema, a domain vocabulary the base model mangles, or latency and cost targets that a smaller tuned model can hit where a larger general one cannot.
The two are not exclusive. A good number of our production systems fine-tune a small model for format and tone, then ground every response through retrieval. The framing to avoid is treating this as a single either-or decision made once at the start.
The test we actually apply
Before committing either way, we build an evaluation set from real queries — a hundred or so, drawn from support tickets or search logs, with agreed correct answers. It is unglamorous work and it is the difference between a system you can improve and a demo you can only argue about.