In this article
The essential idea
RAG supplies relevant information at response time. Fine-tuning changes learned behaviour through training examples. First determine whether the failure comes from missing knowledge, an inconsistent task or the surrounding application.
Two failures that look deceptively similar
Consider a service assistant that quotes last year’s warranty terms. Now consider an assistant that has the correct terms but produces an unusable response format. Both give a poor answer, but the first lacks applicable evidence while the second struggles with a repeatable behaviour. Buying a more elaborate training pipeline before distinguishing them can address the wrong problem.
Collect failed examples and inspect what information the model actually received. If the current policy was absent, start with retrieval or an application lookup. If the evidence was present, check instructions, output constraints and the difficulty of the task before considering training. This diagnosis also gives you a baseline for comparing alternatives.
What retrieval changes in the response
In a RAG workflow, the application retrieves passages relevant to a question and places them in the context used to generate a response. That makes a document collection available without encoding every revision into model weights. It does not ensure that the latest document was indexed, that the right passage was selected or that the answer faithfully follows it.
For warranty questions, attach product, country and effective date to each source. Retrieve only material the user can access and show the supporting passage. If the request concerns a live shipment status, query the operational system instead: a periodically indexed document is a poor substitute for an authoritative current transaction.
Knowledge or behaviour?
Two different levers; a project may use both.
RAG
Retrieve current evidence at question time. Maintain sources, permissions and citations.
Fine-tuning
Adapt behaviour using training examples. Maintain the dataset and evaluate the resulting model.
What training can address
Fine-tuning uses examples to adapt a supported model to a task or response behaviour. A possible candidate is consistently mapping varied messages to a stable business taxonomy after simpler prompting has been evaluated. The examples must represent the distinctions you need, including ambiguous and out-of-scope inputs. Repeating a few polished answers is not a representative dataset.
Keep a separate evaluation set that the training process never sees. Compare the adapted model with the original on quality, latency, usage cost and maintenance effort. A changed taxonomy may require relabelling and another training cycle. Fine-tuning does not by itself create document permissions, current source citations or a live connection to business records.
A decision sequence for the warranty assistant
Start with a small collection of real questions and the evidence needed to answer them. Establish whether a straightforward application query, a search interface or a prompted model already solves the task. Introduce retrieval when relevant knowledge cannot fit reliably into a fixed instruction. Improve missing metadata and retrieval before changing the generation model.
Only then test whether a persistent behavioural failure justifies fine-tuning. If both evidence selection and response consistency are problems, combining methods can be reasonable, but evaluate each addition separately. Otherwise, a gain from better documents may be incorrectly attributed to training, leaving you with a costly component whose contribution nobody can explain.
Choose the intervention from the failure
A diagnostic aid, not a universal ranking of methods.
Scroll horizontally to read the table →
| Observed problem | First investigation | Possible intervention |
|---|---|---|
| Policy missing from context | Is it indexed, permitted and retrievable? | Retrieval and document lifecycle |
| Current record required | Which system owns the live value? | Authorised API lookup |
| Inconsistent classification | Are labels and instructions unambiguous? | Prompt baseline, then evaluate training |
| Forbidden information returned | Where is authorisation enforced? | Fix application access controls |
Compare the cost of change, not just the first demo
Write down what happens when a policy changes, a document is deleted, a category is renamed and a new language is introduced. RAG requires ingestion, indexing, retrieval and evaluation operations. Fine-tuning requires dataset preparation, training, version management and regression checks. Both still require an application that handles identity, errors and user feedback.
The best first investment is often a shared evaluation set with explicit expected evidence and failure reasons. It survives changes of vendor and architecture. You can then make a defensible choice based on your workload, rather than treating RAG and fine-tuning as competing labels or assuming one is always cheaper.
Sources & further reading
Documentation consulted on .
FAQ / DECISIONS
Frequently asked questions
Is RAG cheaper than fine-tuning?+
Not automatically. Compare ingestion, retrieval, model usage, evaluation, operations and change frequency against the fine-tuning alternative.