When an AI assistant gives poor answers, training the model is not always the right fix. First identify whether it lacks the right information or struggles to use the information it already has.
RAG supplies information
Retrieval-augmented generation, usually shortened to RAG, means finding relevant content and giving it to the model before it answers. Think of an assistant consulting an approved handbook rather than answering entirely from memory.
This is useful for company policies, product documentation, and other information that changes. You can update the source material without retraining the model. However, retrieval does not guarantee correctness: the system still needs to find the right passages and use them accurately.
Fine-tuning changes learned behaviour
Fine-tuning means giving a model additional training examples to improve how it performs a task. It can help with specialised response patterns or with using evidence more consistently. Research has also shown that fine-tuning and retrieval can work together, rather than being competing choices.
The practical distinction is simple: use retrieval to supply relevant knowledge, and investigate fine-tuning when repeated task failures remain despite good information.
| What goes wrong | What to investigate first |
|---|---|
| The assistant quotes an outdated policy | Source freshness and retrieval |
| It cannot find an existing answer | Document organisation and search |
| It receives the correct passage but mishandles it | Instructions, examples, then fine-tuning |
| It needs a customer’s current account status | A permission-controlled connection to the account system |
These are starting points for diagnosis, not guarantees that one technique will solve every case.
Your decision plan:
- 1Collect examples of failed answers.
- 2Check whether the required information reached the model.
- 3Fix missing or poor retrieval before adding training.
- 4Test clearer instructions and examples.
- 5Consider fine-tuning if a consistent behaviour problem remains.
Remember: do not use model training to compensate for a broken information pipeline.
Related: RAG and enterprise knowledge search