If you want an AI assistant that knows your business, two terms come up early: RAG and fine-tuning. They solve different problems, and choosing the wrong one is a common reason AI projects stall after the first demo.
The short version
RAG gives the model the right information at the moment it answers. Fine-tuning changes how the model behaves by training it further on examples.
Think of a new employee. RAG is handing them the company handbook and letting them look things up. Fine-tuning is training them until a particular way of working becomes habit.
How RAG works
RAG stands for retrieval-augmented generation. When someone asks a question, the system:
- Searches your content, such as help articles, policies, product specs or past tickets, for the passages most likely to hold the answer.
- Passes those passages to the model along with the question.
- Lets the model write an answer from what it was given, and point to the source.
Your content usually sits in a search index, often a vector database, which finds passages by meaning rather than exact wording.
RAG is a good fit when:
- Answers need to reflect your latest information.
- Your content changes often.
- People need to see where an answer came from.
- Different users are allowed to see different documents.
What to watch: the answer can only be as good as the search. If the right passage isn't found, the model can't use it. In practice, most RAG problems are search problems, not model problems. Cleaning up documents, splitting them sensibly and testing with real questions matter more than most teams expect.
How fine-tuning works
Fine-tuning takes an existing model and trains it further on a set of examples, usually pairs of inputs and the outputs you want. The model picks up patterns: a format, a tone, a way of sorting things, a narrow task done the same way every time.
Fine-tuning is a good fit when:
- You need the same output structure every time, such as a summary with fixed sections.
- You need a specific tone or writing style.
- The task is narrow and repeated, such as sorting support tickets into your own categories.
- You want a smaller, cheaper model to handle a task nearly as well as a larger one.
What to watch: fine-tuning is not a reliable way to teach a model facts. It may still mix up or invent details, and whenever your information changes you have to train again. You also need good examples, often hundreds or more, and someone has to prepare and check them.
Side by side
| RAG | Fine-tuning | |
|---|---|---|
| Keeping knowledge up to date | Update the documents | Train again |
| Showing sources | Built in | Not built in |
| Controlling format and tone | Some, through instructions | Strong |
| What you need to start | Your existing documents | Prepared training examples |
| Usual place in a project | First step | Later, if there's a clear gap |
A simple way to decide
Work through these in order:
- Does the answer depend on facts in your documents or data? Start with RAG.
- Do those facts change often? RAG again.
- Is the real problem the shape of the output rather than the knowledge? Improve the instructions first. If that isn't enough, consider fine-tuning.
- Do you need lower cost or faster responses on a narrow task at high volume? Fine-tuning a smaller model can help.
Many production systems end up using both: RAG for knowledge, and a light fine-tune for format or tone. Most teams should still start with RAG and well-written instructions, measure the results, and fine-tune only when they can point to a specific gap.
If your assistant also needs to take actions, such as updating a record or issuing a refund, that's a step beyond both. See how we approach AI agents with human approval.
Before you build either one
- Collect real questions your users ask, along with good answers. This becomes your test set, and it's the most useful thing you can prepare.
- Decide what "good" means for you. Correct? Cites a source? Says "I don't know" when it should?
- Check where your data lives and who is allowed to see what. If your documents are spread across systems, a little data engineering up front saves a lot of trouble later.
How Placix can help
Placix builds generative AI solutions on your own data, and we usually start the way this article suggests: with your real questions, a clear definition of a good answer, and the simplest approach that works. From there we take it from pilot to production, with evaluation, access controls and monitoring built in.
- Not sure where AI fits yet? Start with an AI strategy and readiness review to find the use case with the best return.
- Want to build it in-house? We can also find the engineers to do it.


