Fine-tune vs retrieval: what a 42-second fine-tune taught us about voice and facts

We trained a tiny AI model on 60 example replies for a made-up bakery. After 42 seconds of training it answered in the bakery’s exact voice. It also invented a phone number, a street address and a set of opening hours. That is the whole lesson of fine-tuning in one test: it teaches a model how to talk, not what is true.

Everything below comes from one test run on 12 October 2026: a 135-million-parameter model, a 60-row dataset we wrote ourselves, and a Mac CPU. It is a small demonstration, not a benchmark, and results will differ with your model, data and hardware.

The short version

Fine-tune a model when you want a consistent tone, format or sign-off. Give it retrieval (your documents, looked up at answer time) when it has to state facts that can change, like opening hours, prices and policies. You can combine the two.

What we did

• Model: SmolLM2-135M-Instruct, a very small open model (about 260 MB).
• Data: 60 rows of customer question plus the reply we wanted, for a fictional shop called Sunrise Bakery. Twelve topics, five phrasings each, and the same friendly voice and sign-off every time.
• Training: 4 passes over the data (epochs), LoRA adapters, 60 steps, no GPU. The trainer reported about 42 seconds. The loss fell from about 3.7 to under 1.
• Test: four questions the model had never seen, including one about the capital of France.

What came out

• The original model rambled about grammar and punctuation when asked what time the shop closes.
• The tuned model answered all four questions as the bakery, with the sign-off, even the France question.
• It also made things up: “We take orders at 24 hours”, a 1-800 phone number and “123 Maple Street”. None of that was in the data.

Why it invents facts

With 60 examples and a model this small, the quickest thing to learn is the pattern: greet, answer in one line, sign off. A language model always has to produce an answer-shaped sentence, so when it does not know the fact it fills in something plausible. A bigger model and more data help, but they do not turn fine-tuning into a fact database.

Fine-tune or retrieval?

• Fine-tune for: tone and voice, output format, a consistent sign-off, a narrow repeated task.
• Use retrieval for: opening hours, prices, policies, product details, anything that changes or must be exactly right.
• Both: retrieval supplies the facts, a light fine-tune supplies the style.

This matches published guidance. Google Cloud’s guide to tuning versus retrieval frames tuning as behaviour and retrieval as knowledge, and a 2023 comparison study (arXiv:2312.05934) found retrieval was the better way to add new facts to a model.

If you want to try it on Windows

We built TuneMyLLM, a Windows app that does the download, dataset and training steps with no code. It has a free plan with limits: 2 saved fine-tunes, 1,000 dataset rows per run and a 1,024-token training context. Premium removes those limits; see the Microsoft Store listing for current prices. Our demo was recorded on a Mac CPU because that is what we had on hand. On a Windows PC with an NVIDIA graphics card it is much faster and you can train larger models. For a rough sense of video memory, Unsloth’s requirements docs list around 5 GB for a 7-billion-parameter model with 4-bit training. Check your own card.

Watch the 18-second version on YouTube Shorts. The full beginner walkthrough goes public on our YouTube channel on 12 October at 17:30 IST. The app is on the Microsoft Store.

Limits of this test

• One tiny model, one made-up dataset, one run. Do not treat the 42 seconds as a promise.
• We did not measure whether a larger model would still invent facts. Many do.
• Always check every fact a tuned model states before you let it talk to customers.