Let the code do the maths and the model do the words

LLMs are good at turning structured information into readable prose and unreliable as the source of that information. When a report or an explanation contains numbers, a wrong number looks exactly as confident as a right one.

The pattern: a facts packet

• Code does every calculation: totals, quantities, dates, thresholds.
• It builds a compact facts packet: only the verified values the text is allowed to mention.
• The model explains using only those facts.
• Code checks the result before it goes anywhere.

Check the output, not just the input

Good inputs aren't enough; a model can still invent a figure. In Opspilot, every number in an LLM-written explanation is checked by code against the numbers the code produced. On the first real run, 33 of 34 passed. The one that was rejected had invented an amount, so the system showed the code's own wording instead.

The same idea elsewhere

• Reading messy text. Opspilot's WhatsApp order reader matches products by code first and the model second, and anything doubtful goes to review. It never guesses.
• Answering from documents. My SOP Assistant answers from retrieved context only, cites the pages it used, and refuses when the context doesn't cover the question.
• Reports. Pipelines where the charts and figures come from pandas and the model only writes the narrative.

Why local models make the checking more important

A local model (via Ollama) keeps business data on the machine, which matters for real documents. Smaller models are cheaper to run and less reliable, which is exactly why 'verify in code' beats 'trust the model'. I measured it with an answer key: on 16 sample WhatsApp orders, qwen2.5:14b got 15 fully right and 44 of 45 lines exact, with zero wrong products or quantities. qwen2.5:7b got 14 of 16 and also 44 of 45 with zero wrong, at twice the speed. The lines it wasn't sure about went to review.

Rules of thumb

• Numbers come from code, never from the model.
• Give the model only the facts it's allowed to use.
• Verify the output against those facts.
• Have a plain fallback for when verification fails.
• Measure with an answer key, not vibes.