Back to insights

Technology

A model is not an audit trail

Four paper records connected by a red thread beside an opaque graphite cube

The snapshot

Auditors already sample. A file can live with an answer that is not certain, as long as a reviewer can see what was tested, how it was tested, what came back, and who accepted it.

A model can sit inside that kind of work. The weights cannot be the file. Ask the model later why it flagged an item and it writes a new account of the decision. It does not retrieve the run. That account is not a workpaper.

The public conversation is about reinvention. This note is about the record.

The trap

Picture a working paper that says: “Invoice 804 looks like a duplicate because the amounts are close.” Three weeks later a reviewer asks the model again and gets a different reason. The invoice is not on the file. Nobody signed the exception.

The paragraph sounded like evidence. It was a fresh explanation, written at the moment someone asked. The invoice, the ledger line, and the person who agreed they were the same cost were never kept.

This is an illustration, so the point can be said out loud. It is not a client matter.

What the record is

Four things, kept outside the model, make the run reviewable:

  • What went in. The documents and the question the tool was given.
  • How it was run. Which tool, and enough of the setup that another person can see the procedure.
  • What came out. The result, frozen when it was produced.
  • Who accepted it. A person, not the tool.

If those four are on the file, there is a trail. If the file only says “we used AI,” there is not. A sentence that names the tool does not show the run.

The same rule applies to the records a Uganda file already has to hold: the e-invoice, the ledger, the payroll, the bank line. A summary of them does not replace them. Sampling stays valid because the sample that was drawn is kept. A model stays valid the same way. The run that happened is kept.

What to do

  • If you sign the file: the tool can be on it when the four things are kept. A later explanation from the tool cannot be the evidence.
  • If you run the finance system: the chat answer is not the books. Keep what the tool read, and who approved anything it proposed for posting.
  • If you prepare the working paper: do not paste the tool’s “why.” Keep the source, the result, and your sign-off.
  • If you sit on the board: ask whether this decision can be opened in three weeks without asking the tool again.