Back to insights

Technology

Full-population testing works only when the rule comes first

A stream of paper slips passing through a defined gate, with red exceptions held for review

The snapshot

A data tool can read every line in a sales day book. That is useful. It is not, by itself, an audit procedure. The procedure is still the question, the population, the rule that flags an item, and the person who looks at what was flagged.

Speed is the reason teams buy the tool. Control is the reason the file can be reviewed later. They have to be set up together. A fast extract with no rule written down is a printout. A careful rule that nobody ran on the real file is a memo.

The trap

A team loads twelve months of payments and asks the tool for “anything unusual.” It returns two hundred rows. Someone scans the first screen, exports a PDF, and moves on. The population is not named. The threshold is not written. Next year a different person runs a different cut and gets a different list. Neither list is tied to a conclusion.

The file looks busy. It does not show what was tested.

This is an illustration, so the point can be said out loud. It is not a client matter.

Where judgement stays

Full-population work is worth it when the rule is simple enough to explain:

  • The population. Which system, which period, which accounts, and what was left out.
  • The rule. The test in plain words: duplicates above a stated amount, payments to new vendors in the first thirty days, journals posted after the close.
  • The result. The exception list, frozen, with a count of what was in and what was flagged.
  • The look. Who opened the exceptions that mattered, and what they did with them.

The tool can apply the rule to every row. It cannot decide that a flag is a misposting, a timing difference, or nothing. That decision is still a person’s, and it belongs on the file next to the list.

Sampling does not disappear. It moves. You may sample the exceptions, or sample the items the rule was never built to see. You do not sample because the tool was slow. You sample because judgement still has a job.

What to do

  • If you lead the audit: write the rule before the extract is praised. If the rule cannot be said in one sentence, it is not ready to run on the whole file.
  • If you prepare the working paper: keep the population, the rule, the frozen list, and the name of the person who cleared the items that were followed up.
  • If you run finance: a dashboard is not the books. When a tool proposes a journal, a person still posts it, against the source.
  • If you sit on the audit committee: ask which tests were run on the full population, and which exceptions were left without a look.