The Reasoning Audit

When a chain-of-thought output is wrong, the reasoning trace tells you where it went wrong. This is the primary diagnostic value of visible reasoning: not just that the output is wrong, but which step produced the error. A reasoning audit examines the chain step by step, identifies the first step that contains an error or unsupported inference, and uses that diagnosis to fix either the prompt or the input. Without the reasoning trace, you are guessing at the cause.

1. 1. Obtain the reasoning trace If the prompt did not request visible reasoning, re-run it with the instruction to show reasoning. You need the intermediate steps to audit, not just the conclusion.
2. 2. Verify each step independently Read each step in the reasoning chain and assess: is this step logically valid given the previous steps and the source material? Mark the first step that contains an error, an unsupported inference, or a false premise.
3. 3. Classify the error type Is the error a false premise (wrong starting assumption)? A logical non-sequitur (invalid inference from valid premises)? An extrapolation (going beyond the available data)? A hallucination (introducing facts not in the source)? Each error type has a different fix.
4. 4. Fix the prompt, not just the output If step three in the reasoning chain consistently goes wrong, the prompt is not constraining that step correctly. Add an explicit constraint or instruction for that step. If the model consistently introduces a false premise, the prompt is not grounding the starting assumptions sufficiently.