Reading Reasoning Traces

A chain-of-thought trace is a diagnostic artifact, not a narrative to read for understanding. When output quality is low, the reasoning trace tells you where the model went wrong and what caused it. Reading a trace productively requires knowing what failure signatures look like.

1. Step 1: Find the First Divergence Read forward through the reasoning until you find the first step that is wrong. This is the point of failure. Everything downstream of a wrong step is built on that error — fixing a downstream step without fixing the root error is treating a symptom.
2. Step 2: Classify the Error Is the wrong step a factual error (the model asserted something false)? A missed constraint (the model ignored a requirement)? A wrong inference (the logic is invalid)? Each error type has a different fix. See the failure taxonomy in Lesson 8.
3. Step 3: Check the Premise What did the model assume at step 1? Many reasoning failures originate in an implicit assumption the model made before the first visible step. If the premise is wrong, the entire reasoning chain is built on a false foundation — no amount of fixing downstream steps will correct it.
4. Step 4: Check Constraint Retention List the constraints stated in the prompt. Verify each one appears in the reasoning at the point where it is relevant. Models frequently acknowledge constraints at step 1 and then ignore them by step 4. Constraint dropout is a common and fixable failure.

Audit checklist for a CoT trace:

□ Step 1 premise: Is the initial problem framing correct?
□ Constraint inventory: Are all stated constraints represented in the reasoning?
□ First divergence: At which step does the reasoning produce an incorrect claim?
□ Error type: Factual / Missed constraint / Invalid inference / Hallucination / Loop / Premature conclusion
□ Downstream impact: How many steps are built on the error?
□ Root cause: Is this a prompt gap (missing information/constraint) or a model failure (wrong inference from correct information)?

Prompt fix hypothesis:
[ ] Add explicit constraint: ______
[ ] Add grounding information: ______
[ ] Add reasoning structure requirement: ______
[ ] Reduce scope to prevent over-reasoning: ______