PM-101 · Module 3

Structured Improvement

3 min read

Prompt iteration is an engineering discipline. The same rules that apply to any other specification-and-test cycle apply here: change one variable at a time, record what changed and why, test against the same input, compare against a baseline. Random iteration — rewriting everything and hoping the output improves — produces accidental results that cannot be reproduced or explained. Controlled iteration produces knowledge.

  1. 1. Establish a baseline Before iterating, save the current prompt and its output. This is your baseline. Every subsequent version is measured against it. Without a baseline, you cannot tell whether a change improved things — you can only guess.
  2. 2. Diagnose the failure Using the failure taxonomy from Lesson 7, identify exactly what is wrong. Which component is missing or insufficient? Assign the failure to one category before touching the prompt.
  3. 3. Change one component Add or modify the single component that addresses the identified failure. If the output had wrong format, add a format specification. Do not also rephrase the task statement or adjust the role. One variable.
  4. 4. Test against the same input Run the modified prompt against the exact same input as the baseline. Changing the input at the same time as the prompt means you do not know which change produced the result.
  5. 5. Compare and document Did the output improve on the targeted dimension? Did anything regress? Document what you changed, why, and what the effect was. After three cycles, you have institutional knowledge about what that prompt needs.

Do This

  • Diagnose the failure mode before changing anything
  • Change one prompt component per iteration
  • Keep a version log: what changed, why, and what the result was
  • Test against consistent inputs so results are comparable

Avoid This

  • Do not rewrite the entire prompt when one component fails
  • Do not change inputs and the prompt in the same test
  • Do not iterate without documenting — undocumented changes cannot be understood or rolled back
  • Do not rely on subjective judgment alone — define what "better" means before you test