Synthesis Quality Control

Synthesis quality has four dimensions. Completeness: does the synthesis address all the sources in the input set, or were some silently dropped? Fidelity: do the synthesized findings accurately represent the underlying source material, or has information been distorted in the merging process? Emergence: does the synthesis produce findings that were not present in any individual source, or is it merely a reorganized aggregation? Calibration: are the confidence levels appropriate given the evidence quality, or has the synthesis process artificially inflated or deflated confidence?

1. Completeness Check Compare the synthesis output against the source inventory. Every source should be represented in at least one finding. If a source contributed nothing to the synthesis, either the source was redundant (document why) or the synthesis missed its contribution (fix it).
2. Fidelity Spot-Check Select three findings at random and trace each one back to its supporting sources. Verify that the synthesis accurately represents what the sources said. Spot-checking catches distortions that systematic review might miss because it uses the same analytical frame that produced the distortion.
3. Emergence Validation Identify every finding that was not present in any single source. These emergent findings are the value-add of synthesis. If there are none, the synthesis operation did not produce new intelligence. If there are several, validate each one — emergent findings are also where synthesis errors are most likely to occur.