Baseline Collection

You cannot measure improvement without a baseline. The baseline that is collected after the pilot starts is not a baseline — it is a rationalization. I have seen this more times than I can count: pilot results are reported without a pre-pilot baseline, or the baseline is estimated retrospectively based on people's recollection of how long things used to take. These numbers are useless and every analytically capable stakeholder knows they are useless. Collect the baseline before the pilot starts. Collect it systematically. Document how it was collected so the measurement methodology is defensible.

1. Identify Baseline Metrics For every primary pilot metric, identify what the pre-pilot state is and how to measure it objectively. Task completion time: pull the last 4-8 weeks of system timestamps. Error rate: pull the last 4-8 weeks of quality review records. Throughput: pull the last 4-8 weeks of volume data. If the metric cannot be measured objectively in the pre-pilot state, it cannot be measured credibly in the post-pilot state — choose a different metric.
2. Collect for Long Enough to Be Representative A one-week baseline will be distorted by any unusual conditions in that specific week — a holiday, a system outage, an unusually high-volume period. Collect baseline data for a minimum of four weeks. Eight weeks is better. The baseline should represent normal operating conditions, not any particular week's anomalies.
3. Document the Baseline Methodology Record how the baseline data was collected, what time period it covers, and what exclusions were applied (if any). When you present pilot results to skeptics, the baseline methodology will be scrutinized. A documented, defensible methodology converts a skeptic's first question ("how did you measure the baseline?") into a confirmation of rigor rather than a challenge to the data.