How Flagridge caught its own rules not reaching the work

The approach

1. Read the logs, not the reports.

   A report said the rules existed. The logs showed whether a session had received them. Passing tests would not have shown the gap either, because tests cover the code, not whether the rules reach the work.

2. Write the pass marks before measuring.

   The specification of the fix set its pass marks in writing before anything was measured: one for whether the core rules reach every working session, others for how agents then use them. For each mark it also named the next step if it was missed, so a miss could not be explained away afterwards. The marks are not quoted here, because a mark quoted without its measurement would read as a result.

3. Expect the gap to return.

   Three times the method's description of itself differed from what ran, and each time a measurement caught it, not a report. In July the core described standing instructions that no session ever loaded. In October came the gap described above. The third time it appears to be the measuring script itself. Since mid-October one of the agents has handed back its results by a route the script does not read, so half of the measurement no longer describes what it was built to describe. The first two gaps were corrected. The third is not fixed yet, and the page says so rather than quietly dropping the affected half.

   July October third