Give the strict coverage floor room for the surface it measures #265

Merged

stephen merged 1 commit from fix/coverage-floor-headroom into main

2026-09-06 20:22:25 +00:00

stephen commented

2026-09-06 20:07:36 +00:00

Owner

The coverage gate is set flush against a number that moves on its own, so it fails on measurement
drift instead of on regressions. It did exactly that last week, and it cost a round of debugging to
establish that nothing had actually regressed.

The sequence:

#255      floor set to 61, against 61.11% observed on run 483   ->  0.11 pt headroom
run 504   61.08%  green
#262      merges, adds instrumented lines to api/pull_core.rs
run 505   60.98%  FAILS

No file lost covered lines between runs 504 and 505. The denominator grew. That is fj#249, which is
open because the runner measures a wider and less stable strict surface than the calibrated local one.

#264 has since brought the number back to 61.36% on run 509 by covering the edit_text and
run_git_config glue, so main is green again. That is the right fix for the coverage itself, but it
leaves 0.36 points of headroom against a surface that took about 0.10 points from each of the last two
merges. Three or four ordinary merges and the same false failure recurs, on someone else's PR.

What changed

COV_MIN 61 -> 60 in the coverage job, about a point under the observed number rather than flush
against it.
The step's echoed rationale now records run 509's surface and, more importantly, says why the gap
is there, so the floor does not get re-set flush against the measurement next time.
CLAUDE.md carries the same note next to the make coverage-strict description.

A genuine regression is a point or more of covered lines and still fails the build. What stops failing
the build is instrumentation noise.

Notes

This is a deliberate loosening and it should not be permanent. When fj#249 closes and the CI surface
matches the calibrated local one, the floor goes back to tracking real coverage and can be raised on
purpose rather than pinned to whatever the runner last reported.

The `coverage` gate is set flush against a number that moves on its own, so it fails on measurement drift instead of on regressions. It did exactly that last week, and it cost a round of debugging to establish that nothing had actually regressed. The sequence: ``` #255 floor set to 61, against 61.11% observed on run 483 -> 0.11 pt headroom run 504 61.08% green #262 merges, adds instrumented lines to api/pull_core.rs run 505 60.98% FAILS ``` No file lost covered lines between runs 504 and 505. The denominator grew. That is fj#249, which is open because the runner measures a wider and less stable strict surface than the calibrated local one. #264 has since brought the number back to 61.36% on run 509 by covering the `edit_text` and `run_git_config` glue, so `main` is green again. That is the right fix for the coverage itself, but it leaves 0.36 points of headroom against a surface that took about 0.10 points from each of the last two merges. Three or four ordinary merges and the same false failure recurs, on someone else's PR. ## What changed - `COV_MIN` 61 -> 60 in the `coverage` job, about a point under the observed number rather than flush against it. - The step's echoed rationale now records run 509's surface and, more importantly, says *why* the gap is there, so the floor does not get re-set flush against the measurement next time. - `CLAUDE.md` carries the same note next to the `make coverage-strict` description. A genuine regression is a point or more of covered lines and still fails the build. What stops failing the build is instrumentation noise. ## Notes This is a deliberate loosening and it should not be permanent. When fj#249 closes and the CI surface matches the calibrated local one, the floor goes back to tracking real coverage and can be raised on purpose rather than pinned to whatever the runner last reported.

stephen added 1 commit

2026-09-06 20:07:36 +00:00

Give the strict coverage floor room for the surface it measures

ci / check (pull_request) Successful in 11m23s

Details

ci / live-e2e (pull_request) Successful in 2m10s

Details

ci / coverage (pull_request) Successful in 2m20s

Details

a31187d1bf

The `coverage` gate is set flush against a number that moves on its own, so it
fails on measurement drift instead of on regressions. It did exactly that last
week and cost a round of debugging to establish that nothing had regressed.

The sequence: #255 set the floor to 61 against 61.11% observed on run 483, so
0.11 points of headroom. Run 504 was green at 61.08%. #262 merged, adding
instrumented lines to `api/pull_core.rs`, and run 505 came in at 60.98% and
failed. No file lost covered lines between those two runs; the denominator
grew. That is fj#249, which is open because the runner measures a wider and
less stable strict surface than the calibrated local one.

#264 has since bought the number back to 61.36% on run 509 by covering the
`edit_text` and `run_git_config` glue, so `main` is green again. But that is
0.36 points of headroom against a surface that took 0.10 points from the last
two merges apiece, which is three or four ordinary merges before the same
false failure recurs.

Move the floor to 60, about a point under the observed number, and put the
reasoning in the step so it does not get re-set flush against the measurement
next time. A genuine regression is a point or more of covered lines and still
fails the build. The floor goes back to tracking real coverage, and can be
raised deliberately, when fj#249 closes.

stephen merged commit f7fdce40fc into main

2026-09-06 20:22:25 +00:00

stephen deleted branch fix/coverage-floor-headroom

2026-09-06 20:22:25 +00:00

stephen referenced this pull request from a commit

2026-09-06 20:22:27 +00:00

Give the strict coverage floor room for the surface it measures (#265)

stephen referenced this pull request

2026-09-06 22:06:06 +00:00

Record sixteen of the twenty-one undocumented changes in [Unreleased] #266

stephen referenced this pull request from a commit

2026-09-07 02:07:12 +00:00