run logs: find a run far back in the task list by bisection instead of an incomplete walk (CLS-1060) #303

Merged
stephen merged 1 commit from stephen/cls-1060-task-lookup into main 2026-11-01 02:02:45 +00:00
Owner

fj run logs finds a run's task ids by walking GET /actions/tasks newest first (Forgejo 16 ignores the run filter), up to 40 pages of 50. On a very busy repository, a run older than that read as an incomplete task list, so its jobs reported a retrieval error even though their logs exist.

  • After four pages without resolving a wanted task id, list_run_tasks now bisects the page numbers. The list is newest first, so a page's lowest id falls as the page number rises. It finds the first page holding a task at or below the newest still-unresolved id, and resumes the walk one page early (tasks added between reads push entries down a page). A run 10,000 pages back takes about 20 reads.
  • It can jump again after another four pages without progress, and always aims at the newest unresolved id, so tasks of one run separated by a rerun (a recent task and one thousands back) are each reached.
  • The jump only runs when there are wanted ids and a total_count. A run in the first pages never pays for a probe, the older-server path (no wanted ids) still walks to exhaustion, and a list without total_count still walks to the cap and reports incomplete.

Failure paths

  • Server clamps the page size: the last page is computed from the page length the server actually returned, not from 50.
  • Stale total_count (the list grew): a task is called absent only when the page after the computed last page is empty. Otherwise the walk carries on to the cap and reports the list incomplete, never a shorter answer.
  • A probe fails: the tasks every earlier read resolved (walk and probes) are kept and the list is marked incomplete with the failing page; a first-page failure is still an error.
  • Wanted task older than the whole list: complete, nothing found, in a few reads.
  • Retries and concurrency: reads are idempotent; duplicates across pages are dropped by task id.

Fix scope

  • Invariant: an incomplete or unproven read never certifies absence, and tasks already resolved are never discarded. Paths checked: list_run_tasks (both entry points: RemoteTasks for the Loki fallback and the older-server exhaustive walk) and the probe pages. Self-review (three rounds) found the assumed page size, the stale total, probes dropping the tasks they read, and a jump aimed at already-found ids; each has a test.
  • Tests (mocked /actions/tasks, counted requests): a run 10,000 pages back, the last page, a task older than the list, a failing probe, a clamped page size, a list longer than its total, tasks separated by a rerun, tasks further apart than the cap, a probe that read a task before a later probe failed, and a run near the top making exactly one read. cargo test --bin fj workflow_logs passes (55). cargo clippy -D warnings already fails on pr_line_comment.rs with this toolchain, untouched here; nothing new in this file.

Known edge cases (follow-up)

  • Tasks deleted between probe reads can still shift a page the wrong way and skip an entry; the existing walk has the same exposure.
`fj run logs` finds a run's task ids by walking `GET /actions/tasks` newest first (Forgejo 16 ignores the `run` filter), up to 40 pages of 50. On a very busy repository, a run older than that read as an incomplete task list, so its jobs reported a retrieval error even though their logs exist. - After four pages without resolving a wanted task id, `list_run_tasks` now bisects the page numbers. The list is newest first, so a page's lowest id falls as the page number rises. It finds the first page holding a task at or below the newest still-unresolved id, and resumes the walk one page early (tasks added between reads push entries down a page). A run 10,000 pages back takes about 20 reads. - It can jump again after another four pages without progress, and always aims at the newest unresolved id, so tasks of one run separated by a rerun (a recent task and one thousands back) are each reached. - The jump only runs when there are wanted ids and a `total_count`. A run in the first pages never pays for a probe, the older-server path (no wanted ids) still walks to exhaustion, and a list without `total_count` still walks to the cap and reports incomplete. Failure paths - Server clamps the page size: the last page is computed from the page length the server actually returned, not from 50. - Stale `total_count` (the list grew): a task is called absent only when the page after the computed last page is empty. Otherwise the walk carries on to the cap and reports the list incomplete, never a shorter answer. - A probe fails: the tasks every earlier read resolved (walk and probes) are kept and the list is marked incomplete with the failing page; a first-page failure is still an error. - Wanted task older than the whole list: complete, nothing found, in a few reads. - Retries and concurrency: reads are idempotent; duplicates across pages are dropped by task id. Fix scope - Invariant: an incomplete or unproven read never certifies absence, and tasks already resolved are never discarded. Paths checked: `list_run_tasks` (both entry points: `RemoteTasks` for the Loki fallback and the older-server exhaustive walk) and the probe pages. Self-review (three rounds) found the assumed page size, the stale total, probes dropping the tasks they read, and a jump aimed at already-found ids; each has a test. - Tests (mocked `/actions/tasks`, counted requests): a run 10,000 pages back, the last page, a task older than the list, a failing probe, a clamped page size, a list longer than its total, tasks separated by a rerun, tasks further apart than the cap, a probe that read a task before a later probe failed, and a run near the top making exactly one read. `cargo test --bin fj workflow_logs` passes (55). `cargo clippy -D warnings` already fails on `pr_line_comment.rs` with this toolchain, untouched here; nothing new in this file. Known edge cases (follow-up) - Tasks deleted between probe reads can still shift a page the wrong way and skip an entry; the existing walk has the same exposure.
run logs: find a run far back in the task list by bisection instead of an incomplete walk (CLS-1060)
All checks were successful
ci / check (pull_request) Successful in 12m38s
ci / coverage (pull_request) Successful in 2m43s
ci / live-e2e (pull_request) Successful in 4m13s
6d97929708
list_run_tasks walked /actions/tasks newest first and gave up after 40 pages, so a run more than 2000 tasks back on a busy repository read as an incomplete task list. After four pages it now bisects the page numbers (the list is newest first, so a page's lowest id falls as the page number rises) to the first page holding a task at or below the wanted ids, and resumes the walk one page early. A probe that fails is an incomplete list, wanted ids older than the whole list are absent, and a list without a total_count still walks to the cap.
gerri approved these changes 2026-11-01 01:44:42 +00:00
gerri left a comment

Automated Codex review (read-only) of head 6d97929708.

No new correctness, security, or failure-path problems found against STANDARDS.md. Checked probe retention, repeated jumps, completion decisions, and both RemoteTasks entry points.

The accepted deletion-between-probes edge case remains unchanged, as documented under “Known edge cases (follow-up).”

BLOCKING: 0

Reviewed head 6d97929708 at 2026-11-01T01:44:41Z by codex exec (read-only sandbox).

Automated Codex review (read-only) of head 6d97929708a4244e7f04fa34f10d851e61b5085e. No new correctness, security, or failure-path problems found against STANDARDS.md. Checked probe retention, repeated jumps, completion decisions, and both RemoteTasks entry points. The accepted deletion-between-probes edge case remains unchanged, as documented under “Known edge cases (follow-up).” BLOCKING: 0 --- Reviewed head 6d97929708a4244e7f04fa34f10d851e61b5085e at 2026-11-01T01:44:41Z by codex exec (read-only sandbox).
Sign in to join this conversation.
No description provided.