doxa-life/doxa-maps
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

documentation: which gates need a browser, and what exit 75 means

Browse source 
238 checks; 148 run anywhere, 87 need a real browser, 3 shell out. A newcomer
cannot tell from the name -- check:packet-render sounds like a file check and
drives a browser; check:composites-mount sounds like it needs one and does not.
The test is one grep for playwright/chromium, and the doc says so.

The part worth writing down is exit 75. run-parallel.mjs:262 defines three
outcomes, not two: 75 (EX_TEMPFAIL) is NOT MEASURED -- the gate could not look --
and a gate that exits 0 while announcing SKIP is also NOT MEASURED. Only another
non-zero is a FAIL. Two traps follow: exit 0 alone does not mean a gate looked at
anything, and your own `timeout` produces 124, which says you stopped it, not
that the code failed.

Also states the rule the split exists for -- one machine, one browser seat, so
one person runs them -- and why it is not just logistics: a source read and a
render are different measurements. Four catalogue gates can pass while a page
still paints English; and of 16 cards parked as "needs a browser", 7 were about
markup, a path, a manifest field or a gate, and settled headlessly.

Every number measured on main d8baaefd2 rather than estimated, including
correcting gates:pre from 13 to 15 after re-running it.
This commit is contained in:
Doxa Maps 2026-09-28 07:50:38 +00:00
parent 1a48e26a82
commit a02458a171
1 changed files with 105 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

105
documentation/HOWTO-which-gates-need-a-browser.md Normal file
View file

@ -0,0 +1,105 @@
# Which gates need a browser
There are **238** checks in this repo. **148 run anywhere. 87 need a real browser.** If you are not
the person who owns the browser seat, that second group is not yours to run — and knowing which is
which saves you from a whole class of wrong answer.
> **The short version.** A gate that reads *files* is headless. A gate that asks *what the page
> actually painted* needs a browser. Run the first kind freely; hand the second kind to whoever owns
> the browser.
---
## Telling them apart
Open the gate's file and look for one import:
```bash
grep -l 'playwright\|chromium' internal/check-*.mjs
```
If it imports `playwright` or launches `chromium`, it needs a browser. That is the whole test — the
name does not tell you. `check:packet-render` sounds like a file check and drives a browser;
`check:composites-mount` sounds like it needs one and does not (it mounts every composite through
Vite's SSR loader, no browser at all).
### Measured on `main` (2026-09-28)
| | Count |
|---|---:|
| headless — run these yourself | **148** |
| **browser — not yours unless you own the seat** | **87** |
| neither (shells out to `bun`/`php`) | 3 — `check:mobile`, `check:wp-embed`, `smoke:playground` |
The browser group is, broadly: everything that renders a map (`check:map-paints`,
`check:map-data-painted`, `check:auto-engine`, `check:maplibre-drops`), everything about layout
(`check:fits-screen`, `check:shell-scroll`, `check:filter-width`, `check:scrim-viewport`),
everything that prints (`check:poster-prints`, `check:print-geometry`, `check:packet-render`),
everything about right-to-left shaping (`check:rtl-shaping`, `check:embedded-rtl`), the English-leak
family (`check:english-leaks`, `check:leaf-english-leaks`, `check:host-locale-contract`), and every
`smoke:*`.
## Why the split exists, and why it is not negotiable
One machine, one browser seat. Two agents running Playwright at once fight over it, and the loser
does not fail — **it reports that it could not look**, which is a third outcome most people's mental
model does not have (see below). So the rule is: **one person runs the browser gates.** In this
repo that is the QA agent; ask them rather than starting a second browser.
This is not a limitation to work around. A source read and a render are *different measurements*,
and confusing them is the most common way to report something false:
- `check:leaf-english-leaks` can only find a leak by comparing what the catalogue says with **what
the page shows**. Four catalogue gates can all pass — keys exist, differ per locale, are reachable,
are in scope — while a page still paints English. Catalogue-green is not render-green.
- Equally, a headless check often settles a question people assume needs pixels. Of 16 cards parked
as "needs a browser", **7 turned out to be about markup, a path, a manifest field or a gate** —
all readable without one.
**Ask what property the claim is about**, then pick the tool. Not the other way round.
---
## ★ Exit 75 means NOT MEASURED. It is not a pass and it is not a failure.
This is the one that bites hardest, because it looks like success.
`internal/run-parallel.mjs:262` states the contract:
> THREE OUTCOMES, never two. Exit **75** (`EX_TEMPFAIL`) from ANY step is **NOT MEASURED**: the gate
> could not look (no browser seat, no slot, no built tree), which is neither a pass nor a failure. A
> gate that exits **0 while announcing SKIP** is also NOT MEASURED. Only any other non-zero exit is a
> FAIL.
Two traps follow from that, and both have caught people here:
1. **Exit 0 is not enough.** A gate can exit 0 and print `SOMETHING: SKIP` — it looked at nothing.
The runner marks that NOT MEASURED; a human reading only `$?` records a pass.
2. **Your own `timeout` produces exit 124, which is also not a failure.** Killing a gate at 180 s
because you were impatient tells you nothing about the code. If you must bound a run, bound it
generously and read exit 124 as "I stopped it", never as "it failed".
The runner prints all three: `✓` pass, `✗` fail, `⊘` not measured. **Read the symbol, not the tail of
the log** — a truncated log with a green last line looks exactly like a pass.
---
## What to do, depending on who you are
**Landing a change:** run `bun run gates:pre` (all headless, 15 checks, ~30 s on a fresh clone) plus
whichever headless gates cover what you touched. Do not run the browser group.
**You think a browser gate would catch your bug:** say so, name the gate, and say what result would
change your mind. A prediction someone else can falsify is worth more than a run you did yourself.
**You own the browser seat:** you are the only one. When a gate comes back NOT MEASURED, say
"not measured" — resist reporting it as either colour.
## A worked example of getting this wrong
`check:packets-all` reads generated packets from a gitignored scratch directory. It went red on 43
files left over from a run three days earlier, reporting *"0 prayer tick boxes"* about packets
generated before tick boxes existed. Nothing was broken. The gate was grading leftovers.
The fix was not to loosen it: it now skips any packet older than the renderer that produces it, and
says so. **A stale input is NOT MEASURED, not FAIL** — the same three-outcome rule, one level down.