Every check is run by a chain, declared on demand with a reason, or listed for its owner
check:no-orphan-gates only REPORTED scripts outside the chains, and the list grew to 87 main checks.
That day two of them hid real defects: check:i18n-never-translated, while a dead key identical in all
11 catalogues sat on development for hours, and check:auto-engine, which asserted a retired engine
rule that nothing ran. The report also matched by substring, so check:hardcoded-english counted as
run because its :self-test is in gates:pre.
The arm now decides. Every check:/verify:/smoke: script must be:
reached by exact name from gates:pre, gates:post, the browser suite's GATES or the build,
following what those scripts `bun run`;
on demand in internal/fixtures/on-demand-checks.json WITH its reason (check:settled, whose
reason this gate already documented: it fails inside any build by design);
baselined in internal/fixtures/orphan-checks-baseline.json, today's 131 orphans, for their
owners to chain, declare or delete. That list only shrinks (--shrink).
A new unchained script fails by name. A self-test runs first on every invocation. Proven two-sided on
the real tree: a planted unchained script exits 1 naming it; swapping exact matching for the old
substring test makes the self-test refuse; restored, it passes.
This commit is contained in:
parent
eb3f48ec14
commit
afc3e9edda
4 changed files with 250 additions and 13 deletions
|
|
@ -28,11 +28,17 @@
|
|||
* again. The tag is cheap to add and makes the author state which kind of thing they wrote.
|
||||
*
|
||||
* WHAT IT ASSERTS: every internal/*.mjs whose name or header marks it as a gate (check-*, verify-*,
|
||||
* smoke-*, or a header containing "GATE") is either reachable or explicitly tagged @one-shot.
|
||||
* smoke-*, or a header containing "GATE") is either reachable or explicitly tagged @one-shot; and every
|
||||
* check:/verify:/smoke: SCRIPT is run by a standing chain (by exact name), declared on demand with a
|
||||
* reason, or in the shrink-only baseline of 2026-09-29's orphans (the second arm, below).
|
||||
*
|
||||
* node internal/check-no-orphan-gates.mjs check (the chain arm's self-test runs first, every time)
|
||||
* node internal/check-no-orphan-gates.mjs --shrink drop baselined scripts since chained, declared or deleted
|
||||
* node internal/check-no-orphan-gates.mjs --init write the orphan baseline, only if none exists
|
||||
*
|
||||
* NEGATIVE-TESTED: removing the check:shell-scroll entry from package.json makes this fail by name.
|
||||
*/
|
||||
import { readFileSync, readdirSync } from 'node:fs'
|
||||
import { readFileSync, readdirSync, writeFileSync } from 'node:fs'
|
||||
import { resolve, dirname, join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { execSync } from 'node:child_process'
|
||||
|
|
@ -126,18 +132,106 @@ if (orphans.length) {
|
|||
// mid-build". What it must not do is
|
||||
// stay silent — a gate that could run and does not is indistinguishable from one
|
||||
// that is broken, which is the exact failure verify-shell-scroll had.
|
||||
//
|
||||
// ★ SO IT IS A DECISION NOW, NOT A REPORT (2026-09-29). Reported, the list only grew: 87 check/verify/
|
||||
// smoke scripts sat outside every chain, and that day two of them hid real defects. check:i18n-never-
|
||||
// translated was unchained while a dead key, identical in all 11 catalogues, stayed on development for
|
||||
// hours; check:auto-engine asserted a retired engine rule and nothing ran it. Every check:/verify:/smoke:
|
||||
// script must now be one of:
|
||||
// REACHED run by a standing chain: gates:pre, gates:post, the browser suite's GATES, the build, or
|
||||
// anything those `bun run` in turn;
|
||||
// ON DEMAND named in internal/fixtures/on-demand-checks.json WITH its reason (check:settled's is above);
|
||||
// BASELINED in internal/fixtures/orphan-checks-baseline.json, the orphans found that day, for their
|
||||
// owners to chain, declare or delete. That list only shrinks (--shrink).
|
||||
// A new script that is none of these FAILS, so "not chained" is something someone decided and wrote down.
|
||||
//
|
||||
// ⚠ REACHED MEANS BY EXACT NAME. This arm's first form, and the census that prompted the change, tested
|
||||
// `waves.includes(name)`, a substring: check:hardcoded-english read as run because
|
||||
// check:hardcoded-english:self-test is in gates:pre. A gate's self-test is not the gate.
|
||||
const ON_DEMAND_REL = 'internal/fixtures/on-demand-checks.json'
|
||||
const ORPHANS_REL = 'internal/fixtures/orphan-checks-baseline.json'
|
||||
const IS_CHECK = /^(check|verify|smoke):/
|
||||
|
||||
/** Script names a command runs: `bun run <name>` (or npm/pnpm/yarn), and run-parallel's list after ` -- `. */
|
||||
function namesRunBy(cmd, scripts) {
|
||||
const out = new Set([...String(cmd).matchAll(/\b(?:bun|npm|pnpm|yarn)\s+run\s+([^\s;&|)"']+)/g)].map((m) => m[1]))
|
||||
if (/run-parallel/.test(cmd) && cmd.includes(' -- ')) for (const t of cmd.split(' -- ')[1].split(/\s+/)) out.add(t)
|
||||
return [...out].filter((n) => Object.prototype.hasOwnProperty.call(scripts, n))
|
||||
}
|
||||
|
||||
/** The verdict, pure so the self-test can drive it: which checks are unreached and undeclared. */
|
||||
export function unchainedChecks({ scripts, suite, onDemand = {}, baseline = [] }) {
|
||||
const roots = ['gates:pre', 'gates:post', ...suite, ...Object.keys(scripts).filter((k) => /^(pre|post)?build(:|$)/.test(k))]
|
||||
const reached = new Set()
|
||||
const queue = roots.filter((r) => Object.prototype.hasOwnProperty.call(scripts, r))
|
||||
for (const r of queue) reached.add(r)
|
||||
while (queue.length) for (const n of namesRunBy(scripts[queue.pop()], scripts)) if (!reached.has(n)) { reached.add(n); queue.push(n) }
|
||||
const checks = Object.keys(scripts).filter((k) => IS_CHECK.test(k))
|
||||
const base = new Set(baseline)
|
||||
const unreached = checks.filter((k) => !reached.has(k))
|
||||
return {
|
||||
checks, unreached,
|
||||
fresh: unreached.filter((k) => !(k in onDemand) && !base.has(k)),
|
||||
// An on-demand entry with no reason, or naming a script that no longer exists, is not a decision.
|
||||
badDeclared: Object.entries(onDemand).filter(([k, why]) => !String(why || '').trim() || !(k in scripts)).map(([k]) => k),
|
||||
// Baselined entries since chained, declared or deleted: fine, and --shrink tidies them.
|
||||
gone: [...base].filter((k) => reached.has(k) || k in onDemand || !(k in scripts)),
|
||||
}
|
||||
}
|
||||
|
||||
// ── self-test, on every run: each rule proven able to fail on a planted case ──
|
||||
{
|
||||
const S = (extra = {}) => ({ 'gates:pre': 'node internal/run-parallel.mjs --label=pre -- check:a check:b:self-test',
|
||||
'gates:post': 'node internal/run-parallel.mjs --label=post -- check:agg', 'check:a': 'node a', 'check:b': 'node b',
|
||||
'check:b:self-test': 'node b --self-test', 'check:agg': 'bun run check:c && bun run check:d', 'check:c': 'node c',
|
||||
'check:d': 'node d', ...extra })
|
||||
const cases = [
|
||||
['a chained tree passes', { scripts: { ...S(), 'check:b': 'node b' }, suite: [], baseline: ['check:b'] }, 0],
|
||||
['a new check in no chain fails', { scripts: S({ 'check:new': 'node n' }), suite: [], baseline: ['check:b'] }, 1],
|
||||
['the same check in a chain passes', { scripts: S({ 'check:new': 'node n', 'gates:post': 'node internal/run-parallel.mjs --label=post -- check:agg check:new' }), suite: [], baseline: ['check:b'] }, 0],
|
||||
['a gate chained only through its :self-test fails', { scripts: S(), suite: [], baseline: [] }, 1],
|
||||
['a check the browser suite lists passes', { scripts: S({ 'check:e': 'node e' }), suite: ['check:e'], baseline: ['check:b'] }, 0],
|
||||
['a declared on-demand check passes', { scripts: S({ 'check:f': 'node f' }), suite: [], onDemand: { 'check:f': 'needs a device' }, baseline: ['check:b'] }, 0],
|
||||
]
|
||||
let ok = true
|
||||
for (const [name, input, want] of cases) {
|
||||
const got = unchainedChecks(input).fresh.length
|
||||
if ((want === 0) !== (got === 0)) { ok = false; console.error(` ✗ self-test: ${name} -> ${got} unchained`) }
|
||||
}
|
||||
if (unchainedChecks({ scripts: S({ 'check:f': 'node f' }), suite: [], onDemand: { 'check:f': ' ' }, baseline: ['check:b'] }).badDeclared.length !== 1) {
|
||||
ok = false; console.error(' ✗ self-test: an on-demand entry with no reason was accepted')
|
||||
}
|
||||
if (!ok) { console.error('NO-ORPHAN-GATES: SELF-TEST FAILED — the chain arm cannot be trusted'); process.exit(1) }
|
||||
}
|
||||
|
||||
const pkg = JSON.parse(pkgRaw)
|
||||
const waves = `${pkg.scripts['gates:pre'] || ''} ${pkg.scripts['gates:post'] || ''}`
|
||||
const checks = Object.keys(pkg.scripts).filter((k) => /^(check|verify|smoke):/.test(k)
|
||||
&& !k.endsWith(':self-test'))
|
||||
const unwired = checks.filter((k) => !waves.includes(k))
|
||||
console.log(` wave membership : ${checks.length - unwired.length}/${checks.length} `
|
||||
+ `check/verify script(s) run in gates:pre or gates:post`)
|
||||
if (unwired.length) {
|
||||
console.log(` ${unwired.length} run only by hand — expected for gates needing a served tree, a`)
|
||||
console.log(' device, a token or an argument. If one of these needs no precondition, wire it:')
|
||||
for (const k of unwired.slice(0, 8)) console.log(` · ${k}`)
|
||||
if (unwired.length > 8) console.log(` …and ${unwired.length - 8} more`)
|
||||
const suiteSrc = readFileSync(join(ROOT, 'internal/verify-browser-all.mjs'), 'utf8')
|
||||
const suite = [...(suiteSrc.split('const GATES = [')[1] || '').split('\n]')[0].matchAll(/script:\s*'([^']+)'/g)].map((m) => m[1])
|
||||
const readJson = (rel, d) => { try { return JSON.parse(readFileSync(join(ROOT, rel), 'utf8')) } catch { return d } }
|
||||
const onDemand = readJson(ON_DEMAND_REL, { checks: {} }).checks || {}
|
||||
const orphanBase = readJson(ORPHANS_REL, null)
|
||||
if (process.argv.includes('--init') || process.argv.includes('--shrink')) {
|
||||
const v = unchainedChecks({ scripts: pkg.scripts, suite, onDemand, baseline: orphanBase?.unreached || [] })
|
||||
const init = process.argv.includes('--init')
|
||||
if (init && orphanBase) { console.error(`NO-ORPHAN-GATES: --init refused, ${ORPHANS_REL} exists (use --shrink)`); process.exit(1) }
|
||||
const keep = init ? v.unreached.filter((k) => !(k in onDemand)) : orphanBase.unreached.filter((k) => !v.gone.includes(k))
|
||||
writeFileSync(join(ROOT, ORPHANS_REL), JSON.stringify({ about: 'check/verify/smoke scripts that no standing chain ran on 2026-09-29 and that nobody had declared on-demand. Each owner chains it, declares it in on-demand-checks.json with a reason, or deletes it. This list only shrinks (--shrink).', unreached: keep.sort() }, null, 1) + '\n')
|
||||
console.log(`NO-ORPHAN-GATES: orphan baseline ${init ? 'written' : 'shrunk'}: ${keep.length} script(s)`)
|
||||
process.exit(0)
|
||||
}
|
||||
const verdict = unchainedChecks({ scripts: pkg.scripts, suite, onDemand, baseline: orphanBase?.unreached || [] })
|
||||
console.log(` chains : ${verdict.checks.length - verdict.unreached.length}/${verdict.checks.length} check/verify/smoke `
|
||||
+ `script(s) reached by exact name; ${Object.keys(onDemand).length} declared on demand; `
|
||||
+ `${(orphanBase?.unreached || []).length - verdict.gone.length} baselined for their owners`
|
||||
+ (verdict.gone.length ? ` (${verdict.gone.length} since settled; tidy with --shrink)` : ''))
|
||||
if (verdict.fresh.length || verdict.badDeclared.length) {
|
||||
console.error(`NO-ORPHAN-GATES: FAIL — ${verdict.fresh.length} script(s) that no chain runs and nobody declared`
|
||||
+ (verdict.badDeclared.length ? `, ${verdict.badDeclared.length} on-demand entr(y/ies) with no reason or no script` : ''))
|
||||
for (const k of verdict.fresh) console.error(` ✗ ${k}`)
|
||||
for (const k of verdict.badDeclared) console.error(` ✗ ${k} (in ${ON_DEMAND_REL})`)
|
||||
console.error(' Chain it (gates:pre / gates:post, or the browser suite once it has run), or declare it in')
|
||||
console.error(` ${ON_DEMAND_REL} with the reason it must run by hand.`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
console.log(`NO-ORPHAN-GATES: PASS — ${checked} gate-shaped file(s); all reachable (${tagged} tagged @one-shot).`)
|
||||
|
|
|
|||
6
internal/fixtures/on-demand-checks.json
Normal file
6
internal/fixtures/on-demand-checks.json
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
{
|
||||
"about": "check/verify/smoke scripts that must run BY HAND, each with the reason no chain runs it. Being listed here is a decision someone wrote down; check:no-orphan-gates fails on an entry with no reason or no script.",
|
||||
"checks": {
|
||||
"check:settled": "asserts the tree is QUIET (no builder running, no recent churn), so it fails inside any build by design: it passed twice standalone, was chained on that evidence, and failed the very next build (2026-09-24)"
|
||||
}
|
||||
}
|
||||
136
internal/fixtures/orphan-checks-baseline.json
Normal file
136
internal/fixtures/orphan-checks-baseline.json
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
{
|
||||
"about": "check/verify/smoke scripts that no standing chain ran on 2026-09-29 and that nobody had declared on-demand. Each owner chains it, declares it in on-demand-checks.json with a reason, or deletes it. This list only shrinks (--shrink).",
|
||||
"unreached": [
|
||||
"check:app-start-page",
|
||||
"check:basemap-labels",
|
||||
"check:card-deck",
|
||||
"check:catalog-currency",
|
||||
"check:click-sheet",
|
||||
"check:composed-filters",
|
||||
"check:composite",
|
||||
"check:coverage-consistent",
|
||||
"check:coverage-consistent:poison",
|
||||
"check:csv-shape:self-test",
|
||||
"check:dashboard-filter-a11y",
|
||||
"check:deck-click",
|
||||
"check:dev-assets",
|
||||
"check:doc-scripts:self-test",
|
||||
"check:dxlife",
|
||||
"check:element-census-vs-live",
|
||||
"check:embed-base",
|
||||
"check:embed-contract",
|
||||
"check:embed-contract:poison",
|
||||
"check:embed-targets:self-test",
|
||||
"check:embedded-rtl",
|
||||
"check:engine-contract:poison",
|
||||
"check:english-leaks",
|
||||
"check:experimental-shelf",
|
||||
"check:export-poster-prints",
|
||||
"check:export-poster-prints:maplibre",
|
||||
"check:facet-cascade:offline",
|
||||
"check:facet-cascade:self-test",
|
||||
"check:facet-changes-map",
|
||||
"check:facet-changes-map:built",
|
||||
"check:facet-changes-map:self-test",
|
||||
"check:facet-one-field:self-test",
|
||||
"check:filter-multivalue",
|
||||
"check:filter-width",
|
||||
"check:fits-screen",
|
||||
"check:fits-screen:self-test",
|
||||
"check:fits-screen:self-test:b",
|
||||
"check:fits-screen:self-test:c",
|
||||
"check:fits-screen:self-test:d",
|
||||
"check:fits-screen:self-test:e",
|
||||
"check:frozen-labels:self-test",
|
||||
"check:gallery-embed",
|
||||
"check:gallery-embed:iframe",
|
||||
"check:gallery-embed:js",
|
||||
"check:gallery-embed:live-cors",
|
||||
"check:gallery-embed:maplibre",
|
||||
"check:gospel-all",
|
||||
"check:gospel-internet",
|
||||
"check:gospel-tables-staged",
|
||||
"check:hardcoded-english",
|
||||
"check:heartbeat:self-test",
|
||||
"check:host-locale-contract",
|
||||
"check:hover-channel",
|
||||
"check:kit-options-reachable",
|
||||
"check:language-code-link",
|
||||
"check:layout-key-namespaced",
|
||||
"check:leaf-english-leaks",
|
||||
"check:leaf-english-leaks:self-test",
|
||||
"check:leaves-mount",
|
||||
"check:leaves-mount:poison",
|
||||
"check:legend-export-reachable",
|
||||
"check:legend-mirrors",
|
||||
"check:link-coverage",
|
||||
"check:link-coverage:poison",
|
||||
"check:link-rot",
|
||||
"check:map-data-painted",
|
||||
"check:map-data-painted:poison",
|
||||
"check:map-embed-cells",
|
||||
"check:map-paints",
|
||||
"check:map-paints:poison",
|
||||
"check:map-unmount-mid-init",
|
||||
"check:map-unmount-race",
|
||||
"check:maplibre-drops",
|
||||
"check:mfe-two-tab",
|
||||
"check:mobile",
|
||||
"check:no-conflict-markers",
|
||||
"check:no-conflict-markers:poison",
|
||||
"check:no-public-binds",
|
||||
"check:packet-render",
|
||||
"check:packet-render:expect-funnel",
|
||||
"check:poster-card-fits",
|
||||
"check:poster-degrades",
|
||||
"check:poster-prints",
|
||||
"check:poster-raster",
|
||||
"check:poster-wall-localised",
|
||||
"check:prayer-walk-planner",
|
||||
"check:print-geometry",
|
||||
"check:published",
|
||||
"check:rtl-raw",
|
||||
"check:rtl-raw:poison",
|
||||
"check:rtl-shaping",
|
||||
"check:rtl-shaping:poison",
|
||||
"check:scrim-viewport",
|
||||
"check:scrim-viewport:self-test",
|
||||
"check:selector-fields:self-test",
|
||||
"check:series-isolation",
|
||||
"check:shared-reached",
|
||||
"check:shot-distinctness",
|
||||
"check:shot-fidelity",
|
||||
"check:shot-fidelity:strict",
|
||||
"check:shot-fonts",
|
||||
"check:shots",
|
||||
"check:snapshot-guard",
|
||||
"check:snapshot-guard:poison",
|
||||
"check:spa-tab-order",
|
||||
"check:stale-sweep-claim",
|
||||
"check:studio-all",
|
||||
"check:studio-rail:self-test",
|
||||
"check:surfaces-read-the-tier",
|
||||
"check:surfaces-read-the-tier:poison",
|
||||
"check:synced-css:self-test",
|
||||
"check:tab-radius:self-test",
|
||||
"check:tauri",
|
||||
"check:translator-contacts",
|
||||
"check:translator-contacts:poison",
|
||||
"check:tree-weight",
|
||||
"check:upg-map-data",
|
||||
"check:verification-tags",
|
||||
"check:vision-card-numbers",
|
||||
"check:zoning-declared:self-test",
|
||||
"smoke:aria-labels",
|
||||
"smoke:gallery",
|
||||
"smoke:playground",
|
||||
"smoke:share",
|
||||
"smoke:studio",
|
||||
"verify:browser-all",
|
||||
"verify:browser-all:plugin-gallery",
|
||||
"verify:full",
|
||||
"verify:interop",
|
||||
"verify:interop:self-test",
|
||||
"verify:interop:self-test:camera-freeze"
|
||||
]
|
||||
}
|
||||
|
|
@ -392,6 +392,7 @@
|
|||
"check:approx-pin-note": "people-group pins say they are localities, not towns or boundaries, in ONE shared wording (library/components/legends/ApproxPinNote.vue): the colocation index finds groups sharing a coordinate, the note names them only for a shared point, never prints a raw coordinate, and renders from each locale's catalogue. Headless.",
|
||||
"check:auto-engine": "BROWSER. The default engine (auto) is MapLibre, token or not: on one map leaf, no token AND a host token alone both draw with MapLibre, paint, and make zero api.mapbox.com requests; ?provider=mapbox with that token draws Mapbox and carries it. In gates:post, NOT MEASURED outside the browser slot.",
|
||||
"check:no-new-driver-references": "shipped text (library, plugins, internal, docs, skills, README) neither quotes nor cites the Driver: a line that refers to the Driver and is not in internal/fixtures/driver-references-baseline.json FAILS. Forward-only: the baseline, keyed by file and line text with counts (not line numbers), may only shrink as owners sweep their areas (--shrink tidies it; it never grows). Matches the role word only, not a GPU or device driver. Node only.",
|
||||
"check:no-orphan-gates": "every gate-shaped internal/*.mjs is reachable or tagged @one-shot; and every check:/verify:/smoke: script is run by a standing chain, matched by EXACT name (gates:pre, gates:post, the browser suite GATES, the build, and what those bun run), or declared in internal/fixtures/on-demand-checks.json with its reason, or in the shrink-only baseline of the 2026-09-29 orphans. A chained :self-test does not chain its gate. --shrink / --init maintain the baseline.",
|
||||
"check:plural-forms": "the three things a CLDR pluralRules map can silently break. (1) FORM COUNT: a locale must author as many forms as it has CLDR categories or the pick clamps onto the wrong one (ru 4, ar 6). (2) THE tStatic PATH: seven bundles install no vue-i18n and read through tStatic, which does NO plural selection, so a pipe key there reaches the page as the literal \"a | b | c\". (3) ZERO: a locale with an explicit CLDR zero category must author it. Written as the precondition for installing the map, and kept as the guard after.",
|
||||
"check:backlog-refs": "a card may not depend on a card that does not exist. A dangling depends_on can never be done, so the card can never become ready and nothing says so — it reads as ordinary blocked work forever. FAILS only for depends_on on an OPEN card; a dangling parent or a stale link on a done card is REPORTED, because failing on filing debt teaches people to delete the reference rather than restore the card. Reads all three phase backlogs — references legitimately cross phases. SKIPs when _qa-cycle is absent.",
|
||||
"check:basemap-labels": "the BASEMAP's place names follow the reader's language on BOTH engines (library/maps/localize-labels.js): reads the RENDERED country labels at ?lang=en and ?lang=ru — Mapbox via the Standard basemap's place-labels featureset, MapLibre via each symbol's evaluated text — and requires them to flip. Serves this checkout on an OS-assigned port; --dir <tree> for a negative control. Negative-tested: a bundle built before 319064a1 FAILS on both engines.",
|
||||
|
|
|
|||
Loading…
Reference in a new issue