packets: give CI something to check, instead of a sweep that always SKIPs
Some checks are pending
build-gate / build (push) Waiting to run
Some checks are pending
build-gate / build (push) Waiting to run
check:packets-all is in gates:post. CI runs it on every build. It has SKIPped every single time, because library/scripts/posters/out/ is gitignored — both the records cache and every generated packet — so a clean checkout has none. Every packet assertion in this tree has therefore never once been evaluated there: the 12 prayer points, the wheel's 12 labels, page 1's geography paths, the print layer, the build stamp. Several ticks of work accumulating on a gate that had only ever graded files in my own working tree. A SKIP that is always taken is indistinguishable from an unwired gate — and the wave-membership census cannot tell them apart, because membership measures invocation, not evaluation. That census is how check:packets-all was created in the first place, which makes this the same class recurring one level up. I raised this at the end of the last tick and deferred it on the premise that closing it "adds a live-API call to every build". The premise was wrong. packet.mjs already had an OFFLINE mode, and a fixture needs no network at all. The fixture is the smallest zone — Tibet, 43 slugs, 20KB — with image_url stripped so CI can never fetch 43 photos from someone else's CDN. It is DERIVED, not hand-maintained: membership comes from the committed clusters.json, so check:packet-fixture regenerates and content-compares it. Re-adding an image_url fails; dropping a record fails naming the slug. A fixture packet stamps itself "fixture", and "fixture nomap" in CI, which has no poster sources either. `fixture` implies nophotos in the gate — without that it reported "every source record carries an image_url", a true sentence about real records and a false one about this packet. A real bug in my own change, caught only by running the script through its REAL invocation path: packet.mjs fails closed on a missing VITE_API_BASE_URL BEFORE loadRecords() ever looks at PACKET_RECORDS, so `bun run packet:ci` exited 1 on a tree that had everything it needed. Every hand-test had passed because I was supplying a dummy host. The guard now exempts fixture builds and still fails closed otherwise. From a genuinely clean state — the whole out/ directory moved away — the sweep now reports "1 packet(s) checked, 1 passed". i18n verified through that same path: a partial block (1 string, 5/12 points, 3/12 rhythms) renders 12 points, 12/12 wheel labels, wheel 3 translated and 9 English, 0 blanks. check:packet-fixture joins gates:post (34 -> 35), README quote re-synced in the same commit. Wave 35/35, packets-all 11/11 here and 2/2 in the studio.
This commit is contained in:
parent
9484a98bbd
commit
3c17245c73
7 changed files with 166 additions and 6 deletions
|
|
@ -83,6 +83,21 @@ jobs:
|
|||
- name: Install dependencies (locked)
|
||||
run: bun install --frozen-lockfile
|
||||
|
||||
# ★ BUILD ONE PACKET BEFORE THE GATES, so check:packets-all has something to check.
|
||||
# library/scripts/posters/out/ is gitignored — both the records cache and every
|
||||
# generated packet — so a clean CI checkout had NONE, that sweep SKIPped on every
|
||||
# single build, and every packet assertion in this tree (the 12 prayer points, the
|
||||
# wheel's 12 labels, page 1's geography, the print layer) was never once evaluated
|
||||
# here. A SKIP that is always taken is indistinguishable from an unwired gate.
|
||||
# It builds from a 20KB COMMITTED fixture: no API, no network, no photo fetch. The
|
||||
# packet stamps itself "fixture nomap" (CI has no poster sources either), which the
|
||||
# gate reads and accepts as declared — while still asserting everything that does
|
||||
# not depend on live data. It runs BEFORE build:unlocked because gates:post is
|
||||
# inside that script.
|
||||
- name: Build one packet from the committed fixture
|
||||
run: bun run packet:ci
|
||||
|
||||
|
||||
# THE AUTHORITATIVE BUILD. build:unlocked is the real chain (clean -> token
|
||||
# -> per-bundle vite -> tree -> drop/parse checks -> verify -> gallery). We
|
||||
# call :unlocked, NOT `bun run build`, because CI IS the single builder now —
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ The scripts these commands are grounded in (from this branch's `package.json`):
|
|||
"build": "bash internal/build-lock.sh",
|
||||
"build:unlocked": "bun run gates:pre && bun run library-index && bun run check:library-index && bun run clean && bun run token && bun run build:bundles && bun run tree && bun run gates:post && bun run gallery",
|
||||
"gates:pre": "node internal/run-parallel.mjs --label=pre -- check:template-literal check:mjs-parse check:sfc-parse check:tab-radius check:synced-css check:wp-embed check:doc-scripts",
|
||||
"gates:post": "node internal/run-parallel.mjs --label=post -- check:drop check:parse verify check:sri check:public-hygiene check:label-joins check:facet-cascade check:facet-one-field check:heartbeat check:kpi-unit check:composite-reg check:descriptions check:no-abs-paths check:no-codename check:dep-names check:composite-srcs check:embed-targets check:nested-token check:i18n-parity check:tabs-localized check:packets-all check:packet-strings-parity check:zone-census check:share-policy check:library-index check:csv-shape check:capability-citations check:catalog-identity check:no-orphan-gates check:quoted-scripts check:done-evidence check:script-flags check:backlog-status check:capability-exemplars",
|
||||
"gates:post": "node internal/run-parallel.mjs --label=post -- check:drop check:parse verify check:sri check:public-hygiene check:label-joins check:facet-cascade check:facet-one-field check:heartbeat check:kpi-unit check:composite-reg check:descriptions check:no-abs-paths check:no-codename check:dep-names check:composite-srcs check:embed-targets check:nested-token check:i18n-parity check:tabs-localized check:packets-all check:packet-strings-parity check:zone-census check:share-policy check:library-index check:csv-shape check:capability-citations check:catalog-identity check:no-orphan-gates check:quoted-scripts check:done-evidence check:script-flags check:backlog-status check:capability-exemplars check:packet-fixture",
|
||||
"build:bundles": "node internal/run-parallel.mjs --bundles", // the 22 vite builds, 4 at a time
|
||||
"preview": "bun x vite preview",
|
||||
"serve": "node internal/serve-strict.mjs", // 404s a directory, like production
|
||||
|
|
|
|||
|
|
@ -346,7 +346,9 @@ if (r.cards && r.imgs + r.nophoto !== r.cards) {
|
|||
}
|
||||
// The packet's own stamp is authoritative; the env var stays as an override for
|
||||
// artefacts generated before data-packet-build existed.
|
||||
const expectPhotos = process.env.PACKET_EXPECT_PHOTOS !== '0' && !/\bnophotos\b/.test(build)
|
||||
// `fixture` implies no photos: a fixture packet is built from a committed records subset
|
||||
// with image_url stripped precisely so CI never fetches ~43 images from someone else's CDN.
|
||||
const expectPhotos = process.env.PACKET_EXPECT_PHOTOS !== '0' && !/\b(nophotos|fixture)\b/.test(build)
|
||||
if (r.cards && expectPhotos && r.imgs === 0) {
|
||||
fails.push(`${r.cards} people cards and NOT ONE photo — the packet spec is "20 per page `
|
||||
+ 'WITH IMAGES", and every source record carries an image_url, so this is a skipped or '
|
||||
|
|
|
|||
112
internal/generate-packet-fixture.mjs
Normal file
112
internal/generate-packet-fixture.mjs
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
#!/usr/bin/env node
|
||||
// generate-packet-fixture — a small COMMITTED records subset, so CI can build a packet.
|
||||
//
|
||||
// ★ WHY THIS EXISTS. library/scripts/posters/out/ is gitignored — both the records cache
|
||||
// and every generated packet — so on a clean checkout check:packets-all finds nothing and
|
||||
// SKIPs. MEASURED 2026-09-24: CI runs that sweep on every build through gates:post, and it
|
||||
// has SKIPped every time. Every packet assertion in this tree — the 12 prayer points, the
|
||||
// wheel's 12 labels, page 1's geography paths, the print layer, the build stamp — has never
|
||||
// once been evaluated in CI. A SKIP that is always taken is indistinguishable from an
|
||||
// unwired gate, and the wave-membership census cannot tell them apart because membership
|
||||
// measures INVOCATION, not EVALUATION.
|
||||
//
|
||||
// ★ WHY A FIXTURE RATHER THAN A LIVE FETCH. Building a real packet needs 2,107 records
|
||||
// (887KB, live data that goes stale) and, with photos, ~95 CDN requests. Neither belongs in
|
||||
// every CI run. This carries ONE zone's records — the smallest, Tibet at 43 slugs, 20KB —
|
||||
// with image_url stripped so it can never reach for a photo.
|
||||
//
|
||||
// ★ AND IT IS DERIVED, NOT HAND-MAINTAINED. Membership comes from clusters.json, which IS
|
||||
// committed, so this can be regenerated and content-compared (check:packet-fixture) exactly
|
||||
// like the capability-exemplar table. A fixture nobody can verify is a fixture that rots.
|
||||
//
|
||||
// ⚠ IT IS NOT A SUBSTITUTE FOR REAL PACKETS. PACKET_RECORDS is opt-in and never the default:
|
||||
// a fixture packet proves the RENDERER and the GATE work end to end, not that today's live
|
||||
// data is sound. The 11-locale sweep against real records stays the real verification.
|
||||
//
|
||||
// Usage: node internal/generate-packet-fixture.mjs [--check]
|
||||
// (reads the live records cache; run after a normal `node library/scripts/posters/packet.mjs`)
|
||||
import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'node:fs'
|
||||
import { resolve, dirname, join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..')
|
||||
const CACHE = join(ROOT, 'library/scripts/posters/out/_records.json')
|
||||
const CLUSTERS = join(ROOT, 'plugin-designs/my-upg-100-list/data/clusters.json')
|
||||
const OUT_DIR = join(ROOT, 'library/scripts/posters/fixtures')
|
||||
const OUT = join(OUT_DIR, 'records-tibet.json')
|
||||
// The smallest zone by slug count, so the fixture stays small. Its membership lives in
|
||||
// clusters.json; if that zone is ever renamed this script fails loudly rather than
|
||||
// silently emitting an empty fixture.
|
||||
const ZONE = 'asia-china-tibet-autonomous-region'
|
||||
|
||||
const CHECK = process.argv.includes('--check')
|
||||
|
||||
if (!existsSync(CLUSTERS)) {
|
||||
console.log('PACKET-FIXTURE: SKIP — no clusters.json.')
|
||||
process.exit(0)
|
||||
}
|
||||
const clusters = JSON.parse(readFileSync(CLUSTERS, 'utf8'))
|
||||
const zones = [...(clusters.lists || []), ...(clusters.asia_lists || []), ...(clusters.uncovered_lists || [])]
|
||||
const zone = zones.find((z) => z && z.id === ZONE)
|
||||
if (!zone || !zone.selector || zone.selector.type !== 'pg_slugs') {
|
||||
console.error(`PACKET-FIXTURE: FAIL — ${ZONE} is not a pg_slugs zone in clusters.json.`)
|
||||
console.error(' The fixture is keyed to that zone; pick another and update ZONE here.')
|
||||
process.exit(1)
|
||||
}
|
||||
const want = new Set(zone.selector.values || [])
|
||||
|
||||
// ── CHECK MODE: the committed fixture must still match clusters.json ────────
|
||||
if (CHECK) {
|
||||
if (!existsSync(OUT)) {
|
||||
console.error(`PACKET-FIXTURE: FAIL — ${OUT.slice(ROOT.length + 1)} does not exist.`)
|
||||
console.error(' Fix: node internal/generate-packet-fixture.mjs (then commit it)')
|
||||
process.exit(1)
|
||||
}
|
||||
const have = JSON.parse(readFileSync(OUT, 'utf8'))
|
||||
const haveSlugs = new Set(have.map((r) => r.slug))
|
||||
const missing = [...want].filter((s) => !haveSlugs.has(s))
|
||||
const extra = [...haveSlugs].filter((s) => !want.has(s))
|
||||
const withPhotos = have.filter((r) => r.image_url).length
|
||||
console.log(`PACKET-FIXTURE: ${have.length} record(s) for ${ZONE}.`)
|
||||
const fails = []
|
||||
if (missing.length) fails.push(`${missing.length} slug(s) in clusters.json are absent: ${missing.slice(0, 4).join(', ')}`)
|
||||
if (extra.length) fails.push(`${extra.length} record(s) are no longer in the zone: ${extra.slice(0, 4).join(', ')}`)
|
||||
// image_url must stay stripped, or CI starts fetching ~43 photos from someone else's CDN.
|
||||
if (withPhotos) fails.push(`${withPhotos} record(s) still carry image_url — the fixture must never fetch photos`)
|
||||
if (fails.length) {
|
||||
console.error(`\nPACKET-FIXTURE: FAIL — ${fails.length} problem(s)`)
|
||||
for (const f of fails) console.error(' - ' + f)
|
||||
console.error('\n Fix: node internal/generate-packet-fixture.mjs (then commit it)')
|
||||
process.exit(1)
|
||||
}
|
||||
console.log('PACKET-FIXTURE: PASS — the committed fixture matches clusters.json and fetches no photos.')
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
// ── GENERATE ────────────────────────────────────────────────────────────────
|
||||
if (!existsSync(CACHE)) {
|
||||
console.error('PACKET-FIXTURE: FAIL — no records cache to derive from.')
|
||||
console.error(` Expected ${CACHE.slice(ROOT.length + 1)}.`)
|
||||
console.error(' Run a normal packet build first: node library/scripts/posters/packet.mjs')
|
||||
process.exit(1)
|
||||
}
|
||||
const recs = JSON.parse(readFileSync(CACHE, 'utf8'))
|
||||
const subset = (Array.isArray(recs) ? recs : recs.records || []).filter((r) => r && want.has(r.slug))
|
||||
if (subset.length !== want.size) {
|
||||
console.error(`PACKET-FIXTURE: FAIL — matched ${subset.length} of ${want.size} slug(s) in the cache.`)
|
||||
console.error(' The cache is stale or partial; re-run a full packet build and try again.')
|
||||
process.exit(1)
|
||||
}
|
||||
// STRIP image_url. A fixture that carries photo URLs would make CI fetch ~43 images from
|
||||
// Joshua Project's CDN on every build, which is both slow and rude; the packet degrades to
|
||||
// typographic cards, which is exactly what a fixture should exercise.
|
||||
const lean = subset.map((r) => {
|
||||
const c = { ...r }
|
||||
delete c.image_url
|
||||
c.has_photo = false
|
||||
return c
|
||||
})
|
||||
mkdirSync(OUT_DIR, { recursive: true })
|
||||
writeFileSync(OUT, JSON.stringify(lean))
|
||||
const kb = (JSON.stringify(lean).length / 1024).toFixed(0)
|
||||
console.log(`[packet-fixture] wrote ${OUT.slice(ROOT.length + 1)} — ${lean.length} records for ${ZONE}, ${kb}KB`)
|
||||
1
library/scripts/posters/fixtures/records-tibet.json
Normal file
1
library/scripts/posters/fixtures/records-tibet.json
Normal file
File diff suppressed because one or more lines are too long
|
|
@ -52,8 +52,14 @@ const LANG = process.env.PACKET_LANG || 'en';
|
|||
const ONLY = process.env.ONLY || '';
|
||||
const NOPHOTOS = process.env.NOPHOTOS === '1';
|
||||
const API = process.env.VITE_API_BASE_URL || process.env.MAP_APP_API_URL;
|
||||
if (!API) {
|
||||
console.error('[packet] No API base — set VITE_API_BASE_URL. Failing closed.');
|
||||
// FAIL CLOSED without an API — EXCEPT when records come from a committed fixture, which
|
||||
// is the one case where no API is needed at all. Measured 2026-09-24: this guard runs
|
||||
// BEFORE loadRecords() ever looks at PACKET_RECORDS, so `bun run packet:ci` exited 1 on a
|
||||
// tree that had everything it needed. It only passed in hand-testing because I had been
|
||||
// supplying a dummy host — the real invocation path is what exposed it.
|
||||
if (!API && !process.env.PACKET_RECORDS) {
|
||||
console.error('[packet] No API base — set VITE_API_BASE_URL, or PACKET_RECORDS for a fixture build.');
|
||||
console.error(' Failing closed.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
|
|
@ -109,6 +115,21 @@ const { evaluate } = await import(join(DATA, 'selector-evaluator.js'));
|
|||
const { normalizeRecords, extractRecords } = await import(join(DATA, 'normalize.js'));
|
||||
|
||||
async function loadRecords() {
|
||||
// ★ A COMMITTED FIXTURE, so CI can build a packet at all. out/ is gitignored — both the
|
||||
// records cache AND every generated packet — so on a clean checkout check:packets-all
|
||||
// finds nothing and SKIPs. MEASURED 2026-09-24: CI runs the sweep on EVERY build via
|
||||
// gates:post and it has SKIPped every single time, which means every packet assertion in
|
||||
// this tree (the 12 points, the wheel labels, page 1's geography, the print layer, the
|
||||
// build stamp) has never once been evaluated there. A SKIP that is always taken is
|
||||
// indistinguishable from an unwired gate, and the wave-membership census cannot see the
|
||||
// difference because membership measures INVOCATION, not EVALUATION.
|
||||
// PACKET_RECORDS points at a small committed subset instead of the live API or the
|
||||
// gitignored cache. It is deliberately NOT the default: real packets must use real data.
|
||||
const fixture = process.env.PACKET_RECORDS;
|
||||
if (fixture) {
|
||||
if (!existsSync(fixture)) throw new Error(`PACKET_RECORDS set but missing: ${fixture}`);
|
||||
return JSON.parse(readFileSync(fixture, 'utf8'));
|
||||
}
|
||||
const cache = join(__dirname, 'out', '_records.json');
|
||||
if (process.env.OFFLINE && existsSync(cache)) return JSON.parse(readFileSync(cache, 'utf8'));
|
||||
const fields = 'slug,name,country_code,population,religion,image_url,has_photo,picture_credit,latitude,longitude';
|
||||
|
|
@ -232,7 +253,13 @@ for (const z of zones) {
|
|||
// session, packet-strings.json was 303s newer than the packets while being byte-identical
|
||||
// to HEAD, because a probe restored it from backup. Record the INTENT, which is a fact
|
||||
// about the build, not the clock.
|
||||
const provenance = [NOPHOTOS ? 'nophotos' : null, mapMiss ? 'nomap' : null]
|
||||
// `fixture` is a build fact like the others: a fixture packet is built from a committed
|
||||
// records subset with image_url stripped, so it legitimately has no photos and its
|
||||
// records are not today's live data. Without this the gate reports "every source record
|
||||
// carries an image_url, so this is a skipped or failed photo fetch" — a true sentence
|
||||
// about real records and a false one about this packet.
|
||||
const provenance = [process.env.PACKET_RECORDS ? 'fixture' : null,
|
||||
NOPHOTOS ? 'nophotos' : null, mapMiss ? 'nomap' : null]
|
||||
.filter(Boolean).join(' ') || 'full';
|
||||
const stamped = html.replace('<html ', `<html data-packet-build="${provenance}" `);
|
||||
writeFileSync(join(OUT, name), stamped);
|
||||
|
|
|
|||
|
|
@ -35,6 +35,9 @@
|
|||
"posters:export": "cd library/scripts/posters && node export-public.mjs",
|
||||
"posters:previews": "cd library/scripts/posters && python3 make-previews.py",
|
||||
"packets": "cd library/scripts/posters && VITE_API_BASE_URL=https://pray.doxa.life node packet.mjs # the prayer packets. READS out/posters/<zone>.html for page 1's map, which `bun run posters` writes — without it the packet still builds and page 1 is simply BLANK, so the script now warns per-zone AND in its summary. PACKET_LANG=<loc> for one language, ONLY=<zoneKey> for one zone, NOPHOTOS=1 to skip the photo fetch.",
|
||||
"packet-fixture": "node internal/generate-packet-fixture.mjs # regenerate the committed records subset (smallest zone, image_url stripped) that lets CI build a packet at all — out/ is gitignored, so a clean checkout has neither the records cache nor any packet.",
|
||||
"check:packet-fixture": "node internal/generate-packet-fixture.mjs --check # the committed fixture must still match clusters.json and must never carry image_url (CI would fetch 43 photos from someone else's CDN).",
|
||||
"packet:ci": "sh -c 'F=\"$PWD/library/scripts/posters/fixtures/records-tibet.json\"; PACKET_RECORDS=\"$F\" ONLY=asia-china-tibet-autonomous-region node library/scripts/posters/packet.mjs' # build ONE packet from the committed fixture, no network. This is what makes check:packets-all evaluate something in CI instead of SKIPping, as it had on every build.",
|
||||
"token": "node internal/write-token.mjs",
|
||||
"smoke": "node internal/smoke.mjs",
|
||||
"smoke:embeds": "node internal/embed-smoke.mjs",
|
||||
|
|
@ -154,7 +157,7 @@
|
|||
"check:doc-scripts:self-test": "node internal/check-doc-quotes-scripts.mjs --self-test",
|
||||
"check:legend-source": "node internal/check-legend-matches-source.mjs # the numbers the maps SHOW must match the numbers the live SOURCE has. Every other gate checks INTERNAL consistency — a page can agree with itself perfectly while its legend has stopped describing reality. Offline SKIPs (exit 0); only a reachable endpoint that disagrees fails.",
|
||||
"gates:pre": "node internal/run-parallel.mjs --label=pre -- check:template-literal check:mjs-parse check:sfc-parse check:tab-radius check:synced-css check:wp-embed check:doc-scripts",
|
||||
"gates:post": "node internal/run-parallel.mjs --label=post -- check:drop check:parse verify check:sri check:public-hygiene check:label-joins check:facet-cascade check:facet-one-field check:heartbeat check:kpi-unit check:composite-reg check:descriptions check:no-abs-paths check:no-codename check:dep-names check:composite-srcs check:embed-targets check:nested-token check:i18n-parity check:tabs-localized check:packets-all check:packet-strings-parity check:zone-census check:share-policy check:library-index check:csv-shape check:capability-citations check:catalog-identity check:no-orphan-gates check:quoted-scripts check:done-evidence check:script-flags check:backlog-status check:capability-exemplars",
|
||||
"gates:post": "node internal/run-parallel.mjs --label=post -- check:drop check:parse verify check:sri check:public-hygiene check:label-joins check:facet-cascade check:facet-one-field check:heartbeat check:kpi-unit check:composite-reg check:descriptions check:no-abs-paths check:no-codename check:dep-names check:composite-srcs check:embed-targets check:nested-token check:i18n-parity check:tabs-localized check:packets-all check:packet-strings-parity check:zone-census check:share-policy check:library-index check:csv-shape check:capability-citations check:catalog-identity check:no-orphan-gates check:quoted-scripts check:done-evidence check:script-flags check:backlog-status check:capability-exemplars check:packet-fixture",
|
||||
"build:bundles": "node internal/run-parallel.mjs --bundles",
|
||||
"check:maplibre-drops": "node internal/check-maplibre-layer-drops.mjs",
|
||||
"check:tabs-localized": "node internal/check-tabs-localized.mjs # a chosen locale must CHANGE the rendered tab text. check:i18n-parity compares catalogs to each OTHER, so it cannot tell a shipped translation from a dead one; tabs.json is reached only via a DYNAMIC key (SemanticTreeLegend.vue:40), invisible to any grep. Renders it and reads the result.",
|
||||
|
|
|
|||
Loading…
Reference in a new issue