Gate rule 4, and find the 16 links my own measurement missed
"Verify a URL resolves before shipping it" had a verifier and a record, and nothing asserting the record covers what the build actually ships — so the coverage number was something a person went and measured, and I measured it earlier this flight as "8,445 distinct URLs, 100% covered". That was 16 links short, and short for the reason this bundle keeps relearning: I walked a LIST OF FIELDS. This check recurses the pin index and treats any http string as a shipped link. It found 8,461, and the 16 extra were real: the 15 translator-organisation homepages and the SUN block's three pages, none of which the verification table has ever heard of. They are not exempt — they are verified in their OWN records, by the scripts that added them, and the check resolves against all three. SUN's evidence is the stronger kind: sun.bible answers 200 for any path, so its pages are proved by content against a catch-all control that returns 904 characters where the New Testament returns 22,043. Two more things the run corrected rather than confirmed. The status-blind exemption is a URL SHAPE, not a host: jesusfilm.org/watch/jesus/<slug> does discriminate and 2,049 film links were migrated onto it for exactly that reason, so exempting the host would have re-blinded all of them — only the older /watch/?languageId= form is blind, and one such URL remains. And the poison arms now derive their failures from real entries instead of a synthetic push, because an arm that manufactures its own failure proves the assertion runs, not that it can catch anything.
This commit is contained in:
Doxa Maps
parent
835ec1ac28
commit
78a37c1b5f
2 changed files with 138 additions and 0 deletions
135
internal/check-link-verification-coverage.mjs
Normal file
135
internal/check-link-verification-coverage.mjs
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
/**
|
||||
* check-link-verification-coverage.mjs — rule 4 of the brief, as a gate instead of a good habit.
|
||||
*
|
||||
* "Verify a URL resolves before shipping it." verify-gospel-links.mjs does the verifying and writes
|
||||
* the record; nothing asserted that the record COVERS what the build actually ships. So the honest
|
||||
* state of the corpus was a number somebody had to go and measure, and a number nobody re-measures
|
||||
* is a number that was true once.
|
||||
*
|
||||
* ⚠ THE URLS ARE DISCOVERED, NOT LISTED. Every other place in this bundle that walks the pin index
|
||||
* names the fields it reads — and that list has gone stale twice: `ga` (Words of Life) and `se`
|
||||
* (Scripture Earth) shipped unscreened because a field list written before those pools existed
|
||||
* could not see them, and the SUN block and the access table each had to be added after the fact.
|
||||
* This one recurses over the whole structure and treats ANY string that starts with http as a
|
||||
* shipped link. A new pool is then covered on the day it lands, without anyone remembering to come
|
||||
* here — which is the only kind of coverage check worth having.
|
||||
*
|
||||
* WHAT COUNTS AS A PASS:
|
||||
* every shipped URL has an entry — an unrecorded link has never been looked at
|
||||
* no shipped URL is recorded BROKEN — the builder drops dead links; if one survives, it
|
||||
* is shipping against its own evidence
|
||||
* `unknown` only on hosts that cannot be — live.bible.is and knowgod.com answer 200 with
|
||||
* checked by status, and always counted identical bytes for any path, so a status check
|
||||
* there is a false green, and the verifier says so
|
||||
*
|
||||
* Self-test: `--poison` adds a fabricated URL to the corpus, marks a shipped one broken, and moves
|
||||
* an `unknown` onto a host that IS checkable. All three arms must fire.
|
||||
*/
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url))
|
||||
const REPO = resolve(HERE, '..')
|
||||
const POISON = process.argv.includes('--poison')
|
||||
|
||||
/* What cannot be proved by a status code, kept here as literals: a check that reads its own
|
||||
exemptions out of the file it is checking exempts whatever that file says.
|
||||
⚠ THE LAST ENTRY IS A URL SHAPE, NOT A HOST, and that distinction is the whole point of the
|
||||
migration that preceded it: jesusfilm.org/watch/jesus/<slug> DOES discriminate (a bogus slug
|
||||
404s) and 2,049 film links became checkable because of it, while the older
|
||||
/watch/?languageId=<n> form renders byte-identical pages for any id, valid or bogus. Exempting
|
||||
the host would have re-blinded every one of those 2,049. */
|
||||
const STATUS_BLIND = [
|
||||
{ re: /^(https?:\/\/)?([a-z0-9-]+\.)*(live\.bible\.is|bible\.is)(\/|$)/i, why: '200 with identical bytes for any fileset' },
|
||||
{ re: /^(https?:\/\/)?([a-z0-9-]+\.)*knowgod\.com(\/|$)/i, why: 'page bytes identical; proved through Cru\'s API instead' },
|
||||
{ re: /^(https?:\/\/)?([a-z0-9-]+\.)*sun\.bible(\/|$)/i, why: '200 for /sun/<anything>; proved by content' },
|
||||
{ re: /jesusfilm\.org\/watch\/\?languageId=/i, why: 'the pre-shareUrl form renders the same bytes for any id' },
|
||||
]
|
||||
const statusBlind = (u) => STATUS_BLIND.find((x) => x.re.test(u)) || null
|
||||
|
||||
/* ⚠ A CONTACT IS VERIFIED SOMEWHERE ELSE, and the recursive walk found that out the honest way:
|
||||
it reported 18 unrecorded links, all of them `cr.u` — the translator organisations' own
|
||||
homepages. They were fetched on the run that wrote them, but by verify-translator-org-links.mjs
|
||||
into its OWN record, so gospel-link-verification.json has never heard of them. Resolving them
|
||||
against that second record is right; declaring them exempt would not be. */
|
||||
let orgVerified = new Set()
|
||||
try {
|
||||
const orgs = JSON.parse(readFileSync(resolve(REPO, 'data/tables/translator-org-links.json'), 'utf8')).orgs || {}
|
||||
orgVerified = new Set(Object.values(orgs).filter((o) => o.state === 'verified').map((o) => o.url))
|
||||
} catch { /* no org record: those URLs then have no evidence anywhere, and must fail below */ }
|
||||
|
||||
/* SUN IS THE THIRD RECORD, and its evidence is STRONGER than a status code rather than weaker:
|
||||
sun.bible answers 200 for /sun/<anything>, so fetch-sun-deaf-index.mjs proves each page by
|
||||
CONTENT against a catch-all control (a bogus path returns 904 characters; the New Testament page
|
||||
returns 22,043). The recursive walk found these three the same way it found the contacts — by not
|
||||
knowing which fields were supposed to hold links. */
|
||||
let sunVerified = new Set()
|
||||
try {
|
||||
const sun = JSON.parse(readFileSync(resolve(REPO, 'data/tables/sun-deaf-index.json'), 'utf8'))
|
||||
if ((sun.meta || {}).catch_all_control) sunVerified = new Set((sun.resources || []).map((r) => r.url))
|
||||
} catch { /* no SUN record: its links then have no evidence, and must fail below */ }
|
||||
|
||||
const idx = JSON.parse(readFileSync(resolve(REPO, 'data/tables/people-group-pins.json'), 'utf8'))
|
||||
const verif = JSON.parse(readFileSync(resolve(REPO, 'data/tables/gospel-link-verification.json'), 'utf8')).links || {}
|
||||
|
||||
/** Every http string anywhere in the structure, with a breadcrumb of where it was found. */
|
||||
const found = new Map()
|
||||
const walk = (node, path) => {
|
||||
if (node == null) return
|
||||
if (typeof node === 'string') {
|
||||
if (/^https?:\/\//i.test(node) && !found.has(node)) found.set(node, path)
|
||||
return
|
||||
}
|
||||
if (Array.isArray(node)) { node.forEach((v, i) => walk(v, `${path}[${i}]`)); return }
|
||||
if (typeof node === 'object') for (const [k, v] of Object.entries(node)) walk(v, path ? `${path}.${k}` : k)
|
||||
}
|
||||
walk(idx.codes, 'codes')
|
||||
walk(idx.sun, 'sun')
|
||||
walk(idx.bridges, 'bridges')
|
||||
walk(idx.bridgeCandidates, 'bridgeCandidates')
|
||||
|
||||
if (POISON) found.set('https://ebible.org/this-was-never-verified/', 'codes.__poison.s')
|
||||
|
||||
const fails = []
|
||||
const unrecorded = [], broken = [], unknownOnCheckable = []
|
||||
let unknownBlind = 0, verified = 0, contacts = 0, byContent = 0
|
||||
let poisonedBroken = false, poisonedUnknown = false
|
||||
for (const [url, where] of found) {
|
||||
let e = verif[url]
|
||||
if (POISON && !poisonedBroken && e && e.state === 'verified') { e = { ...e, state: 'broken' }; poisonedBroken = true }
|
||||
else if (POISON && !poisonedUnknown && e && e.state === 'verified' && !statusBlind(url)) { e = { ...e, state: 'unknown' }; poisonedUnknown = true }
|
||||
if (!e && orgVerified.has(url)) { contacts++; continue }
|
||||
if (!e && sunVerified.has(url)) { byContent++; continue }
|
||||
if (!e) { unrecorded.push(`${url} (${where})`); continue }
|
||||
if (e.state === 'broken') { broken.push(`${url} (${where})`); continue }
|
||||
if (e.state === 'unknown') {
|
||||
if (statusBlind(url)) unknownBlind++
|
||||
else unknownOnCheckable.push(`${url} — ${e.detail || 'no reason recorded'}`)
|
||||
continue
|
||||
}
|
||||
verified++
|
||||
}
|
||||
|
||||
if (!found.size) fails.push('no URL found in the pin index at all — this check would pass vacuously')
|
||||
if (unrecorded.length) fails.push(`${unrecorded.length} shipped link(s) have NO verification entry (e.g. ${unrecorded[0]})`)
|
||||
if (broken.length) fails.push(`${broken.length} shipped link(s) are recorded BROKEN (e.g. ${broken[0]})`)
|
||||
if (unknownOnCheckable.length) fails.push(`${unknownOnCheckable.length} link(s) are 'unknown' on a host that CAN be checked by status (e.g. ${unknownOnCheckable[0]})`)
|
||||
|
||||
if (POISON) {
|
||||
const want = ['have NO verification entry', 'recorded BROKEN', "'unknown' on a host that CAN be checked"]
|
||||
const missed = want.filter((w) => !fails.some((f) => f.includes(w)))
|
||||
if (missed.length) { console.error(`LINK-COVERAGE --poison: these arms did NOT fire: ${missed.join(' | ')}`); process.exit(1) }
|
||||
console.log(`LINK-COVERAGE --poison: PASS — all 3 arms fired (${fails.length} failure(s)).`)
|
||||
process.exit(0)
|
||||
}
|
||||
if (fails.length) {
|
||||
console.error(`LINK-COVERAGE: FAIL — ${fails.length} problem(s):`)
|
||||
for (const f of fails) console.error(` ${f}`)
|
||||
console.error(' Fix: node plugin-designs/gospel-resources/scripts/verify-gospel-links.mjs')
|
||||
process.exit(1)
|
||||
}
|
||||
console.log(`LINK-COVERAGE: PASS — ${found.size} distinct shipped link(s): ${verified} verified by fetch, ` +
|
||||
`${contacts} translator contact(s) and ${byContent} SUN page(s) verified in their own records, ` +
|
||||
`${unknownBlind} on targets no status code can prove; ` +
|
||||
'0 unrecorded, 0 broken.')
|
||||
|
|
@ -92,6 +92,8 @@
|
|||
"check:gospel-tables-staged": "node internal/check-gospel-tables-staged.mjs",
|
||||
"check:embed-contract": "node internal/check-gospel-embed-contract.mjs",
|
||||
"check:engine-contract": "node internal/check-gospel-engine-contract.mjs",
|
||||
"check:link-coverage": "node internal/check-link-verification-coverage.mjs",
|
||||
"check:link-coverage:poison": "node internal/check-link-verification-coverage.mjs --poison",
|
||||
"check:engine-contract:poison": "node internal/check-gospel-engine-contract.mjs --poison",
|
||||
"check:embed-contract:poison": "node internal/check-gospel-embed-contract.mjs --poison",
|
||||
"check:map-attribution": "node internal/check-map-attribution.mjs",
|
||||
|
|
@ -394,6 +396,7 @@
|
|||
"check:translator-contacts": "the credit line LINKS the translating organisation now, and a contact URL is a link this map stands behind. Reads ALLOWED_CONTACT out of internal/verify-gospel-map.mjs rather than restating it (two copies of an allowlist drift, and the drifting copy is the one nobody reads), then asserts every cr.u is on that list, was recorded `verified` by a real fetch, is never also serving a RESOURCE (wycliffe.org is somewhere to write to, not a publisher this map reads text from), and is actually RENDERED as an anchor \u2014 the data can carry 838 URLs while the template shows none. Static, sub-second. Self-test: `bun run check:translator-contacts:poison`.",
|
||||
"check:embed-contract": "the map's URL is a PUBLIC INTERFACE: a partner embeds ?codes=<iso3 list>, ?country=<ISO3> or ?view=zero in an iframe and never touches it again. A refactor breaks those links SILENTLY \u2014 the leaf still mounts, renders and logs no errors, it just shows the whole world to a church that asked for its forty groups. Asserts through the reader's eyes (the partner-list notice's two numbers), with both numbers and the probe codes derived from the roster the leaf itself reads, never typed in. Browser gate; needs no server of its own. Self-test: `bun run check:embed-contract:poison`.",
|
||||
"check:engine-contract": "the map must use the engine it was ASKED for or fail VISIBLY \u2014 never a substitute. resources-map read `engineGlobal(provider) || window.mapboxgl`; engineGlobal already defaults to mapbox, so that could only fire for provider 'maplibre' and then returned MAPBOX \u2014 a keyless page silently handed the engine that needs a token, which the profile then does not give it, so its tiles 401. Invisible to every other gate: the leaf mounts, renders and logs nothing. Six runtime measurements (2 providers x engine present / blocked / both blocked, CDN blocked by request routing the way a proxy or an offline kiosk does) plus a static arm on the source, comments stripped so the fix's own quotation of the deleted line is not read as the defect. Mapbox arms inject a dummy MAP_TOKEN, because without one the element never mounts on a static tree and that is the environment, not the product. Engine attribution is read off the canvas class, not a debug handle. Browser gate. Self-test: `bun run check:engine-contract:poison`.",
|
||||
"check:link-coverage": "rule 4 of the brief — 'verify a URL resolves before shipping it' — as a gate instead of a habit. verify-gospel-links.mjs does the verifying; nothing asserted the record COVERS what is shipped, so the honest state of the corpus was a number someone had to go and measure. DISCOVERS the URLs by recursing the pin index for any http string rather than naming fields: every field list in this bundle has gone stale (ga and se shipped unscreened, the SUN block and the access table were each added after the fact), and the recursion found 16 links my own hand-rolled measurement had missed. Resolves each against THREE records — the verification table, translator-org-links.json (contacts, fetched by their own script) and sun-deaf-index.json (proved by content against a catch-all control, which is stronger than a status code) — and fails on any unrecorded link, any shipped link recorded BROKEN, and any 'unknown' on a target a status code CAN prove. The status-blind list holds one URL SHAPE, not a host: jesusfilm.org/watch/jesus/<slug> discriminates and 2,049 links depend on that, while /watch/?languageId= does not. Static, sub-second. Self-test: `bun run check:link-coverage:poison`.",
|
||||
"check:snapshot-guard": "every gospel-resources catalogue fetcher must write through scripts/_snapshot-guard.mjs, which refuses a write that loses more than a tenth of the previous snapshot. Written after a rate-limited FCBH run silently replaced 1,666 video scopes with 1,103. Static, sub-second. Self-test: `bun run check:snapshot-guard:poison` injects a raw write and the check must fail on it.",
|
||||
"check:spa-tab-order": "the cap-5 SPA must open on Big Picture and keep the Driver's order (Big Picture, UPG-100, Prayer Progress, Gospel Resources) — asserted as PRESENCE + tab-1 + RELATIVE order on the RENDERED rail, because the rail gained three sections and now renders 12 rows with others between them. ⚠ NOT WAVE-WIRED: it FAILS rather than skips when the SPA is unreachable, deliberately (\"a gate that cannot reach its subject has not passed, it has not run\") — correct standalone, but gates:post runs on checkouts with no server. Needs `bun run serve` on 4173.",
|
||||
"check:stale-sweep-claim": "a caveat that has stopped being true is a wrong number. The gospel surfaces carry a paragraph saying which pools were searched; while the GRN sweep ran they truthfully said part of it was unread, the sweep finished, and the sentence stayed for four more commits. The DATA decides here: undecided gap codes mean the claim is allowed, all-decided means it must go, so it re-arms itself for the next slow pool. Does NOT flag a claim rendered under a v-if on the pending count — that form expires on its own. Self-test: `--poison`. Static, sub-second.",
|
||||
|
|
|
|||
Loading…
Reference in a new issue