A half-finished catalogue fetch can no longer overwrite a good one
The FCBH fetcher read per-fileset video scope for 1,666 languages and that file was
committed. A later run, while live.bible.is was rate-limiting, produced 1,103 and
overwrote the good file with the degraded one. Nothing failed, nothing warned, and the
artefact silently lost a third of its content. It was caught only because git status
happened to be read, and the fix was a git checkout.
That is the third time this shape has appeared here:
- a VERIFY_LIMIT smoke run replaced the full verification table with 30 rows;
- dropping a dead link removed it from the corpus, so the next run forgot it was
dead and the build put it straight back;
- this one.
Every snapshot in this bundle is built from a network read, and a network read has a bad
day. The dangerous case is never the run that throws - that leaves the old file alone -
it is the run that half-succeeds and writes anyway.
scripts/_snapshot-guard.mjs now stands in front of every catalogue write. A snapshot may
grow freely and may shrink a little, but losing more than a tenth of the previous run is
treated as a failed fetch: the existing file is left untouched and the script exits 1.
It is deliberately not an equality check, because catalogues do genuinely lose entries
when a publisher retires something, and a gate nobody can satisfy gets switched off.
SNAPSHOT_FORCE=1 is the one loud way past it.
Proven by replaying the exact failure: feeding it the 1,103-entry version refuses the
write, names the real numbers (563 fewer, 13.8%, below the 90% floor) and leaves the
file byte-identical to the good one. SNAPSHOT_FORCE=1 writes it. An equal write passes.
All six fetchers are wired: ebible, arclight, arclight-share-urls, door43, zume, fcbh.
Re-running zume, door43 and ebible for real passes the guard at 52, 208 and 1,237 codes
and produces byte-identical data, so the guard perturbs nothing.
The one exception is the checkpoint write in the resumable share-urls fetcher, which
starts from the existing file as a cache and can only grow. It must now declare itself
with a CHECKPOINT-WRITE-OK marker, so the exemption is a statement rather than an
oversight.
New gate check:snapshot-guard fails if any fetch-*.mjs writes OUT directly without that
marker, or never calls writeSnapshot at all - a guard that is easy to bypass is a guard
that will be bypassed. It refuses to run against an empty file list rather than passing
vacuously, and --poison injects a raw write and requires the check to catch it. Chained
into build:unlocked and its doc copy kept in sync.
This commit is contained in:
doxa-maps-88655c70
parent
c4fbc469ac
commit
0e0b142719
10 changed files with 164 additions and 15 deletions
|
|
@ -25,7 +25,7 @@ The scripts these commands are grounded in (from this branch's `package.json`):
|
|||
// claiming to be ground truth, and this one had silently stopped being it.
|
||||
"dev": "bun x vite",
|
||||
"build": "bash internal/build-lock.sh",
|
||||
"build:unlocked": "bun run check:template-literal && bun run check:mjs-parse && bun run check:coverage-consistent && bun run check:sfc-parse && bun run check:tab-radius && bun run check:synced-css && bun run check:wp-embed && bun run check:doc-scripts && bun run library-index && bun run check:library-index && bun run clean && bun run token && for d in plugin-designs/*/; do BUNDLE=$(basename \"$d\") bun x vite build || exit 1; done && bun run tree && bun run check:drop && bun run check:parse && bun run verify && bun run check:sri && bun run check:public-hygiene && bun run check:label-joins && bun run check:facet-cascade && bun run check:facet-one-field && bun run check:heartbeat && bun run check:kpi-unit && bun run check:composite-reg && bun run check:descriptions && bun run check:no-abs-paths && bun run check:no-codename && bun run check:dep-names && bun run check:composite-srcs && bun run check:embed-targets && bun run check:nested-token && bun run gallery",
|
||||
"build:unlocked": "bun run check:template-literal && bun run check:mjs-parse && bun run check:coverage-consistent && bun run check:snapshot-guard && bun run check:sfc-parse && bun run check:tab-radius && bun run check:synced-css && bun run check:wp-embed && bun run check:doc-scripts && bun run library-index && bun run check:library-index && bun run clean && bun run token && for d in plugin-designs/*/; do BUNDLE=$(basename \"$d\") bun x vite build || exit 1; done && bun run tree && bun run check:drop && bun run check:parse && bun run verify && bun run check:sri && bun run check:public-hygiene && bun run check:label-joins && bun run check:facet-cascade && bun run check:facet-one-field && bun run check:heartbeat && bun run check:kpi-unit && bun run check:composite-reg && bun run check:descriptions && bun run check:no-abs-paths && bun run check:no-codename && bun run check:dep-names && bun run check:composite-srcs && bun run check:embed-targets && bun run check:nested-token && bun run gallery",
|
||||
"preview": "bun x vite preview",
|
||||
"serve": "node internal/serve-strict.mjs", // 404s a directory, like production
|
||||
"cap:sync": "./node_modules/.bin/cap sync android",
|
||||
|
|
|
|||
69
internal/check-snapshot-guard.mjs
Normal file
69
internal/check-snapshot-guard.mjs
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
#!/usr/bin/env node
|
||||
/**
|
||||
* check-snapshot-guard.mjs — every catalogue fetcher must write through the snapshot guard.
|
||||
*
|
||||
* WHY. A fetcher writes a snapshot built from a NETWORK read, and a network read has a bad
|
||||
* day. The dangerous case is not the run that throws — that leaves the old file alone — it
|
||||
* is the run that half-succeeds and writes anyway. Measured in this bundle: the FCBH fetcher
|
||||
* read video scope for 1,666 languages, was committed, then a later run during a rate-limit
|
||||
* produced 1,103 and silently overwrote the good file. Nothing failed and nothing warned.
|
||||
*
|
||||
* scripts/_snapshot-guard.mjs refuses a write that loses more than a tenth of the previous
|
||||
* snapshot. This check exists so the next fetcher someone adds cannot quietly skip it: a
|
||||
* guard that is easy to bypass is a guard that will be bypassed.
|
||||
*
|
||||
* THE ONE ALLOWED EXCEPTION is a checkpoint write in a resumable fetcher, which starts from
|
||||
* the existing file as a cache and can only grow. It must say so on the line above, with the
|
||||
* marker below, so the exemption is a deliberate statement rather than an oversight.
|
||||
*
|
||||
* Self-test: --poison injects a raw write and confirms this check fails on it.
|
||||
*/
|
||||
import { readFileSync, readdirSync } from 'node:fs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url))
|
||||
const DIR = resolve(HERE, '..', 'plugin-designs/gospel-resources/scripts')
|
||||
const POISON = process.argv.includes('--poison')
|
||||
const MARKER = 'CHECKPOINT-WRITE-OK'
|
||||
|
||||
const files = readdirSync(DIR).filter((f) => /^fetch-.*\.mjs$/.test(f))
|
||||
if (!files.length) {
|
||||
console.error('SNAPSHOT-GUARD-CHECK: FAIL — no fetch-*.mjs found; this check would pass vacuously.')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const bad = []
|
||||
let guarded = 0
|
||||
let exempt = 0
|
||||
for (const f of files) {
|
||||
let src = readFileSync(resolve(DIR, f), 'utf8')
|
||||
if (POISON && f === files[0]) src += '\nwriteFileSync(OUT, JSON.stringify(out))\n'
|
||||
const lines = src.split('\n')
|
||||
let usesGuard = false
|
||||
lines.forEach((line, i) => {
|
||||
if (/writeSnapshot\s*\(/.test(line)) usesGuard = true
|
||||
if (!/writeFileSync\s*\(\s*OUT\b/.test(line)) return
|
||||
// a checkpoint write must be declared on one of the three lines above it
|
||||
const above = lines.slice(Math.max(0, i - 4), i).join('\n')
|
||||
if (above.includes(MARKER)) { exempt++; return }
|
||||
bad.push(`${f}:${i + 1} writes OUT directly with writeFileSync and carries no ${MARKER} marker`)
|
||||
})
|
||||
if (usesGuard) guarded++
|
||||
else bad.push(`${f} never calls writeSnapshot() — its snapshot is unguarded`)
|
||||
}
|
||||
|
||||
if (bad.length) {
|
||||
console.error(`SNAPSHOT-GUARD-CHECK: FAIL — ${bad.length} problem(s):`)
|
||||
for (const b of bad) console.error(` ${b}`)
|
||||
if (POISON) {
|
||||
console.log('SNAPSHOT-GUARD-CHECK: SELF-TEST PASS — the injected raw write was caught.')
|
||||
process.exit(0)
|
||||
}
|
||||
process.exit(1)
|
||||
}
|
||||
if (POISON) {
|
||||
console.error('SNAPSHOT-GUARD-CHECK: SELF-TEST FAIL — an injected raw write was NOT caught; this check proves nothing.')
|
||||
process.exit(1)
|
||||
}
|
||||
console.log(`SNAPSHOT-GUARD-CHECK: PASS — ${guarded}/${files.length} fetcher(s) write through the guard; ${exempt} declared checkpoint write(s).`)
|
||||
|
|
@ -54,7 +54,7 @@
|
|||
"check:facet-changes-map:self-test": "node internal/check-facet-changes-map.mjs --self-test # reverts the fix IN THE LIVE PAGE and requires the gate to FAIL. SELF_TEST_MODE=colour (default, isolates the colour assertion — the reported bug) | props | both | frozen (blocks the source update, isolating the set assertion). All four modes verified caught.",
|
||||
"check:heartbeat": "node internal/check-heartbeat-threshold.mjs # the HEARTBEAT ZONE threshold (the choropleth's top bin) must come from config AND the shipped config must still render today's ramp. Two sides: (B) a configured threshold must MOVE the edge — catches re-hard-coding; (A2) dashboard.config.js's own map.* must still resolve to 5/20/60/150/Infinity — catches the shipped default silently changing. A2 was MISSING at first and heartbeatThreshold:999 shipped past this gate printing 'default ramp unchanged'. A grep for '150' cannot do either job: it false-positives on the legitimate default and false-negatives on a re-hard-coding at 200. run check:heartbeat:self-test for the NEGATIVE tests (3 cases).",
|
||||
"check:heartbeat:self-test": "node internal/check-heartbeat-threshold.mjs --self-test # THREE regression cases, each required to fire ITS OWN assertion: [literal] resolver replaced by the old hard-coded array -> B; [shipped-default] dashboard.config.js heartbeatThreshold changed -> A2; [shipped-mode] heartbeatMode flipped to quantile -> A2. Poisons IN MEMORY, never the file. Testing that the FEATURE works is not testing that the REGRESSION is caught — this gate shipped with only the first and missed the second.",
|
||||
"build:unlocked": "bun run check:template-literal && bun run check:mjs-parse && bun run check:coverage-consistent && bun run check:sfc-parse && bun run check:tab-radius && bun run check:synced-css && bun run check:wp-embed && bun run check:doc-scripts && bun run library-index && bun run check:library-index && bun run clean && bun run token && for d in plugin-designs/*/; do BUNDLE=$(basename \"$d\") bun x vite build || exit 1; done && bun run tree && bun run check:drop && bun run check:parse && bun run verify && bun run check:sri && bun run check:public-hygiene && bun run check:label-joins && bun run check:facet-cascade && bun run check:facet-one-field && bun run check:heartbeat && bun run check:kpi-unit && bun run check:composite-reg && bun run check:descriptions && bun run check:no-abs-paths && bun run check:no-codename && bun run check:dep-names && bun run check:composite-srcs && bun run check:embed-targets && bun run check:nested-token && bun run gallery",
|
||||
"build:unlocked": "bun run check:template-literal && bun run check:mjs-parse && bun run check:coverage-consistent && bun run check:snapshot-guard && bun run check:sfc-parse && bun run check:tab-radius && bun run check:synced-css && bun run check:wp-embed && bun run check:doc-scripts && bun run library-index && bun run check:library-index && bun run clean && bun run token && for d in plugin-designs/*/; do BUNDLE=$(basename \"$d\") bun x vite build || exit 1; done && bun run tree && bun run check:drop && bun run check:parse && bun run verify && bun run check:sri && bun run check:public-hygiene && bun run check:label-joins && bun run check:facet-cascade && bun run check:facet-one-field && bun run check:heartbeat && bun run check:kpi-unit && bun run check:composite-reg && bun run check:descriptions && bun run check:no-abs-paths && bun run check:no-codename && bun run check:dep-names && bun run check:composite-srcs && bun run check:embed-targets && bun run check:nested-token && bun run gallery",
|
||||
"check:template-literal": "node internal/template-literal-check.mjs",
|
||||
"verify:map-paint": "node internal/verify-map-paint.mjs",
|
||||
"verify:gospel-map": "node internal/verify-gospel-map.mjs # the gospel-resources MAP: pins render (not merely load), a tap opens a named group, EVERY link in the index is on a screened publisher, the table deep-link carries the group's ISO, and the gap filter leaves only tier 0. Needs a server (bun run serve); SKIPS loudly without one.",
|
||||
|
|
@ -122,6 +122,7 @@
|
|||
"check:spa-tab-order": "node internal/check-spa-tab-order.mjs",
|
||||
"check:backlog-status": "node internal/check-backlog-status-enum.mjs",
|
||||
"check:mjs-parse": "node internal/check-mjs-parses.mjs",
|
||||
"check:snapshot-guard": "node internal/check-snapshot-guard.mjs # every gospel-resources catalogue fetcher must write through scripts/_snapshot-guard.mjs, which refuses a write that loses more than a tenth of the previous snapshot. Written after a rate-limited FCBH run silently replaced 1,666 video scopes with 1,103. Static, sub-second. Self-test: --poison injects a raw write and the check must fail on it.",
|
||||
"check:coverage-consistent": "node internal/check-gospel-coverage-consistent.mjs # the gospel-resources surfaces must agree: no resource on an ISO RESERVED code (und/mul/mis/zxx), every resourced code NAMED, table reach recounted off the pins, no invented URL, tiers reconcile. Static, sub-second. Self-test: --poison, 6/6 clauses must fail. Written after the same drift defect was found by hand three times.",
|
||||
"check:synced-css:self-test": "node internal/check-synced-css-copies.mjs --self-test",
|
||||
"serve:permissive": "python3 -m http.server 4173 --bind 127.0.0.1 --directory public # LOOPBACK ONLY. This was --bind 0.0.0.0, which published the built tree on every interface of whatever machine ran a gate; measured on the build VPS it answered HTTP 200 on the box's PUBLIC IP while a browser gate was running. Every consumer (verify-browser-all, check:a11y-button-names, check:nav-reachable, smoke:share-all) already defaults to 127.0.0.1, so nothing needed the wider bind. Override with BASE_URL/BASE if you deliberately want to reach it from another host.",
|
||||
|
|
|
|||
68
plugin-designs/gospel-resources/scripts/_snapshot-guard.mjs
Normal file
68
plugin-designs/gospel-resources/scripts/_snapshot-guard.mjs
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
/**
|
||||
* _snapshot-guard.mjs — refuse to replace a good catalogue snapshot with a worse one.
|
||||
*
|
||||
* THE FAILURE THIS EXISTS FOR, MEASURED, NOT IMAGINED. The FCBH fetcher read per-fileset
|
||||
* video scope for 1,666 languages and that file was committed. A later run of the same
|
||||
* script, while live.bible.is was rate-limiting, got 1,103 — and overwrote the good file
|
||||
* with the degraded one. Nothing failed, nothing warned, and the artefact silently lost a
|
||||
* third of its content. It was caught only because `git status` happened to be read.
|
||||
*
|
||||
* It is the third time this shape has appeared in this bundle:
|
||||
* - a VERIFY_LIMIT smoke run replaced the full verification table with 30 rows;
|
||||
* - dropping a dead link removed it from the corpus, so the next run forgot it was dead
|
||||
* and the build put it straight back;
|
||||
* - this one.
|
||||
* Every snapshot here is built from a NETWORK read, and a network read has a bad day. The
|
||||
* dangerous case is not the run that throws — that leaves the old file alone — it is the
|
||||
* run that half-succeeds and writes anyway.
|
||||
*
|
||||
* THE RULE: a snapshot may grow freely, and may shrink a little, but a large drop is
|
||||
* treated as a failed run rather than as news. The count compared is whatever the caller
|
||||
* says identifies the snapshot's content (languages, codes, entries).
|
||||
*
|
||||
* Deliberately NOT a hard equality check: catalogues genuinely do lose entries when a
|
||||
* publisher retires something, and a gate nobody can satisfy gets switched off. The
|
||||
* threshold is a floor on the fraction retained, and it is overridable in one obvious,
|
||||
* loud way for the case where the drop is real.
|
||||
*/
|
||||
import { readFileSync, writeFileSync } from 'node:fs'
|
||||
|
||||
/** Default: a run that lost more than 10% of the previous snapshot is not trusted. */
|
||||
const FLOOR = Number(process.env.SNAPSHOT_FLOOR || 0.9)
|
||||
const FORCE = process.env.SNAPSHOT_FORCE === '1'
|
||||
|
||||
/**
|
||||
* @param {string} out absolute path to write
|
||||
* @param {string} body the serialised snapshot
|
||||
* @param {(json:any)=>number} countOf how to count what matters in this snapshot
|
||||
* @param {string} label what the count is, for the message
|
||||
*/
|
||||
export function writeSnapshot(out, body, countOf, label) {
|
||||
const next = countOf(JSON.parse(body))
|
||||
let prev = null
|
||||
try { prev = countOf(JSON.parse(readFileSync(out, 'utf8'))) } catch { prev = null }
|
||||
|
||||
if (prev !== null && prev > 0 && next < prev * FLOOR) {
|
||||
const lost = prev - next
|
||||
const pct = ((100 * lost) / prev).toFixed(1)
|
||||
if (!FORCE) {
|
||||
console.error(
|
||||
`SNAPSHOT GUARD: REFUSING TO WRITE ${out}\n` +
|
||||
` this run produced ${next} ${label}; the file on disk has ${prev}.\n` +
|
||||
` That is ${lost} fewer (${pct}%), below the ${(FLOOR * 100).toFixed(0)}% floor.\n` +
|
||||
` A half-finished network read looks exactly like this, and overwriting would\n` +
|
||||
` destroy good data silently. The existing file is UNCHANGED.\n` +
|
||||
` If the drop is real (a publisher retired entries), rerun with SNAPSHOT_FORCE=1.`,
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
console.warn(`SNAPSHOT GUARD: ${next} ${label} vs ${prev} on disk (${pct}% lost) — writing anyway because SNAPSHOT_FORCE=1.`)
|
||||
writeFileSync(out, body)
|
||||
return
|
||||
}
|
||||
|
||||
writeFileSync(out, body)
|
||||
if (prev === null) console.log(`SNAPSHOT GUARD: no previous ${out} to compare; wrote ${next} ${label}.`)
|
||||
else if (next < prev) console.log(`SNAPSHOT GUARD: ${next} ${label} (was ${prev}) — a small drop, within the floor.`)
|
||||
else console.log(`SNAPSHOT GUARD: ${next} ${label} (was ${prev}).`)
|
||||
}
|
||||
|
|
@ -45,7 +45,7 @@
|
|||
* Inside the filter, and named explicitly in the standing instruction. Only featureFilm is
|
||||
* taken; Arclight also carries short films and other media that are not screened here.
|
||||
*/
|
||||
import { writeFileSync } from 'node:fs'
|
||||
import { writeSnapshot } from './_snapshot-guard.mjs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
|
|
@ -82,12 +82,12 @@ for (const r of all) {
|
|||
}
|
||||
stats.kept = Object.keys(codes).length
|
||||
|
||||
writeFileSync(OUT, JSON.stringify({
|
||||
writeSnapshot(OUT, JSON.stringify({
|
||||
_note: 'Arclight (JESUS Film Project) catalogue by ISO 639-3. Only languages with counts.featureFilm >= 1. Existence is established from the API: the /watch/ URL answers 200 for ANY id and cannot verify anything.',
|
||||
generated: new Date().toISOString().slice(0, 10),
|
||||
source: API,
|
||||
stats,
|
||||
codes,
|
||||
}))
|
||||
}), (j) => Object.keys(j.codes || {}).length, 'Arclight ISO 639-3 codes')
|
||||
console.log(JSON.stringify(stats))
|
||||
console.log(`${stats.kept} ISO 639-3 codes -> ${OUT}`)
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@
|
|||
* authoritative, so it is asked. Rate-limited, resumable, and cached so it is asked once.
|
||||
*/
|
||||
import { readFileSync, writeFileSync, existsSync } from 'node:fs'
|
||||
import { writeSnapshot } from './_snapshot-guard.mjs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
|
|
@ -49,16 +50,21 @@ for (let i = 0; i < entries.length; i++) {
|
|||
if (u) { out[code] = u; fetched++ } else missing++
|
||||
} else missing++
|
||||
} catch { missing++ }
|
||||
// The CHECKPOINT write is deliberately unguarded: this script resumes from the existing
|
||||
// file as a cache, so `out` starts full and only grows. It cannot shrink, which is the
|
||||
// thing the guard exists to catch. The FINAL write below is guarded like every other
|
||||
// snapshot.
|
||||
// CHECKPOINT-WRITE-OK
|
||||
if (fetched % 100 === 0 && fetched) {
|
||||
writeFileSync(OUT, JSON.stringify({ _note: 'Per-language JESUS Film deep links (shareUrl) from Arclight media-components. Partial write; rerun to resume.', generated: new Date().toISOString().slice(0, 10), codes: out }))
|
||||
console.log(` ... ${i + 1}/${entries.length} fetched=${fetched} cached=${cached} missing=${missing}`)
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, DELAY))
|
||||
}
|
||||
writeFileSync(OUT, JSON.stringify({
|
||||
writeSnapshot(OUT, JSON.stringify({
|
||||
_note: 'Per-language JESUS Film deep links (shareUrl) from Arclight media-components 1_jf-0-0. Replaces ?languageId=, which the site ignores.',
|
||||
generated: new Date().toISOString().slice(0, 10),
|
||||
stats: { languages: entries.length, withShareUrl: Object.keys(out).length, missing },
|
||||
codes: out,
|
||||
}))
|
||||
}), (j) => Object.keys(j.codes || {}).length, 'Arclight shareUrl codes')
|
||||
console.log(`DONE languages ${entries.length} shareUrl ${Object.keys(out).length} missing ${missing}`)
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@
|
|||
* ('es-419', 'pt-br', 'sr-latn', 'ur-deva', 'zh-hant') are NOT private use and are the same
|
||||
* language, so they still fold to the base.
|
||||
*/
|
||||
import { writeFileSync } from 'node:fs'
|
||||
import { writeSnapshot } from './_snapshot-guard.mjs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
|
|
@ -148,6 +148,6 @@ const out = {
|
|||
sources,
|
||||
codes: index,
|
||||
}
|
||||
writeFileSync(OUT, JSON.stringify(out, null, 1))
|
||||
writeSnapshot(OUT, JSON.stringify(out, null, 1), (j) => Object.keys(j.codes || {}).length, 'Door43 ISO 639-3 codes')
|
||||
console.log(`\n${Object.keys(index).length} ISO 639-3 codes -> ${OUT}`)
|
||||
console.log(`${viaPart1} entr(ies) joined through an ISO 639-1 tag; ${skippedPrivate} private-use tag(s) skipped`)
|
||||
|
|
|
|||
|
|
@ -28,7 +28,7 @@
|
|||
* ONE TRANSLATION PER LANGUAGE: the one with the most canonical books, tie-broken by verses.
|
||||
* A language with a full Bible should not be represented by a single-gospel portion.
|
||||
*/
|
||||
import { writeFileSync } from 'node:fs'
|
||||
import { writeSnapshot } from './_snapshot-guard.mjs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
|
|
@ -144,6 +144,6 @@ const out = {
|
|||
codes: best,
|
||||
byUrl,
|
||||
}
|
||||
writeFileSync(OUT, JSON.stringify(out))
|
||||
writeSnapshot(OUT, JSON.stringify(out), (j) => Object.keys(j.codes || {}).length, 'eBible ISO 639-3 codes')
|
||||
console.log(JSON.stringify(stats))
|
||||
console.log(`${stats.kept} ISO 639-3 codes -> ${OUT}`)
|
||||
|
|
|
|||
|
|
@ -64,7 +64,7 @@
|
|||
* Testament narrative is also not the gospel, so counting it toward the Scripture+film tier
|
||||
* would overstate what a group has. It lifts a group off "nothing found", and no further.
|
||||
*/
|
||||
import { writeFileSync } from 'node:fs'
|
||||
import { writeSnapshot } from './_snapshot-guard.mjs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
|
|
@ -196,7 +196,7 @@ for (const [iso, b] of Object.entries(best)) {
|
|||
codes[iso] = { u: `${SITE}/${b.id}`, n: b.name || 'Scripture', ln: b.lang, id: b.id, date: b.date, m: b.media.join(', ') }
|
||||
}
|
||||
|
||||
writeFileSync(
|
||||
writeSnapshot(
|
||||
OUT,
|
||||
`${JSON.stringify(
|
||||
{
|
||||
|
|
@ -211,6 +211,9 @@ writeFileSync(
|
|||
null,
|
||||
1,
|
||||
)}\n`,
|
||||
// Both halves matter: a bad run can lose Scripture codes, video scopes, or both.
|
||||
(j) => Object.keys(j.codes || {}).length + Object.keys(j.videoCodes || {}).length,
|
||||
'FCBH code+video entries',
|
||||
)
|
||||
|
||||
console.log(`FCBH: ${rows.length} rows -> ${Object.keys(codes).length} ISO 639-3 codes with Scripture (audio and/or text)`)
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@
|
|||
* an unclear mapping is left out rather than guessed: the locale still ships with its
|
||||
* verified URL, it simply carries no ISO code and so attaches to no people group.
|
||||
*/
|
||||
import { writeFileSync } from 'node:fs'
|
||||
import { writeSnapshot } from './_snapshot-guard.mjs'
|
||||
import { resolve, dirname } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
|
|
@ -236,7 +236,7 @@ for (const [locale, a] of Object.entries(APPROX)) {
|
|||
const unjoined = entries.filter((e) => !e.iso).map((e) => e.locale)
|
||||
const dead = entries.filter((e) => !e.url).map((e) => e.locale)
|
||||
|
||||
writeFileSync(
|
||||
writeSnapshot(
|
||||
OUT,
|
||||
`${JSON.stringify(
|
||||
{
|
||||
|
|
@ -253,6 +253,8 @@ writeFileSync(
|
|||
null,
|
||||
1,
|
||||
)}\n`,
|
||||
(j) => Object.keys(j.byIso || {}).length,
|
||||
'Zume ISO 639-3 codes',
|
||||
)
|
||||
|
||||
console.log(
|
||||
|
|
|
|||
Loading…
Reference in a new issue