Chart the corpus the map draws, not a sixth of it

charts.vue aggregated people-group-resources — 862 languages — while the map beside it
draws 5,067. The charts described a sixth of the corpus and looked authoritative doing it.
Same defect the country sheet had at tick 13, listing 36 groups for Indonesia against 669
pins, and found the same way: by asking which table a surface actually reads.

Now built from the pin index, and every number cross-checked against an independent count
of the same data — 5,067 languages, Scripture 1,036, Full Bible 121, New Testament 1,025,
Film 1,852, Open Bible Stories 101, nothing found 2,758. Six of seven matched exactly.

★ THE SEVENTH DIFFERED BY 2,330,000 AND THAT IS THE INTERESTING ONE. My count said
347,030,459 people behind the gap; the chart said 344,700,459. The difference is exactly
"Arab, Gazan" — the single group with NO ISO 639-3 code. A per-LANGUAGE chart cannot count
a group that has no language, which is correct, but it left the charts and the gaps table
quietly disagreeing by 2.33M. The hero line now says so and points at the table that does
include them. A discrepancy named is a caveat; unnamed it is a contradiction someone finds
later and cannot explain.

THE FOOTER SAID "Pools CHECKED: unknown." over a corpus where we know exactly what was
searched — it was reading meta.sources from the old table. Derived from the codes instead:
"Pools CHECKED: ebible, arclight, door43. Pools NOT checked: grn, api_bible, dbl,
progress_bible." The same understatement the gaps table carried until tick 30.

Audio still renders as not-measured rather than as a zero bar, which is the honest reading
of a pool nobody has looked in — the guard that protects it was rewritten for the new
shape rather than dropped with the old one.

YOUVERSION / BIBLE.COM IS BLOCKED, tried because the brief names it: /languages, /versions
and /api/bible/configuration all return an identical 3,038-byte "Client Challenge" page,
and robots.txt returns that challenge instead of a robots file. Recorded beside GRN as
needing an agreement rather than a scraper.

One self-inflicted note: my edit left an orphaned continuation line and check:sfc-parse
caught it immediately with the right file and line. The build would have failed far from
the edit, which is the failure that gate exists to replace.

Verified: coverage gate, map gate, dashboard and table probes all pass, and the rendered
pools line names all seven pools rather than "unknown".
This commit is contained in:
doxa-maps-88655c70 2026-09-24 09:01:37 +00:00
parent e9338a8757
commit 48bc645c67
10 changed files with 122 additions and 65 deletions

View file

@ -45,6 +45,12 @@ and it needs a credential or an agreement, not a scraper.**
## Not a source at all
- **sign.bible** — a parked domain listed for sale on Epik. Answers 200.
## YouVersion / bible.com — BLOCKED by a bot wall
Named in the brief and tried. www.bible.com/languages, /versions and /api/bible/configuration
all return an IDENTICAL 3,038-byte page titled "Client Challenge", and robots.txt itself
returns that challenge HTML rather than a robots file — so their crawl policy cannot be read.
Same shape as GRN. Needs an API agreement, not a scraper.
## zume.training — no list obtainable
`/api/` is `Disallow`ed in their robots.txt, and the homepage exposes no language switcher
even when rendered. The ~40-language list has to come from them.

View file

@ -97,11 +97,15 @@ async function fetchTable(file) {
async function load() {
loading.value = true; loadErr.value = null
try {
const [resources, byCountry] = await Promise.all([
fetchTable('people-group-resources'),
// people-group-pins is the index the MAP draws from: 5,447 language codes against the
// 862 in people-group-resources. Charting the smaller table meant these charts
// described a sixth of the corpus the map beside them shows — the same defect the
// country sheet had at tick 13, where it listed 36 groups for Indonesia against 669 pins.
const [pins, byCountry] = await Promise.all([
fetchTable('people-group-pins'),
fetchTable('languages-by-country'),
])
tables.value = { resources, byCountry }
tables.value = { pins, byCountry }
} catch (e) { loadErr.value = String(e.message || e) }
finally { loading.value = false }
}
@ -115,43 +119,65 @@ watch(profileConfig, load, { immediate: true })
const splitCountries = (s) => String(s || '').split(',').map((x) => x.trim()).filter(Boolean)
const languages = computed(() => {
const rows = tables.value.resources?.rows || []
// One entry per LANGUAGE, built from the pin index: codes[] carries what exists, pins[]
// carries who speaks it and where. Same shape as before so every chart below is unchanged.
const d = tables.value.pins
if (!d || !d.codes) return []
const { codes, pins } = d
const by = new Map()
for (const r of rows) {
const iso = r.iso639_3
if (!iso) continue // never invent a code
let e = by.get(iso)
for (const [, , , iso3, code, pop] of pins) {
if (!code) continue
let e = by.get(code)
if (!e) {
const c = codes[code] || {}
const obs = (c.d || []).some((x) => /^Open Bible Stories/.test(x.s || ''))
const scope = c.ss || ''
e = {
iso, name: r.language || 'unknown', groups: 0, population: 0,
scripture: false, fullBible: false, nt: false, film: false, jesusFilm: false,
obs: false, audioKnown: false, none: true, unmatchable: false,
iso: code,
name: c.l || 'unknown',
groups: 0,
population: 0,
scripture: !!c.s,
fullBible: scope === 'Full Bible',
// "New Testament" and "Portions of both Testaments" both contain NT books; a
// Full Bible does too. Counting only the exact NT label would undercount.
nt: scope === 'Full Bible' || /New Testament|both Testaments/.test(scope),
film: !!c.f,
jesusFilm: !!c.f,
obs,
// has_audio is unknown for EVERY code in this build — GRN has never been searched.
// The chart renders that as "unknown" rather than as a zero, which is the honest
// reading of a pool nobody has looked in.
audioKnown: c.a === 1,
none: !c.s && !c.f && !(c.d && c.d.length),
unmatchable: !code,
countries: new Set(),
}
by.set(iso, e)
by.set(code, e)
}
e.groups += 1
e.population += Number(r.population) || 0
e.scripture ||= r.has_scripture === true
e.fullBible ||= r.full_bible === true
e.nt ||= r.new_testament === true
e.film ||= r.has_film === true
e.jesusFilm ||= r.jesus_film === true
e.obs ||= r.has_open_bible_stories === true
// has_audio is the STRING "unknown" everywhere in this build — only a literal
// true counts as a known-found audio resource.
e.audioKnown ||= r.has_audio === true
e.none = e.none && r.no_resource_found === true
e.unmatchable ||= r.code_unmatchable === true
for (const c of splitCountries(r.countries)) e.countries.add(c)
e.population += Number(pop) || 0
if (iso3) e.countries.add(iso3)
}
return Array.from(by.values())
})
const meta = computed(() => tables.value.resources?.meta || {})
const checkedPools = computed(() => (meta.value.sources || []).map((s) => s.name).join(', ') || 'unknown')
const uncheckedPools = computed(() => (meta.value.not_included || meta.value.unchecked || [])
.map((s) => `${s.name} (${s.covers})`).join('; ') || 'unknown')
const meta = computed(() => tables.value.pins?.meta || {})
// The pin index records per code which pools were searched, so this is derivable rather
// than unknown. It read meta.sources from the old table, which the pin index does not
// carry, and rendered "Pools CHECKED: unknown" over a corpus where we know exactly what
// was searched — the same understatement the gaps table carried until tick 30.
const poolSets = computed(() => {
const codes = tables.value.pins?.codes || {}
const ck = new Set(), un = new Set()
for (const v of Object.values(codes)) {
for (const x of String(v.ck || '').split(',')) if (x.trim()) ck.add(x.trim())
for (const x of String(v.un || '').split(',')) if (x.trim()) un.add(x.trim())
}
return { ck: [...ck], un: [...un] }
})
const checkedPools = computed(() => poolSets.value.ck.join(', ') || 'unknown')
const uncheckedPools = computed(() => poolSets.value.un.join(', ') || 'unknown')
// The caveat block every chart carries — the pools, and what a missing value means.
const poolCaveats = computed(() => [
@ -161,7 +187,7 @@ const poolCaveats = computed(() => [
])
const nLang = computed(() => languages.value.length)
const nGroups = computed(() => (tables.value.resources?.rows || []).length)
const nGroups = computed(() => (tables.value.pins?.pins || []).length)
const count = (fn) => languages.value.filter(fn).length
// ═══ A FIELD THAT IS NOT IN THE TABLE IS NOT A ZERO ═════════════════════════
@ -172,10 +198,16 @@ const count = (fn) => languages.value.filter(fn).length
// a version behind. So a metric whose FIELD is absent renders as not-measured,
// exactly like a pool that was never queried.
const hasField = (f) => {
const t = tables.value.resources
if (!t) return false
if ((t.columns || []).some((c) => c.field === f)) return true
return !!t.rows?.length && f in t.rows[0]
// Same rule against the pin index, which stores what exists on codes[] rather than as
// table columns: a metric is MEASURED only if at least one code carries the field. For
// audio that is deliberately false — has_audio is unknown for every code in this build,
// because GRN has never been searched — so the bar renders as not-measured rather than
// as a zero claiming no language anywhere has audio.
const codes = tables.value.pins?.codes
if (!codes) return false
const key = f === 'has_audio' ? 'a' : f === 'has_open_bible_stories' ? 'd' : f
for (const v of Object.values(codes)) if (v && key in v) return true
return false
}
const C = {
@ -225,6 +257,20 @@ const topNone = computed(() => languages.value
meta: l,
})))
const nNone = computed(() => count((l) => l.none))
// A per-LANGUAGE chart cannot count a group that has no language code, so these totals
// leave one out. Named rather than left as a silent 2.33M discrepancy against the gaps
// table, which is keyed per people group and does include it.
const noCodePeople = computed(() => {
const d = tables.value.pins
if (!d) return 0
return (d.pins || []).reduce((n, r) => (r[4] ? n : n + (Number(r[5]) || 0)), 0)
})
const noCodeGroups = computed(() => {
const d = tables.value.pins
if (!d) return 0
return (d.pins || []).reduce((n, r) => (r[4] ? n : n + 1), 0)
})
const popNone = computed(() => languages.value.filter((l) => l.none)
.reduce((n, l) => n + l.population, 0))
@ -313,6 +359,11 @@ useShadowStyles(
<div v-if="nLang" class="gc-hero">
<b>{{ fmt(nNone) }}</b> of {{ fmt(nLang) }} languages have <b>no scripture and no film
found in the pools checked</b> — the languages of <b>{{ fmt(popNone) }}</b> people.
<template v-if="noCodeGroups">
<br><small>Counted BY LANGUAGE, so {{ fmt(noCodeGroups) }} people group{{ noCodeGroups === 1 ? '' : 's' }}
with no ISO 639-3 code in the source — {{ fmt(noCodePeople) }} people — cannot appear here.
The "Where Nothing Was Found" table is keyed per people group and does include them.</small>
</template>
<br>Not found is not the same as not existing: four resource pools, including all
audio, were never queried.
</div>

View file

@ -26,7 +26,7 @@
<script>window.MAP_APP_API_URL = "https://pray.doxa.life";</script>
<!-- Bundle IIFE — registers the <gospel-resources-map> custom element. -->
<script src="../gospel-resources.js" integrity="sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI" crossorigin="anonymous"></script>
<script src="../gospel-resources.js" integrity="sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3" crossorigin="anonymous"></script>
<script>
// Mapbox token resolution — the A3 ladder, in precedence order:

View file

@ -26,7 +26,7 @@
<script>window.MAP_APP_API_URL = "https://pray.doxa.life";</script>
<!-- Bundle IIFE — registers the <gospel-resources-map> custom element. -->
<script src="../gospel-resources.js" integrity="sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI" crossorigin="anonymous"></script>
<script src="../gospel-resources.js" integrity="sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3" crossorigin="anonymous"></script>
<script>
// Mapbox token resolution — the A3 ladder, in precedence order:

View file

@ -40,7 +40,7 @@
<script>window.MAP_APP_API_URL = "https://pray.doxa.life";</script>
<!-- Bundle IIFE — registers the <gospel-resources-map> custom element. -->
<script src="../gospel-resources.js" integrity="sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI" crossorigin="anonymous"></script>
<script src="../gospel-resources.js" integrity="sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3" crossorigin="anonymous"></script>
<script>
// Mapbox token resolution — the A3 ladder, in precedence order:

File diff suppressed because one or more lines are too long

View file

@ -26,7 +26,7 @@
<script>window.MAP_APP_API_URL = "https://pray.doxa.life";</script>
<!-- Bundle IIFE — registers the <gospel-resources-map> custom element. -->
<script src="../gospel-resources.js" integrity="sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI" crossorigin="anonymous"></script>
<script src="../gospel-resources.js" integrity="sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3" crossorigin="anonymous"></script>
<script>
// Mapbox token resolution — the A3 ladder, in precedence order:

View file

@ -29,7 +29,7 @@
<script>window.MAP_APP_API_URL = "https://pray.doxa.life";</script>
<!-- Bundle IIFE — registers the <gospel-resources-map> custom element. -->
<script src="../gospel-resources.js" integrity="sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI" crossorigin="anonymous"></script>
<script src="../gospel-resources.js" integrity="sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3" crossorigin="anonymous"></script>
<script>
// Mapbox token resolution — the A3 ladder, in precedence order:

File diff suppressed because one or more lines are too long

View file

@ -556,7 +556,7 @@
"bundle": "gospel-resources",
"scriptUrl": "./gospel-resources/gospel-resources.js",
"output": "doxa-maps/gospel-resources/gospel-resources.js",
"integrity": "sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI",
"integrity": "sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3",
"elements": [
"gospel-resources-map"
],
@ -576,7 +576,7 @@
"scriptUrl": "./gospel-resources/gospel-resources.js",
"embed": {
"iframe": "<iframe src=\"./gospel-resources/charts/index.html?utm_source=YOUR_SOURCE\" width=\"100%\" height=\"600\" style=\"border:none;border-radius:12px;\" allowfullscreen></iframe>",
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"charts\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"charts\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
}
},
{
@ -590,7 +590,7 @@
"scriptUrl": "./gospel-resources/gospel-resources.js",
"embed": {
"iframe": "<iframe src=\"./gospel-resources/data-tables/index.html?utm_source=YOUR_SOURCE\" width=\"100%\" height=\"600\" style=\"border:none;border-radius:12px;\" allowfullscreen></iframe>",
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"data-tables\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"data-tables\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
}
},
{
@ -604,7 +604,7 @@
"scriptUrl": "./gospel-resources/gospel-resources.js",
"embed": {
"iframe": "<iframe src=\"./gospel-resources/gospel-dashboard/index.html?utm_source=YOUR_SOURCE\" width=\"100%\" height=\"600\" style=\"border:none;border-radius:12px;\" allowfullscreen></iframe>",
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"gospel-dashboard\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"gospel-dashboard\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
}
},
{
@ -618,7 +618,7 @@
"scriptUrl": "./gospel-resources/gospel-resources.js",
"embed": {
"iframe": "<iframe src=\"./gospel-resources/resources-list/index.html?utm_source=YOUR_SOURCE\" width=\"100%\" height=\"600\" style=\"border:none;border-radius:12px;\" allowfullscreen></iframe>",
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"resources-list\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"resources-list\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
}
},
{
@ -632,7 +632,7 @@
"scriptUrl": "./gospel-resources/gospel-resources.js",
"embed": {
"iframe": "<iframe src=\"./gospel-resources/resources-map/index.html?utm_source=YOUR_SOURCE\" width=\"100%\" height=\"600\" style=\"border:none;border-radius:12px;\" allowfullscreen></iframe>",
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-UtBJUSgrgVQYbAEnxsDTts7yRZ44jTvVnDHkz8oGvjjFCG5vZjmbXhycapa5iqwI\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"resources-map\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
"script": "<script>window.MAP_APP_API_URL=\"https://pray.doxa.life\";</script>\n<script src=\"./gospel-resources/gospel-resources.js\" integrity=\"sha384-iZfkYQAqPPbbGZjfVRkboOX+xWDdyqqQEwDAQLk3iqyE4wMmrpMoDocigcm0Xyd3\" crossorigin=\"anonymous\"></script>\n<gospel-resources-map profile-config='{\"profile\":\"resources-map\",\"tk\":\"YOUR_MAPBOX_TOKEN\",\"utmSource\":\"YOUR_SOURCE\",\"assetsBase\":\"YOUR_TREE_URL/assets\"}'></gospel-resources-map>"
}
}
]