Cloud CDN build: real Node in the image, the build steps without the gates, and no python3
The first build in the container hung for 18+ minutes: the image had only bun's node shim, and `node --check` under that shim runs each script instead of parsing it. The gates also already passed before a commit reaches main, so the container now runs clean, token, bundles, tree and check:sri only. The public exclusions and the refuse-the-swap check still run on every build. runtime-config.js was written with python3, which the image does not have; bun writes it now. The build's PID goes to /tmp/build-cdn.pid so it can be stopped without touching the container.
This commit is contained in:
parent
a5759491d0
commit
744c3be317
2 changed files with 15 additions and 5 deletions
|
|
@ -7,6 +7,9 @@ FROM oven/bun:1-debian
|
|||
RUN apt-get update && apt-get install -y --no-install-recommends git ca-certificates curl unzip \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY --from=oauth2proxy /bin/oauth2-proxy /usr/local/bin/oauth2-proxy
|
||||
# Real Node for the repo's `node internal/...` scripts: bun's node shim runs `node --check` files
|
||||
# instead of only parsing them, which hung the production build.
|
||||
COPY --from=node:22-bookworm-slim /usr/local/bin/node /usr/local/bin/node
|
||||
# Go, to build Rudder's local engine (backend/rudder-engine, go 1.22) when RUDDER_ENGINE=1.
|
||||
COPY --from=golang:1.22-bookworm /usr/local/go /usr/local/go
|
||||
ENV PATH="/usr/local/go/bin:${PATH}"
|
||||
|
|
|
|||
|
|
@ -1,11 +1,16 @@
|
|||
#!/bin/bash
|
||||
# build-cdn.sh — one production build of the pulled commit into /data/cdn, one at a time.
|
||||
# snapshot (so the pull loop can't change files mid-build) -> apply the public exclusions ->
|
||||
# BUILD_ALLOW=1 bun run build (memory-capped) -> check the output -> atomic swap of /data/cdn/current.
|
||||
# Exit 75 = another build holds the lock.
|
||||
# the build steps WITHOUT the gates (memory-capped) -> check the output -> the exclusion check again
|
||||
# (refuses the swap) -> atomic swap of /data/cdn/current.
|
||||
# No gates here: they already passed before the commit reached main, and a full gate run on a
|
||||
# 2-vCPU container takes most of an hour. The public exclusions are NOT a gate and always run.
|
||||
# Exit 75 = another build holds the lock. To stop a build, kill the PID in /tmp/build-cdn.pid
|
||||
# (never a process group: inside a container the group can be PID 1, i.e. the whole container).
|
||||
set -u
|
||||
exec 9>/tmp/build-cdn.lock
|
||||
flock -n 9 || { echo "another build is running"; exit 75; }
|
||||
echo $$ > /tmp/build-cdn.pid
|
||||
cd /app/repo
|
||||
SHA=$(git rev-parse --short HEAD); SNAP=/tmp/build-$SHA
|
||||
rm -rf "$SNAP"; mkdir -p "$SNAP"
|
||||
|
|
@ -14,11 +19,13 @@ bash /app/repo/deployment/cloud/apply-exclusions.sh "$SNAP"
|
|||
ln -s /app/repo/node_modules "$SNAP/node_modules"
|
||||
cd "$SNAP"
|
||||
echo "$(date -u +%FT%TZ) building $SHA"
|
||||
NODE_OPTIONS="--max-old-space-size=${BUILD_MAX_OLD_SPACE_MB:-1536}" BUILD_ALLOW=1 nice -n 10 bun run build \
|
||||
|| { echo "build failed"; rm -rf "$SNAP"; exit 1; }
|
||||
export NODE_OPTIONS="--max-old-space-size=${BUILD_MAX_OLD_SPACE_MB:-1536}"
|
||||
for step in clean token build:bundles tree check:sri; do
|
||||
nice -n 10 bun run "$step" || { echo "build failed at $step"; rm -rf "$SNAP"; exit 1; }
|
||||
done
|
||||
OUT="$SNAP/public/dx-maps"
|
||||
[ -f "$OUT/manifest.json" ] || { echo "no manifest.json in the output"; rm -rf "$SNAP"; exit 1; }
|
||||
printf 'window.MAP_API_BASE=%s;\n' "$(printf '%s' "${PUBLIC_API_BASE:-}" | python3 -c 'import json,sys;print(json.dumps(sys.stdin.read()))')" > "$OUT/runtime-config.js"
|
||||
bun -e 'console.log("window.MAP_API_BASE=" + JSON.stringify(process.env.PUBLIC_API_BASE || "") + ";")' > "$OUT/runtime-config.js"
|
||||
NEW=/data/cdn/build-$SHA
|
||||
rm -rf "$NEW"; mkdir -p /data/cdn; mv "$SNAP/public" "$NEW"
|
||||
bash /app/repo/deployment/cloud/apply-exclusions.sh --check "$NEW" || { echo "REFUSED: excluded content in the build; not published"; rm -rf "$NEW" "$SNAP"; exit 1; }
|
||||
|
|
|
|||
Loading…
Reference in a new issue