anthodev/lapin-fute · 91a4982

modified .github/RELEASE_CHECKLIST.md +3 -3 Viewed

@@ -8,7 +8,7 @@ This checklist tracks GitHub issue [#9](https://github.com/Anthodev/lapin-fute/i
8	8	- [x] Select `https://lapin-fute.b-cdn.net/` as the stable, static HTTPS configuration-site origin.
9	9	- [x] Build release PBWs with that exact origin and reject release tags that differ from the watch package version.
10	10	- [x] Package one PBW containing both `emery` and `gabbro`, plus `SHA256SUMS`, as GitHub release assets.
11		-- [x] Refresh a real IDFM catalog for production releases; no fixture fallback is available.
	11	+- [x] Keep release tags page-only; manual real-IDFM refreshes compare `sourceRevision` with production and upload the catalog only when its inputs changed or an operator forces it.
12	12	- [x] Upload immutable catalog files before the manifest pointer.
13	13	- [x] Compare every built file with its served bytes and validate every current-revision catalog JSON against the manifest schema and revision.
14	14	- [x] Back up mutable hosted files before deployment and restore them automatically when upload, purge, or served-site verification fails.
@@ -30,7 +30,7 @@ This checklist tracks GitHub issue [#9](https://github.com/Anthodev/lapin-fute/i
30	30	## Deployment and rollback rehearsal
31	31
32	32	1. Merge the reviewed release branch without tagging it.
33		-2. Run Deploy configuration site to Bunny manually with `refresh_catalog=true`.
	33	+2. Run Deploy configuration site to Bunny manually with `refresh_catalog=true`. If the source comparison reports no change, set `force_catalog_upload=true` for this one planned full-deployment rehearsal.
34	34	3. Save the workflow run ID, catalog revision, source revision, file count, and verification result as issue #9 evidence.
35	35	4. Run Roll back Bunny configuration site with that deployment run ID. It restores and byte-verifies the retained pre-deployment mutable files; the prior manifest points back to the already verified immutable catalog revision.
36	36	5. Run Deploy configuration site to Bunny again with `refresh_catalog=true`, save its verification result, and re-run the production configuration journey.
@@ -39,7 +39,7 @@ The workflow also restores its pre-deployment mutable files automatically on a f
39	39
40	40	## Publication gate
41	41
42		-Creating and pushing `v1.0.0` deploys the static site, builds the PBW, and creates the public GitHub release. Submitting through the Rebble Developer Portal is a separate public action.
	42	+Creating and pushing `v1.0.0` deploys the configuration page without re-uploading the catalog, builds the PBW, and creates the public GitHub release. Submitting through the Rebble Developer Portal is a separate public action.
43	43
44	44	Do neither until:
45	45

  - [x] Select `https://lapin-fute.b-cdn.net/` as the stable, static HTTPS configuration-site origin.
  - [x] Build release PBWs with that exact origin and reject release tags that differ from the watch package version.
  - [x] Package one PBW containing both `emery` and `gabbro`, plus `SHA256SUMS`, as GitHub release assets.
--- [x] Refresh a real IDFM catalog for production releases; no fixture fallback is available.
++- [x] Keep release tags page-only; manual real-IDFM refreshes compare `sourceRevision` with production and upload the catalog only when its inputs changed or an operator forces it.
  - [x] Upload immutable catalog files before the manifest pointer.
  - [x] Compare every built file with its served bytes and validate every current-revision catalog JSON against the manifest schema and revision.
  - [x] Back up mutable hosted files before deployment and restore them automatically when upload, purge, or served-site verification fails.
  ## Deployment and rollback rehearsal
 . Merge the reviewed release branch without tagging it.
--2. Run **Deploy configuration site to Bunny** manually with `refresh_catalog=true`.
++2. Run **Deploy configuration site to Bunny** manually with `refresh_catalog=true`. If the source comparison reports no change, set `force_catalog_upload=true` for this one planned full-deployment rehearsal.
 . Save the workflow run ID, catalog revision, source revision, file count, and verification result as issue #9 evidence.
 . Run **Roll back Bunny configuration site** with that deployment run ID. It restores and byte-verifies the retained pre-deployment mutable files; the prior manifest points back to the already verified immutable catalog revision.
 . Run **Deploy configuration site to Bunny** again with `refresh_catalog=true`, save its verification result, and re-run the production configuration journey.
  ## Publication gate
--Creating and pushing `v1.0.0` deploys the static site, builds the PBW, and creates the public GitHub release. Submitting through the Rebble Developer Portal is a separate public action.
++Creating and pushing `v1.0.0` deploys the configuration page without re-uploading the catalog, builds the PBW, and creates the public GitHub release. Submitting through the Rebble Developer Portal is a separate public action.
  Do neither until:

modified .github/workflows/deploy-bunny.yml +42 -2 Viewed

@@ -10,6 +10,11 @@ on:
10	10	required: true
11	11	default: false
12	12	type: boolean
	13	+ force_catalog_upload:
	14	+ description: Upload a rebuilt catalog even when its IDFM source revision is already hosted
	15	+ required: true
	16	+ default: false
	17	+ type: boolean
13	18
14	19	permissions:
15	20	contents: read
@@ -118,12 +123,41 @@ jobs:
118	123	- run: npm ci
119	124
120	125	- name: Refresh production catalog
	126	+ if: ${{ github.event_name == 'workflow_dispatch' && inputs.refresh_catalog }}
121	127	env:
122	128	IDFM_DATASET_TOKEN: ${{ secrets.IDFM_DATASET_TOKEN }}
123	129	run: npm run catalog:refresh
124	130
	131	+ - name: Plan catalog upload
	132	+ id: catalog
	133	+ env:
	134	+ CONFIG_SITE_ORIGIN: ${{ vars.CONFIG_SITE_ORIGIN }}
	135	+ EVENT_NAME: ${{ github.event_name }}
	136	+ FORCE_CATALOG_UPLOAD: ${{ inputs.force_catalog_upload }}
	137	+ run: \|
	138	+ set -euo pipefail
	139	+ if [[ "$EVENT_NAME" != workflow_dispatch ]]; then
	140	+ include_catalog=false
	141	+ reason="release tags update the configuration page only"
	142	+ elif [[ "$FORCE_CATALOG_UPLOAD" = true ]]; then
	143	+ include_catalog=true
	144	+ reason="catalog upload was forced"
	145	+ else
	146	+ include_catalog="$(node scripts/plan-catalog-upload.mjs)"
	147	+ reason="catalog source-revision comparison completed"
	148	+ fi
	149	+ echo "include_catalog=$include_catalog" >> "$GITHUB_OUTPUT"
	150	+ echo "$reason: include_catalog=$include_catalog"
	151	+
125	152	- name: Build static configuration site
126		- run: npm run build:config-site
	153	+ env:
	154	+ INCLUDE_CATALOG: ${{ steps.catalog.outputs.include_catalog }}
	155	+ run: \|
	156	+ if [[ "$INCLUDE_CATALOG" = true ]]; then
	157	+ npm run build:config-site
	158	+ else
	159	+ npm run build:config-page
	160	+ fi
127	161
128	162	- name: Back up mutable production files
129	163	id: backup
@@ -250,9 +284,15 @@ jobs:
250	284	- name: Verify served static site
251	285	env:
252	286	CONFIG_SITE_ORIGIN: ${{ vars.CONFIG_SITE_ORIGIN }}
	287	+ INCLUDE_CATALOG: ${{ steps.catalog.outputs.include_catalog }}
253	288	run: \|
254	289	for attempt in 1 2 3; do
255		- if npm run verify:config-site; then
	290	+ if [[ "$INCLUDE_CATALOG" = true ]]; then
	291	+ command=(npm run verify:config-site)
	292	+ else
	293	+ command=(npm run verify:config-site -- --page-only)
	294	+ fi
	295	+ if "${command[@]}"; then
256	296	exit 0
257	297	fi
258	298	if [[ "$attempt" != 3 ]]; then sleep 10; fi

          required: true
          default: false
          type: boolean
++      force_catalog_upload:
++        description: Upload a rebuilt catalog even when its IDFM source revision is already hosted
++        required: true
++        default: false
++        type: boolean
  permissions:
    contents: read
        - run: npm ci
        - name: Refresh production catalog
++        if: ${{ github.event_name == 'workflow_dispatch' && inputs.refresh_catalog }}
          env:
            IDFM_DATASET_TOKEN: ${{ secrets.IDFM_DATASET_TOKEN }}
          run: npm run catalog:refresh
++      - name: Plan catalog upload
++        id: catalog
++        env:
++          CONFIG_SITE_ORIGIN: ${{ vars.CONFIG_SITE_ORIGIN }}
++          EVENT_NAME: ${{ github.event_name }}
++          FORCE_CATALOG_UPLOAD: ${{ inputs.force_catalog_upload }}
++        run: |
++          set -euo pipefail
++          if [[ "$EVENT_NAME" != workflow_dispatch ]]; then
++            include_catalog=false
++            reason="release tags update the configuration page only"
++          elif [[ "$FORCE_CATALOG_UPLOAD" = true ]]; then
++            include_catalog=true
++            reason="catalog upload was forced"
++          else
++            include_catalog="$(node scripts/plan-catalog-upload.mjs)"
++            reason="catalog source-revision comparison completed"
++          fi
++          echo "include_catalog=$include_catalog" >> "$GITHUB_OUTPUT"
++          echo "$reason: include_catalog=$include_catalog"
++
        - name: Build static configuration site
--        run: npm run build:config-site
++        env:
++          INCLUDE_CATALOG: ${{ steps.catalog.outputs.include_catalog }}
++        run: |
++          if [[ "$INCLUDE_CATALOG" = true ]]; then
++            npm run build:config-site
++          else
++            npm run build:config-page
++          fi
        - name: Back up mutable production files
          id: backup
        - name: Verify served static site
          env:
            CONFIG_SITE_ORIGIN: ${{ vars.CONFIG_SITE_ORIGIN }}
++          INCLUDE_CATALOG: ${{ steps.catalog.outputs.include_catalog }}
          run: |
            for attempt in 1 2 3; do
--            if npm run verify:config-site; then
++            if [[ "$INCLUDE_CATALOG" = true ]]; then
++              command=(npm run verify:config-site)
++            else
++              command=(npm run verify:config-site -- --page-only)
++            fi
++            if "${command[@]}"; then
                exit 0
              fi
              if [[ "$attempt" != 3 ]]; then sleep 10; fi

modified package.json +1 -0 Viewed

@@ -19,6 +19,7 @@
19	19	"measure:radio": "node scripts/measure-radio.mjs",
20	20	"measure:catalog": "node scripts/measure-catalog.mjs",
21	21	"measure:config-page": "node scripts/measure-config-page.mjs",
	22	+ "plan:catalog-upload": "node scripts/plan-catalog-upload.mjs",
22	23	"verify:config-site": "node scripts/verify-config-site.mjs",
23	24	"test": "node scripts/test.mjs",
24	25	"test:contracts": "node --test packages/contracts/test/contracts.test.ts",

      "measure:radio": "node scripts/measure-radio.mjs",
      "measure:catalog": "node scripts/measure-catalog.mjs",
      "measure:config-page": "node scripts/measure-config-page.mjs",
++    "plan:catalog-upload": "node scripts/plan-catalog-upload.mjs",
      "verify:config-site": "node scripts/verify-config-site.mjs",
      "test": "node scripts/test.mjs",
      "test:contracts": "node --test packages/contracts/test/contracts.test.ts",

modified packages/config-page/test/config-site-build.test.ts +75 -4 Viewed

@@ -4,6 +4,7 @@ import { existsSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync
4	4	import { tmpdir } from "node:os";
5	5	import { join } from "node:path";
6	6	import { buildConfigSite } from "../../../scripts/build-config-site.mjs";
	7	+import { planCatalogUpload } from "../../../scripts/plan-catalog-upload.mjs";
7	8	import { verifyConfigSite } from "../../../scripts/verify-config-site.mjs";
8	9
9	10	function fixture() {
@@ -20,12 +21,26 @@ function fixture() {
20	21	return { root, pageSource, staticPath, outputPath };
21	22	}
22	23
23		-test("page-only configuration build neither requires nor publishes a catalog", () => {
	24	+test("page-only configuration build neither requires nor publishes a catalog", async () => {
24	25	const paths = fixture();
25	26	try {
26	27	buildConfigSite({ ...paths, includeCatalog: false });
27	28	assert.equal(readFileSync(join(paths.outputPath, "index.html"), "utf8"), "<h1>Lapin Futé</h1>");
28	29	assert.equal(existsSync(join(paths.outputPath, "catalog")), false);
	30	+ const result = await verifyConfigSite({
	31	+ sitePath: paths.outputPath,
	32	+ origin: "https://config.example.test/",
	33	+ includeCatalog: false,
	34	+ fetcher: async (url) => {
	35	+ const path = join(paths.outputPath, decodeURIComponent(url.pathname.slice(1)));
	36	+ return existsSync(path)
	37	+ ? new Response(readFileSync(path), { status: 200 })
	38	+ : new Response(null, { status: 404 });
	39	+ },
	40	+ });
	41	+ assert.equal(result.revision, null);
	42	+ assert.equal(result.catalogJsonCount, 0);
	43	+ assert.equal(result.fileCount, 4);
29	44	} finally {
30	45	rmSync(paths.root, { recursive: true, force: true });
31	46	}
@@ -54,6 +69,59 @@ test("full configuration build still requires and copies a published catalog", (
54	69	}
55	70	});
56	71
	72	+test("catalog upload planning compares source revisions instead of volatile publication revisions", async () => {
	73	+ const paths = fixture();
	74	+ try {
	75	+ const attribution = [{
	76	+ dataset: "arrets",
	77	+ url: "https://data.iledefrance-mobilites.fr/api/explore/v2.1/catalog/datasets/arrets",
	78	+ retrievedAt: "2026-09-13T00:00:00.000Z",
	79	+ license: "Licence Ouverte 2.0",
	80	+ }];
	81	+ const local = {
	82	+ schemaVersion: 1,
	83	+ revision: "a".repeat(64),
	84	+ sourceRevision: "idfm-v1-same-content",
	85	+ createdAt: "2026-09-13T00:00:00.000Z",
	86	+ attribution,
	87	+ };
	88	+ const manifestPath = join(paths.root, "manifest.json");
	89	+ writeFileSync(manifestPath, JSON.stringify(local));
	90	+
	91	+ const sameSource = await planCatalogUpload({
	92	+ manifestPath,
	93	+ origin: "https://config.example.test/",
	94	+ fetcher: async () => new Response(JSON.stringify({
	95	+ ...local,
	96	+ revision: "b".repeat(64),
	97	+ createdAt: "2026-09-12T00:00:00.000Z",
	98	+ attribution: [{ ...attribution[0], retrievedAt: "2026-09-12T00:00:00.000Z" }],
	99	+ }), { status: 200 }),
	100	+ });
	101	+ assert.equal(sameSource.upload, false);
	102	+
	103	+ const changedSource = await planCatalogUpload({
	104	+ manifestPath,
	105	+ origin: "https://config.example.test/",
	106	+ fetcher: async () => new Response(JSON.stringify({
	107	+ ...local,
	108	+ revision: "b".repeat(64),
	109	+ sourceRevision: "idfm-v1-new-content",
	110	+ }), { status: 200 }),
	111	+ });
	112	+ assert.equal(changedSource.upload, true);
	113	+
	114	+ const missing = await planCatalogUpload({
	115	+ manifestPath,
	116	+ origin: "https://config.example.test/",
	117	+ fetcher: async () => new Response(null, { status: 404 }),
	118	+ });
	119	+ assert.equal(missing.upload, true);
	120	+ } finally {
	121	+ rmSync(paths.root, { recursive: true, force: true });
	122	+ }
	123	+});
	124	+
57	125	test("configuration-site verification checks every local and served file on the current catalog revision", async () => {
58	126	const paths = fixture();
59	127	try {
@@ -193,9 +261,12 @@ test("Bunny releases deploy production tags while preview tags stop after qualit
193	261	assert.doesNotMatch(workflow, /--generate-notes/u);
194	262	assert.match(workflow, /github\.event_name == 'workflow_dispatch' && inputs\.refresh_catalog/u);
195	263	assert.match(workflow, /refresh_catalog:\s+description:[\s\S]+type: boolean/u);
196		- assert.match(workflow, /Refresh production catalog[\s\S]+npm run catalog:refresh/u);
197		- assert.match(workflow, /Build static configuration site[\s\S]+npm run build:config-site/u);
198		- assert.doesNotMatch(workflow, /npm run build:config-page/u);
	264	+ assert.match(workflow, /force_catalog_upload:\s+description:[\s\S]+type: boolean/u);
	265	+ assert.match(workflow, /Refresh production catalog[\s\S]+if: \$\{\{ github\.event_name == 'workflow_dispatch' && inputs\.refresh_catalog \}\}[\s\S]+npm run catalog:refresh/u);
	266	+ assert.match(workflow, /EVENT_NAME[\s\S]+release tags update the configuration page only/u);
	267	+ assert.match(workflow, /include_catalog="\$$node scripts\/plan-catalog-upload\.mjs$"/u);
	268	+ assert.match(workflow, /INCLUDE_CATALOG[\s\S]+npm run build:config-site[\s\S]+npm run build:config-page/u);
	269	+ assert.match(workflow, /npm run verify:config-site -- --page-only/u);
199	270	assert.match(
200	271	workflow,
201	272	/if \[\[ -f var\/config-site\/catalog\/manifest\.json \]\]; then\s+upload_file/u,

  import { tmpdir } from "node:os";
  import { join } from "node:path";
  import { buildConfigSite } from "../../../scripts/build-config-site.mjs";
++import { planCatalogUpload } from "../../../scripts/plan-catalog-upload.mjs";
  import { verifyConfigSite } from "../../../scripts/verify-config-site.mjs";
  function fixture() {
    return { root, pageSource, staticPath, outputPath };
  }
--test("page-only configuration build neither requires nor publishes a catalog", () => {
++test("page-only configuration build neither requires nor publishes a catalog", async () => {
    const paths = fixture();
    try {
      buildConfigSite({ ...paths, includeCatalog: false });
      assert.equal(readFileSync(join(paths.outputPath, "index.html"), "utf8"), "<h1>Lapin Futé</h1>");
      assert.equal(existsSync(join(paths.outputPath, "catalog")), false);
++    const result = await verifyConfigSite({
++      sitePath: paths.outputPath,
++      origin: "https://config.example.test/",
++      includeCatalog: false,
++      fetcher: async (url) => {
++        const path = join(paths.outputPath, decodeURIComponent(url.pathname.slice(1)));
++        return existsSync(path)
++          ? new Response(readFileSync(path), { status: 200 })
++          : new Response(null, { status: 404 });
++      },
++    });
++    assert.equal(result.revision, null);
++    assert.equal(result.catalogJsonCount, 0);
++    assert.equal(result.fileCount, 4);
    } finally {
      rmSync(paths.root, { recursive: true, force: true });
    }
    }
  });
++test("catalog upload planning compares source revisions instead of volatile publication revisions", async () => {
++  const paths = fixture();
++  try {
++    const attribution = [{
++      dataset: "arrets",
++      url: "https://data.iledefrance-mobilites.fr/api/explore/v2.1/catalog/datasets/arrets",
++      retrievedAt: "2026-09-13T00:00:00.000Z",
++      license: "Licence Ouverte 2.0",
++    }];
++    const local = {
++      schemaVersion: 1,
++      revision: "a".repeat(64),
++      sourceRevision: "idfm-v1-same-content",
++      createdAt: "2026-09-13T00:00:00.000Z",
++      attribution,
++    };
++    const manifestPath = join(paths.root, "manifest.json");
++    writeFileSync(manifestPath, JSON.stringify(local));
++
++    const sameSource = await planCatalogUpload({
++      manifestPath,
++      origin: "https://config.example.test/",
++      fetcher: async () => new Response(JSON.stringify({
++        ...local,
++        revision: "b".repeat(64),
++        createdAt: "2026-09-12T00:00:00.000Z",
++        attribution: [{ ...attribution[0], retrievedAt: "2026-09-12T00:00:00.000Z" }],
++      }), { status: 200 }),
++    });
++    assert.equal(sameSource.upload, false);
++
++    const changedSource = await planCatalogUpload({
++      manifestPath,
++      origin: "https://config.example.test/",
++      fetcher: async () => new Response(JSON.stringify({
++        ...local,
++        revision: "b".repeat(64),
++        sourceRevision: "idfm-v1-new-content",
++      }), { status: 200 }),
++    });
++    assert.equal(changedSource.upload, true);
++
++    const missing = await planCatalogUpload({
++      manifestPath,
++      origin: "https://config.example.test/",
++      fetcher: async () => new Response(null, { status: 404 }),
++    });
++    assert.equal(missing.upload, true);
++  } finally {
++    rmSync(paths.root, { recursive: true, force: true });
++  }
++});
++
  test("configuration-site verification checks every local and served file on the current catalog revision", async () => {
    const paths = fixture();
    try {
    assert.doesNotMatch(workflow, /--generate-notes/u);
    assert.match(workflow, /github\.event_name == 'workflow_dispatch' && inputs\.refresh_catalog/u);
    assert.match(workflow, /refresh_catalog:\s+description:[\s\S]+type: boolean/u);
--  assert.match(workflow, /Refresh production catalog[\s\S]+npm run catalog:refresh/u);
++  assert.match(workflow, /force_catalog_upload:\s+description:[\s\S]+type: boolean/u);
--  assert.match(workflow, /Build static configuration site[\s\S]+npm run build:config-site/u);
++  assert.match(workflow, /Refresh production catalog[\s\S]+if: \$\{\{ github\.event_name == 'workflow_dispatch' && inputs\.refresh_catalog \}\}[\s\S]+npm run catalog:refresh/u);
--  assert.doesNotMatch(workflow, /npm run build:config-page/u);
++  assert.match(workflow, /EVENT_NAME[\s\S]+release tags update the configuration page only/u);
++  assert.match(workflow, /include_catalog="\$\(node scripts\/plan-catalog-upload\.mjs\)"/u);
++  assert.match(workflow, /INCLUDE_CATALOG[\s\S]+npm run build:config-site[\s\S]+npm run build:config-page/u);
++  assert.match(workflow, /npm run verify:config-site -- --page-only/u);
    assert.match(
      workflow,
      /if \[\[ -f var\/config-site\/catalog\/manifest\.json \]\]; then\s+upload_file/u,

added scripts/plan-catalog-upload.mjs +111 -0 Viewed

new file mode 100644
@@ -0,0 +1,111 @@
	1	+import { readFileSync } from "node:fs";
	2	+import { dirname, join, resolve } from "node:path";
	3	+import { fileURLToPath } from "node:url";
	4	+import { isCatalogManifest } from "../packages/config-page/src/catalog-client.js";
	5	+
	6	+const PROJECT_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "..");
	7	+const DEFAULT_MANIFEST_PATH = join(PROJECT_ROOT, "var/catalog/static/manifest.json");
	8	+const MANIFEST_LIMIT_BYTES = 64 * 1024;
	9	+const REQUEST_TIMEOUT_MS = 30_000;
	10	+
	11	+function fail(message) {
	12	+ throw new Error(`plan:catalog-upload failed: ${message}`);
	13	+}
	14	+
	15	+function parseManifest(body, label) {
	16	+ let manifest;
	17	+ try {
	18	+ manifest = JSON.parse(body);
	19	+ } catch {
	20	+ fail(`${label} is not valid JSON`);
	21	+ }
	22	+ if (!isCatalogManifest(manifest)) fail(`${label} does not satisfy the static catalog contract`);
	23	+ return manifest;
	24	+}
	25	+
	26	+function configurationOrigin(value) {
	27	+ let origin;
	28	+ try {
	29	+ origin = new URL(value);
	30	+ } catch {
	31	+ fail("CONFIG_SITE_ORIGIN must be an absolute HTTPS URL");
	32	+ }
	33	+ if (origin.protocol !== "https:"
	34	+ \|\| origin.username !== ""
	35	+ \|\| origin.password !== ""
	36	+ \|\| origin.search !== ""
	37	+ \|\| origin.hash !== "") {
	38	+ fail("CONFIG_SITE_ORIGIN must be an absolute HTTPS URL without credentials, query, or fragment");
	39	+ }
	40	+ if (!origin.pathname.endsWith("/")) origin.pathname += "/";
	41	+ return origin;
	42	+}
	43	+
	44	+async function remoteManifest(origin, fetcher) {
	45	+ const url = new URL("catalog/manifest.json", origin);
	46	+ const response = await fetcher(url, {
	47	+ headers: { Accept: "application/json", "Cache-Control": "no-cache" },
	48	+ redirect: "error",
	49	+ signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
	50	+ });
	51	+ if (response.status === 404) return null;
	52	+ if (!response.ok) fail(`${url.href} returned HTTP ${response.status}`);
	53	+ const declaredLength = Number(response.headers.get("content-length"));
	54	+ if (Number.isFinite(declaredLength) && declaredLength > MANIFEST_LIMIT_BYTES) {
	55	+ fail("hosted catalog manifest exceeds the response limit");
	56	+ }
	57	+ if (response.body === null) fail("hosted catalog manifest has no body");
	58	+ const reader = response.body.getReader();
	59	+ const chunks = [];
	60	+ let bytes = 0;
	61	+ let finished = false;
	62	+ try {
	63	+ while (true) {
	64	+ const { done, value } = await reader.read();
	65	+ if (done) {
	66	+ finished = true;
	67	+ break;
	68	+ }
	69	+ bytes += value.byteLength;
	70	+ if (bytes > MANIFEST_LIMIT_BYTES) fail("hosted catalog manifest exceeds the response limit");
	71	+ chunks.push(Buffer.from(value));
	72	+ }
	73	+ } finally {
	74	+ if (!finished) await reader.cancel();
	75	+ reader.releaseLock();
	76	+ }
	77	+ return parseManifest(Buffer.concat(chunks, bytes).toString("utf8"), "hosted catalog manifest");
	78	+}
	79	+
	80	+export async function planCatalogUpload({
	81	+ manifestPath = DEFAULT_MANIFEST_PATH,
	82	+ origin,
	83	+ fetcher = globalThis.fetch,
	84	+} = {}) {
	85	+ const local = parseManifest(readFileSync(manifestPath, "utf8"), "generated catalog manifest");
	86	+ const remote = await remoteManifest(configurationOrigin(origin), fetcher);
	87	+ if (remote === null) {
	88	+ return { upload: true, reason: "hosted catalog manifest is missing", local, remote };
	89	+ }
	90	+ if (local.schemaVersion !== remote.schemaVersion) {
	91	+ return { upload: true, reason: "catalog schema changed", local, remote };
	92	+ }
	93	+ if (local.sourceRevision !== remote.sourceRevision) {
	94	+ return { upload: true, reason: "IDFM source revision changed", local, remote };
	95	+ }
	96	+ return { upload: false, reason: "IDFM source revision is already hosted", local, remote };
	97	+}
	98	+
	99	+if (process.argv[1] === fileURLToPath(import.meta.url)) {
	100	+ try {
	101	+ const plan = await planCatalogUpload({
	102	+ manifestPath: process.argv[2] ?? DEFAULT_MANIFEST_PATH,
	103	+ origin: process.argv[3] ?? process.env.CONFIG_SITE_ORIGIN,
	104	+ });
	105	+ console.error(`${plan.reason}: ${plan.local.sourceRevision}`);
	106	+ console.log(plan.upload ? "true" : "false");
	107	+ } catch (error) {
	108	+ console.error(error instanceof Error ? error.message : error);
	109	+ process.exitCode = 1;
	110	+ }
	111	+}

++import { readFileSync } from "node:fs";
++import { dirname, join, resolve } from "node:path";
++import { fileURLToPath } from "node:url";
++import { isCatalogManifest } from "../packages/config-page/src/catalog-client.js";
++
++const PROJECT_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "..");
++const DEFAULT_MANIFEST_PATH = join(PROJECT_ROOT, "var/catalog/static/manifest.json");
++const MANIFEST_LIMIT_BYTES = 64 * 1024;
++const REQUEST_TIMEOUT_MS = 30_000;
++
++function fail(message) {
++  throw new Error(`plan:catalog-upload failed: ${message}`);
++}
++
++function parseManifest(body, label) {
++  let manifest;
++  try {
++    manifest = JSON.parse(body);
++  } catch {
++    fail(`${label} is not valid JSON`);
++  }
++  if (!isCatalogManifest(manifest)) fail(`${label} does not satisfy the static catalog contract`);
++  return manifest;
++}
++
++function configurationOrigin(value) {
++  let origin;
++  try {
++    origin = new URL(value);
++  } catch {
++    fail("CONFIG_SITE_ORIGIN must be an absolute HTTPS URL");
++  }
++  if (origin.protocol !== "https:"
++      || origin.username !== ""
++      || origin.password !== ""
++      || origin.search !== ""
++      || origin.hash !== "") {
++    fail("CONFIG_SITE_ORIGIN must be an absolute HTTPS URL without credentials, query, or fragment");
++  }
++  if (!origin.pathname.endsWith("/")) origin.pathname += "/";
++  return origin;
++}
++
++async function remoteManifest(origin, fetcher) {
++  const url = new URL("catalog/manifest.json", origin);
++  const response = await fetcher(url, {
++    headers: { Accept: "application/json", "Cache-Control": "no-cache" },
++    redirect: "error",
++    signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
++  });
++  if (response.status === 404) return null;
++  if (!response.ok) fail(`${url.href} returned HTTP ${response.status}`);
++  const declaredLength = Number(response.headers.get("content-length"));
++  if (Number.isFinite(declaredLength) && declaredLength > MANIFEST_LIMIT_BYTES) {
++    fail("hosted catalog manifest exceeds the response limit");
++  }
++  if (response.body === null) fail("hosted catalog manifest has no body");
++  const reader = response.body.getReader();
++  const chunks = [];
++  let bytes = 0;
++  let finished = false;
++  try {
++    while (true) {
++      const { done, value } = await reader.read();
++      if (done) {
++        finished = true;
++        break;
++      }
++      bytes += value.byteLength;
++      if (bytes > MANIFEST_LIMIT_BYTES) fail("hosted catalog manifest exceeds the response limit");
++      chunks.push(Buffer.from(value));
++    }
++  } finally {
++    if (!finished) await reader.cancel();
++    reader.releaseLock();
++  }
++  return parseManifest(Buffer.concat(chunks, bytes).toString("utf8"), "hosted catalog manifest");
++}
++
++export async function planCatalogUpload({
++  manifestPath = DEFAULT_MANIFEST_PATH,
++  origin,
++  fetcher = globalThis.fetch,
++} = {}) {
++  const local = parseManifest(readFileSync(manifestPath, "utf8"), "generated catalog manifest");
++  const remote = await remoteManifest(configurationOrigin(origin), fetcher);
++  if (remote === null) {
++    return { upload: true, reason: "hosted catalog manifest is missing", local, remote };
++  }
++  if (local.schemaVersion !== remote.schemaVersion) {
++    return { upload: true, reason: "catalog schema changed", local, remote };
++  }
++  if (local.sourceRevision !== remote.sourceRevision) {
++    return { upload: true, reason: "IDFM source revision changed", local, remote };
++  }
++  return { upload: false, reason: "IDFM source revision is already hosted", local, remote };
++}
++
++if (process.argv[1] === fileURLToPath(import.meta.url)) {
++  try {
++    const plan = await planCatalogUpload({
++      manifestPath: process.argv[2] ?? DEFAULT_MANIFEST_PATH,
++      origin: process.argv[3] ?? process.env.CONFIG_SITE_ORIGIN,
++    });
++    console.error(`${plan.reason}: ${plan.local.sourceRevision}`);
++    console.log(plan.upload ? "true" : "false");
++  } catch (error) {
++    console.error(error instanceof Error ? error.message : error);
++    process.exitCode = 1;
++  }
++}

modified scripts/verify-config-site.mjs +24 -7 Viewed

@@ -58,14 +58,23 @@ function parseJson(path, label) {
58	58	}
59	59	}
60	60
61		-function verifyLocalSite(sitePath) {
	61	+function verifyLocalSite(sitePath, includeCatalog) {
62	62	const root = resolve(sitePath);
63	63	if (!statSync(root, { throwIfNoEntry: false })?.isDirectory()) {
64	64	fail(`site directory does not exist: ${root}`);
65	65	}
66	66	const indexPath = join(root, "index.html");
67		- const manifestPath = join(root, "catalog", "manifest.json");
68	67	if (!statSync(indexPath, { throwIfNoEntry: false })?.isFile()) fail("index.html is missing");
	68	+ if (!includeCatalog) {
	69	+ return {
	70	+ root,
	71	+ manifest: null,
	72	+ files: filesUnder(root),
	73	+ catalogJsonCount: 0,
	74	+ };
	75	+ }
	76	+
	77	+ const manifestPath = join(root, "catalog", "manifest.json");
69	78	if (!statSync(manifestPath, { throwIfNoEntry: false })?.isFile()) fail("catalog/manifest.json is missing");
70	79
71	80	const manifest = parseJson(manifestPath, "catalog manifest");
@@ -177,14 +186,15 @@ export async function verifyConfigSite({
177	186	sitePath = DEFAULT_SITE_PATH,
178	187	origin,
179	188	fetcher = globalThis.fetch,
	189	+ includeCatalog = true,
180	190	} = {}) {
181		- const local = verifyLocalSite(sitePath);
	191	+ const local = verifyLocalSite(sitePath, includeCatalog);
182	192	const parsedOrigin = siteOrigin(origin);
183	193	await verifyServedFiles(local, parsedOrigin, fetcher);
184	194	return {
185	195	origin: parsedOrigin.href,
186		- revision: local.manifest.revision,
187		- sourceRevision: local.manifest.sourceRevision,
	196	+ revision: local.manifest?.revision ?? null,
	197	+ sourceRevision: local.manifest?.sourceRevision ?? null,
188	198	fileCount: local.files.length,
189	199	catalogJsonCount: local.catalogJsonCount,
190	200	};
@@ -192,9 +202,16 @@ export async function verifyConfigSite({
192	202
193	203	if (process.argv[1] === fileURLToPath(import.meta.url)) {
194	204	try {
	205	+ const arguments_ = process.argv.slice(2);
	206	+ const pageOnly = arguments_.includes("--page-only");
	207	+ const positional = arguments_.filter((argument) => argument !== "--page-only");
	208	+ if (positional.length > 2 \|\| arguments_.length !== positional.length + (pageOnly ? 1 : 0)) {
	209	+ fail("usage: verify-config-site.mjs [site-path] [origin] [--page-only]");
	210	+ }
195	211	const result = await verifyConfigSite({
196		- sitePath: process.argv[2] ?? DEFAULT_SITE_PATH,
197		- origin: process.argv[3] ?? process.env.CONFIG_SITE_ORIGIN,
	212	+ sitePath: positional[0] ?? DEFAULT_SITE_PATH,
	213	+ origin: positional[1] ?? process.env.CONFIG_SITE_ORIGIN,
	214	+ includeCatalog: !pageOnly,
198	215	});
199	216	console.log(JSON.stringify(result, null, 2));
200	217	} catch (error) {

    }
  }
--function verifyLocalSite(sitePath) {
++function verifyLocalSite(sitePath, includeCatalog) {
    const root = resolve(sitePath);
    if (!statSync(root, { throwIfNoEntry: false })?.isDirectory()) {
      fail(`site directory does not exist: ${root}`);
    }
    const indexPath = join(root, "index.html");
--  const manifestPath = join(root, "catalog", "manifest.json");
    if (!statSync(indexPath, { throwIfNoEntry: false })?.isFile()) fail("index.html is missing");
++  if (!includeCatalog) {
++    return {
++      root,
++      manifest: null,
++      files: filesUnder(root),
++      catalogJsonCount: 0,
++    };
++  }
++
++  const manifestPath = join(root, "catalog", "manifest.json");
    if (!statSync(manifestPath, { throwIfNoEntry: false })?.isFile()) fail("catalog/manifest.json is missing");
    const manifest = parseJson(manifestPath, "catalog manifest");
    sitePath = DEFAULT_SITE_PATH,
    origin,
    fetcher = globalThis.fetch,
++  includeCatalog = true,
  } = {}) {
--  const local = verifyLocalSite(sitePath);
++  const local = verifyLocalSite(sitePath, includeCatalog);
    const parsedOrigin = siteOrigin(origin);
    await verifyServedFiles(local, parsedOrigin, fetcher);
    return {
      origin: parsedOrigin.href,
--    revision: local.manifest.revision,
++    revision: local.manifest?.revision ?? null,
--    sourceRevision: local.manifest.sourceRevision,
++    sourceRevision: local.manifest?.sourceRevision ?? null,
      fileCount: local.files.length,
      catalogJsonCount: local.catalogJsonCount,
    };
  if (process.argv[1] === fileURLToPath(import.meta.url)) {
    try {
++    const arguments_ = process.argv.slice(2);
++    const pageOnly = arguments_.includes("--page-only");
++    const positional = arguments_.filter((argument) => argument !== "--page-only");
++    if (positional.length > 2 || arguments_.length !== positional.length + (pageOnly ? 1 : 0)) {
++      fail("usage: verify-config-site.mjs [site-path] [origin] [--page-only]");
++    }
      const result = await verifyConfigSite({
--      sitePath: process.argv[2] ?? DEFAULT_SITE_PATH,
++      sitePath: positional[0] ?? DEFAULT_SITE_PATH,
--      origin: process.argv[3] ?? process.env.CONFIG_SITE_ORIGIN,
++      origin: positional[1] ?? process.env.CONFIG_SITE_ORIGIN,
++      includeCatalog: !pageOnly,
      });
      console.log(JSON.stringify(result, null, 2));
    } catch (error) {

ci: skip unchanged catalog uploadsUnverified