lanthanides.io — Migration Plan & Stack Decision

lanthanides.io — Migration Plan & Stack Decision

Companion to docs/AUDIT.md (the inventory) and docs/ARCHITECTURE.md (the target layout & contracts). This document justifies the move from static Jekyll to a dynamic Next.js stack and fixes the data strategy, URL contract, and migration sequencing that every later prompt follows. docs/AUDIT.md is the source of truth for what exists and what must be preserved. Where this plan and the audit disagree, the audit wins and this plan is corrected. Scope of this prompt: documentation only — no code, no installs, no deploy, no credentials.

1. Stack decision & justification

Target stack: Next.js (App Router) · TypeScript · Tailwind CSS · Prisma ORM (SQLite for local/dev, Postgres for production).

The current site is a static Jekyll build served from GitHub Pages. It does the reference job well but has no server: it cannot run the price-gauge, accept a seller listing, or manage a notification subscription — the commercial direction in AUDIT.md §6. The migration keeps everything Jekyll does well (versioned open data, SEO, the crown-jewel regulatory tracker) and adds the thin slice of server capability the product roadmap needs.

Explicit trade-off: hosting moves off GitHub Pages

A Node server is now required (API routes, ISR, Prisma). The site can no longer be served by GitHub Pages.

• Deployment target: a Node host (e.g. Vercel, the natural fit for Next.js App Router + ISR + Postgres).
• CNAME is preserved. The apex/www domain stays www.lanthanides.io (current CNAME content). The DNS/host binding moves from GitHub Pages to the Node host; the domain does not change, so no public URL is affected by the host move.
• This prompt does NOT wire up any deploy, host config, or credentials. Choosing and connecting the host is a later, explicitly-authorized step. We only record the decision and the constraint here.

2. Hybrid data strategy

The product is two things wearing one skin: an open-data reference (elements, prices, provenance, regulatory intelligence) and a thin commercial app (listings, alerts, screened offers). They have opposite persistence needs, so they get opposite stores.

2.1 Reference + provenance data stays in versioned files (_data/, _elements/, _articles/)

Everything the audit calls canonical stays exactly where it is, read at build time:

A typed data-access layer (lib/data/, contracts in ARCHITECTURE.md §3) reads these files at build time and returns validated objects; pages render via SSG/ISR. No database sits in front of the reference data.

Why the provenance dataset must NOT be hidden behind a database — this is the load-bearing decision of the whole migration:

1. The data is the product, and its credibility comes from being inspectable. AUDIT.md §6 names the regulatory tracker the differentiator precisely because a funder “can verify it in five minutes and not find it elsewhere.” A database row has no public URL, no diff, no history. A YAML file in git has all three. Hiding the dataset behind an opaque server destroys the one property that makes it defensible.
2. It honors the open-data / CC BY 4.0 licence in substance, not just in a footer. The dataset is published so it can be forked, audited, and cited. Files in git are the open-data export; a DB would force us to re-publish a derivative and ask the world to trust that the export matches the live data.
3. It keeps the existing Python pipeline in scripts/ working unchanged. validate_data.py, normalize_prices.py, fluctuations.py, detect_movements.py, source_breakdown.py, and the regulatory-monitor (run_monitor.py → scraper/ → triage.py → notify/) all read and write these files. The monitor bot commits _data updates every 6 hours (AUDIT.md §1.9, §5). Moving the data into a DB would break this freshness engine or force a costly rewrite.
4. Provenance review is a git-PR flow. The contributor pipeline (AUDIT.md §5) — issue templates → approved label → community-intake.yml opens a PR with two human checkpoints — depends on price changes being a reviewable file diff. That is the consent-/review-gated provenance model documented on /methodology/.
5. Read-heavy, write-rare, build-time-stable. The reference data changes a few times a day at most and is read on every page view. Files + SSG/ISR is the cheapest, fastest, most cacheable shape; a DB query per request would add load and latency for zero benefit.

The data layer also resolves the §4.3 count contradiction (238 price records vs 285 observations) by exposing labeled counts from one place, so home/about/methodology stop disagreeing. It collapses the parallel _data/elements/*.yml store (which only fed the divergent interactive ledger) — price_records.json becomes the single canonical price store.

2.2 Only genuinely dynamic, user-generated data lives in Prisma/SQLite

Three tables, and nothing that belongs in _data/:

These are runtime writes, often private, and explicitly not part of the auditable open dataset. They get a real database, migrations, and typed access — and they stay out of git history.

3. URL preservation (the routing contract)

AUDIT.md §2 is the hard contract. Every public URL the Jekyll site serves must keep working or get an explicit redirect. Jekyll uses permalink: pretty (trailing-slash pages), so the Next app sets trailingSlash: true in next.config to match — every page URL keeps its trailing slash. Machine-readable endpoints that carry a file extension (.xml, .json, .txt, .webmanifest) keep their exact extension-bearing path with no trailing slash and are served by route handlers or static files in public/.

3.1 Top-level pages

★ /framework/ reconciliation. The enumerated route list in the migration prompt omitted /framework. But AUDIT.md §2 (permalink contract), §5 (assets to preserve), and §6 (crown-jewel cluster) make /framework/ a hard contract and a near-unique asset to preserve verbatim. Per the run rule (AUDIT.md is the source of truth), it is included here and in ARCHITECTURE.md. Later prompts must build /framework/ and keep its in-page anchors.

3.2 Element collection (31) — case-sensitive symbols

/elements/<Symbol>/ with original-case symbols → app/elements/[symbol]/page.tsx with generateStaticParams() emitting the exact 31 symbols from element_catalog.yml:

La Ce Pr Nd Sm Y Sc Eu Gd Tb Dy Ho Er Tm Yb Lu Te V Sb W Bi Mo Zr Ta Nb Co Li Ga Ge Se In

⚠️ Case sensitivity is part of the contract: the canonical URL is /elements/Dy/, not /elements/dy/. generateStaticParams emits the cased symbols; a lowercase request should 301 to the cased form (handled in a later prompt, not assumed by the router).

3.3 Article collection (5)

/news/<slug>/ → app/news/[slug]/page.tsx with generateStaticParams() over _articles/*.md:

/news/chinas-invisible-rare-earth-wall/ · /news/chinese-export-licence-timelines/ · /news/ebay-rare-earth-market-research-april-2026/ · /news/mofcom-october-2025-suspension/ · /news/retail-premium-puzzle/

3.4 Machine-readable / feeds / SEO (exact paths, no trailing slash)

Anchors to preserve (deep-linked from multiple pages): /methodology/#display-price, #provenance-chain, #data-sources-breakdown, #oxide-to-metal; and on /framework/: #pricing, #us-side-tariff-stack-may-14-2026. Section ids must match so existing inbound links and in-page navigation keep working.

3.5 The one URL that changes

/prices/ → 301 → /elements/. The Jekyll site has two near-duplicate price tables: /prices/ (“The Ledger”, interactive) and /elements/ (the directory, 4 category tables). They are merged into a single canonical price directory at /elements/ (resolving AUDIT.md §4.3 / viz item #11 — two stores, one truth). /prices/ becomes a permanent redirect to /elements/ so no inbound link breaks. This is the only public page URL that changes.

New routes introduced by the commercial direction have no old equivalent and therefore need no redirect: /data, /tools/price-gauge, /sell, /offers, /alerts, and all /api/*. The nav label “Prices” repoints to /elements; “Contribute” continues to point at the GitHub contributor flow (issue templates), not a new app route.

4. Migration sequencing (prompts 3–8)

The invariant: next build must succeed on every commit from prompt 3 onward (a not-yet-ported page renders a labeled placeholder, never a build error), and the Jekyll site stays buildable in parallel for side-by-side parity diffing until prompt 25. Jekyll’s templating/build files are moved into legacy/ as each route reaches parity (the Next build never imports from legacy/); the entire legacy/ tree is deleted only after prompt 25 signs off full route parity against AUDIT.md §2.

Prompts 9–24 (out of scope here) cover the visualization rebuilds (AUDIT.md §3), content/positioning (§4.5–§4.6), design polish, and PWA/manifest fixes (§4.8). Prompt 25 verifies route parity against AUDIT.md §2 and then removes legacy/.

5. Preservation checklist

Every must-preserve asset from AUDIT.md §5, with its new home. Nothing on this list may be lost in the migration.

End of migration plan. Read with docs/AUDIT.md (what exists) and docs/ARCHITECTURE.md (target layout, route map, data contracts, feature modules). Treat §3 as the hard URL contract and §4 as the build-must-stay-green sequencing rule.