Internal docs publishing

Purpose: keep internal engineering docs (plans, research, rollout strategy, customer-identifying notes) out of every public-facing surface, while still allowing a deliberately redacted external page when sharing is required.

Applies to: anyone editing docs/ (the internal VitePress site), apps/docs/ (the public Starlight product-spec site), or standing up an external share page. For concrete entities referenced abstractly here, see the internal decoder (not published — never committed to a public surface).

Why this matters

Internal docs contain PID-class material: customer names/emails, host/brand domains, IPs, billing-provider ids, monthly bill totals, specific video ids, infra topology, and rollout sequencing. None of it may reach a public URL, a public AI-readable manifest, or a co-located build that carries the real product brand. Code names (H01, C02, S01, …) do not survive co-location with the real brand: a reader who sees the brand plus a descriptor (“the highest-volume rotating-catalog host”) can re-identify the underlying entity. Treat descriptors as PID too.

The two doc surfaces

Surface	Path	Build model	Public?	AI manifest
Internal docs (VitePress)	`docs/`	Globs every `.md` under `srcDir` into a routable page	Internal only	none
Product-spec (Starlight)	`apps/docs/`	Builds only its own content collection	Public	emits `llms.txt` / `llms-full.txt` / `llms-small.txt`

Surface 1 — internal VitePress build

VitePress turns every .md under srcDir into a routable page by default. The sidebar only controls navigation, not URL reachability — an un-listed page is still served at its slug. So an internal tree that is merely “not in the sidebar” is still publicly fetchable if the site is deployed.

Containment is srcExclude, not the sidebar. The internal trees are excluded from the build entirely:

srcExclude: [
  "plans/**",
  "research/**",
  "progress/**",
  "watermarks/**",
],

Rules:

Any new internal-only tree (anything containing plans, research, progress notes, or customer/host-specific material) must be added to srcExclude in the same change that introduces it.
Do not rely on omitting a page from the sidebar to hide it. Sidebar omission ≠ exclusion.
After changing srcExclude, verify the served bytes (see Verify the live bytes) — not just the source config.

Surface 2 — public Starlight product-spec site

The product-spec site builds only the docs content collection (docsLoader() over its own src/content/docs), so a stray .md elsewhere in the repo is not swept in the way VitePress globs. But it has the opposite risk: it is public by design and emits machine-readable manifests via starlight-llms-txt:

/llms.txt, /llms-small.txt — indexes for AI agents
/llms-full.txt — the full machine-readable concatenation of the collection

Anything you place in the public collection is, by design, served to humans and flattened into llms-full.txt for AI agents. There is no “internal” page in this site.

Rules:

Never copy or summarize an internal plan/research doc into the public collection. Not the file, not a paraphrase, not a “lightly cleaned” version. Co-location with the real brand re-identifies code-named entities via their descriptors.
New public pages are present-tense product spec only (Live / Flagged / Planned status badges) — no customers, no hosts, no infra costs, no rollout timing.
Assume every public page ends up verbatim in llms-full.txt. Redact accordingly before, not after.

Every internal plan/research/progress doc carries a banner at the top so a future editor cannot mistake it for shareable material:

> **DO NOT PUBLISH.** Internal working doc. Contains customer-identifying and
> infra/cost detail. Excluded from the VitePress build via `srcExclude`. Never
> copy, summarize, or migrate into the public product-spec collection.

The banner is a backstop, not the control. The control is srcExclude (Surface 1) and not-co-locating (Surface 2).

When something genuinely needs to go to an external reader, do not loosen either doc site. Stand up a separate worker serving a single redacted page:

Brand-detached: no product domain, no product name, no logos that re-identify the system.
noindex: send X-Robots-Tag: noindex, nofollow (and a <meta name="robots" content="noindex,nofollow">) so it stays out of search and AI crawls.
Separate deploy unit: its own worker/route, never a sub-path of either doc site, so it cannot inherit their content or be reached by editing a shared build.
Fully redacted before deploy (checklist below).

See reverting-deploys for pulling a share page back down, and the internal decoder (not published) for the decode of any code-named entity you are tempted to mention.

Redaction checklist

Run this against the rendered output, not just the source. A single surviving real entity fails the page.

Customer / contact names — removed. Replace with a role (“the unprotected paying customer”, “a self-referred school/health host”).
Emails — removed (including in examples, mailto links, and code samples).
Brand / host / product domains — removed. No real hostnames, even partial or obfuscated.
IP addresses — removed (no host IPs, no datacenter no-referrer source IPs, no CDN-internal IPs).
Billing-provider ids — removed (subscription / customer / price ids).
Monthly bill totals — removed. Public unit rates and mechanics are fine; the real blended bill is not.
Specific video ids — removed.
Re-identifying descriptors — generalized. “The ~4M/mo highest-volume host rolled last and alone” → “a high-volume host”. If you must disambiguate across the doc, use a code name and decode it only in the internal decoder (not published).
Rollout sequencing / entitlement gaps — omitted from public surfaces (these re-identify which customer was a blocker).

Prefer roles over code names; prefer code names over real entities; never the real entity.

Verify the live bytes

Source config and rendered output drift. Always confirm what is actually served.

# Surface 1 — an excluded internal tree must NOT be reachable on the deployed site.
# Expect 404 for each excluded slug.
curl -s -o /dev/null -w "%{http_code}\n" https://<internal-docs-host>/plans/<some-internal-slug>
curl -s -o /dev/null -w "%{http_code}\n" https://<internal-docs-host>/research/<some-internal-slug>

# Surface 2 — the public AI manifests must contain ZERO internal material.
# Each grep below should return nothing.
curl -s https://<spec-host>/llms-full.txt | grep -iE '<customer-name>|<host-domain>|<billing-id>'
curl -s https://<spec-host>/llms.txt      | grep -iE 'plan|research|progress|rollout'

# Share worker — must be noindex and brand-detached.
curl -sI https://<share-host>/ | grep -i 'x-robots-tag'   # expect: noindex, nofollow

Notes:

Test against the deployed host, not the local dev server — build-time exclusion can differ from a dev server that serves everything.
A page can be absent from the HTML sidebar yet still 200 at its slug. The 404 check is the real test for Surface 1.
llms-full.txt is the highest-risk artifact: it concatenates the whole public collection into one fetch. Grep it explicitly.

Quick decision table

You want to…	Do
Add an internal plan/research/progress doc	Put it under an excluded tree; add a DO-NOT-PUBLISH banner; confirm the tree is in `srcExclude`
Add a new internal tree	Add it to `srcExclude` in the same change; verify 404 on the deployed host
Document the product publicly	Add a present-tense spec page to the Starlight collection — no customers, hosts, costs, or rollout detail
Share findings with an outside reader	Stand up a separate, redacted, `noindex`, brand-detached worker; run the redaction checklist; verify live bytes
Reference a specific entity	Use a role; if you must disambiguate, a code name decoded only in the internal decoder (not published)