Referrer cohort & delivery policy

Unlisted internal dev doc — audit before publishing publicly

Internal engineering note, published unlisted for review only. Not yet audited for public release — do not cite, index, or treat as public-facing product docs until a human review signs off.

Purpose: define who gets the clean thumbnail vs the watermarked thumbnail, and why each rule exists.

Applies to: the edge watermark router snippet (apps/snippet/watermark-router.snippet.js), the thumbnail worker (apps/thumbnail/worker.ts), the referrer rollout cohort (apps/thumbnail/lib/referrer-rollout-cohort.ts), and the paying-customer bypass allowlists (apps/thumbnail/lib/customers.json). For any concrete host, IP, customer, or video id referenced here by role or code name, decode privately in the internal decoder (not published).

Redaction note: this runbook is deliberately generic. It names system components and uses generic roles / code names only. Never paste a real host, email, IP, Stripe id, or video id into this file.

---

1. Two variants, one principle

Each thumbnail id resolves to two byte variants in R2:

Variant	R2 key shape	Who serves it
clean	/thumbnails/{id}/clean.jpg (private)	worker only; never publicly fetchable
watermarked (wm)	/thumbnails/{id}/wm.jpg (public)	snippet 307-redirect or worker

Determinism principle. The variant a request receives is a pure function of request signals — referrer host, client IP, path, and (for one legacy loop) ASN/UA. The same request always yields the same variant. There is no randomness, no per-user A/B, no time-of-day flip: this keeps the 31-day referrer-blind page-rule cache and smart tiered cache from poisoning one cohort with another cohort’s bytes. If the decision were non-deterministic, a cached wm could be served to a paying customer (or a clean leaked to a watermark cohort) on a cache-key collision.

Corollary — clean variant must stay private. The public R2 bucket backs the redirect, so the clean.jpg keys must be blocked from public access (a Cloudflare rule matching host + starts_with(path,"/thumbnails/") + ends_with(path,"/clean.jpg")). If clean ever becomes publicly fetchable, every watermark decision is trivially bypassable. Keep that rule in sync — see the IMPORTANT block at the top of the snippet.

---

2. Decision order (evaluate top-down, first match wins)

The router evaluates these in a fixed order. Bypass is checked before the watermark decision, so a paying customer is never watermarked even if their referrer would otherwise qualify.

1. Non-GET/HEAD ................................. pass through to worker
2. Root / favicon / malformed (`://` in path) .. pass through
3. Legacy `/.jpg` ............................... 308 -> shared error image
4. Already-public `/thumbnails/*` .............. pass through (already R2)
5. No extractable video id ..................... pass through
6. Known legacy WordPress self-loop (1 host) ... 307 -> cached wm
7. BYPASS: paying IP  -> clean .................. pass through  (checked FIRST)
8. BYPASS: no referrer -> clean ................. pass through
9. BYPASS: paying domain -> clean .............. pass through
10. WM cohort host + HOT id .................... 307 -> cached wm
11. Everything else ............................ pass through (worker decides)

Steps 7–9 are the bypass tier. Step 10 is the watermark tier. The dev/local rule (Section 4) and the rollout cohort (Section 5) layer onto this in the worker.

---

3. Bypass policies (who always gets clean)

3a. No referrer -> clean (explicit)

A request with no Referer header is served clean, by design — this is an explicit early return, not an accident of fall-through. Rationale (from the no-referrer traffic-source analysis): the no-referrer population is dominated by non-eyeballs — a datacenter no-referrer source (DC1/DC2) spoofing a browser UA from a couple of fixed IPs, plus generic SDK/library default UAs (node, okhttp, Dart, bare Mozilla/5.0). Real browsers with a stripped referrer are a minority. Serving them clean costs ~no conversion and avoids punishing privacy-stripped real users. A CDN-internal IP (CFINT) shows enormous tiered-cache volume and is excluded as infra noise, not real demand.

3b. Paying-customer bypass — by domain AND by client IP

Paying customers are bypassed two independent ways, both evaluated before the watermark decision:

Key	Header / signal	Set
client IP	CF-Connecting-IP → True-Client-IP → first X-Forwarded-For	paying-IP allowlist
referrer domain	Referer host, normalized (lowercased, www./m. stripped)	paying-domain allowlist

Either match short-circuits to clean. IP bypass covers server-side / no-referrer integrations a paying customer runs; domain bypass covers their browser embeds. The allowlists are sourced from apps/thumbnail/lib/customers.json (Stripe-synced: stripeSubscriptionId, stripePriceId, entitlement domains/IPs). See an internal entitlements runbook (not published) for how that file is populated.

⚠️ Failure mode: the unprotected paying customer

A paying Stripe record with empty domain/IP entitlements receives zero bypass — it falls straight through to the watermark tier and gets watermarked despite paying. This is a real rollout blocker (code name C02, an unprotected paying customer; contrast C01, correctly protected). The webhook can create the subscription record while the entitlement fields stay blank.

Detection / fix:

• After any customer sync, assert every active subscription has at least one non-empty entitlement (domain or IP) before rollout.
• Treat an active stripeSubscriptionId with empty domain and empty IP entitlements as a paging condition, not a warning.
• Backfill the entitlement from the customer’s known embed host or origin IP, then redeploy the snippet/worker config.

# Sketch: flag active subscriptions with no bypass entitlement.
# (Run against the synced customers file; do not print PID into shared logs.)
bun run scripts/cloudflare/...  # entitlement audit entry point

---

4. Dev / local -> clean (detected by REFERRER HOST, not user-agent)

Local and preview development is served clean, detected from the referrer host, never from the user-agent.

Why referrer host, not UA: dev traffic mostly arrives with a localhost/127.0.0.1 referrer, and UA-based dev detection collides with the large server/library no-referrer bucket (Section 3a). A referrer-host rule is therefore high-precision; a UA rule is not.

Dev/local referrer-host signals (treated as clean):

Signal	Examples
loopback	localhost, 127.0.0.1, [::1]
dev TLDs	*.local, *.test
private ranges (RFC-1918)	10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
tunnels / previews	ngrok hosts, preview/staging subdomains

Hardening note: the live snippet’s shouldWatermarkRequest reaches dev-clean implicitly today (a dev referrer simply isn’t in any watermark cohort, so it falls through to clean). Making it explicit — an early dev-host return — is the safer form, because it stays clean even if a dev host is ever accidentally added to a watermark cohort. Prefer the explicit rule.

---

5. The referrer cohort (who is eligible to be watermarked)

The watermark tier only fires for hosts in the referrer cohort — an explicit, reviewed allowlist of attributable, high-volume referrer domains plus a small prefix family. It is opt-in by host, never blanket.

• Source of truth: apps/thumbnail/lib/referrer-rollout-cohort.ts ([domain, monthlyEstimatedRequests] pairs + a prefix list, e.g. an auction-app family prefix).
• Matching: normalize the referrer host (lowercase, trim, drop trailing dots, strip www.), then check the domain set and the prefix list.
• Inclusion threshold: a candidate must estimate ≥ 50,000 requests / 30 days and be attributable (a specific host/app/family), so the watermark reads as intentional attribution rather than accidental punishment of someone prototyping.

Developer-friendly bias: ambiguous or small direct traffic stays clean by default. New integrations are not watermarked until they are clearly large and attributable. This is why the no-referrer default flipped to clean (Section 3a) and why generic runtimes, crawlers, and shared optimizers are explicitly excluded from the cohort.

Snippet HOT set gating

Even for an eligible cohort host, the snippet only 307-redirects to the cached wm when the id is in the HOT/redirect set (pre-generated wm bytes exist in R2; kept under the 32 KB snippet size limit). A cohort host requesting a cold id falls through to the worker, which makes the same deterministic decision and generates/serves wm on the miss path. The HOT set is purely a performance pin (skip the worker for the hottest ids), not a policy boundary — policy is identical hot or cold.

---

6. Self-referred WM_DOMAINS (deterministically watermarked)

A small set of self-referred school/health hosts (the snippet’s WM_DOMAINS; code names S01–S04) are always watermarked when they request a HOT id. These are large, stable, attributable referrers whose own pages embed the thumbnails; they are a deterministic always-on subset of the cohort, separate from the gradual rollout list. They are checked after the paying-domain bypass, so a WM_DOMAINS host that later becomes a paying customer is bypassed correctly without removing it from the set.

---

7. Cohort roster (roles — decode internally)

Use roles in conversation; reserve code names for disambiguation. Full decode lives in the internal decoder (not published).

Code	Role
H01	the canary host — first flip, watched for still-leaking-clean
H02	single-hero-video host, already pinned
H03	rotating auction-catalog host, high cache decay
H08	highest-volume host (~4M/mo), rolled last and alone
H09	~1.34M/mo host with a second subdomain
H20–H24	non-buyer / cost-control hosts
S01–S04	self-referred school/health hosts (WM_DOMAINS)
K01	gray-market network
C01	protected paying customer (bypass works)
C02	unprotected paying customer (empty entitlements — rollout blocker)
DC1 / DC2	datacenter no-referrer sources
CFINT	excluded CDN-internal IP (tiered-cache noise)

---

8. Operator checklist

Before flipping or expanding a cohort:

• Clean variant (/thumbnails/{id}/clean.jpg) is confirmed private (bypass-prevention rule live).
• Every active subscription has a non-empty domain or IP entitlement (no C02-class records).
• New cohort host is ≥ 50k/30d and attributable; not a runtime/crawler/optimizer.
• Dev/local hosts are not present in any watermark cohort.
• HOT ids for the new host exist in R2 (or accept worker miss-path generation).
• Roll the highest-volume host alone and last (H08), after the canary (H01) is verified clean-free.
• Decode any concrete entity from the internal decoder (not published), never from this file.

To undo a flip, see reverting-deploys (snippet redeploy / cohort rollback is a config/data operation, not an emergency code change).

---

Related

• An internal entitlements runbook (not published) — how customers.json entitlements get populated.
• reverting-deploys — rolling back a snippet or cohort change.
• The internal decoder (not published) — private decode of every role / code name above.