mirror of
https://github.com/pupperpowell/bibdle.git
synced 2026-10-22 05:54:10 -04:00
369 lines
26 KiB
Markdown
369 lines
26 KiB
Markdown
# Bibdle Statistics Dashboard — Implementation Checklist
|
||
|
||
**For:** an AI coding agent (or human) implementing the plan in `statistics-plan.md`.
|
||
**Repository:** Bibdle — SvelteKit 2 · Svelte 5 (runes) · Tailwind 4 · Bun · SQLite + Drizzle ORM.
|
||
**Rule:** read `statistics-plan.md` and `README.md` fully before starting. This checklist is ordered and dependency-aware; do tasks top to bottom unless a task says otherwise.
|
||
|
||
---
|
||
|
||
## 0. How to use this document
|
||
|
||
- Each task has an **ID**, **Goal**, **Files**, **Steps**, **Acceptance**, and **Tests**.
|
||
- Do **not** skip the Phase 0 foundations — every later phase depends on the schema and event log.
|
||
- Prefer pure, testable functions in `src/lib/server/stats/*`; keep Svelte components presentational.
|
||
- When a task is ambiguous, prefer the smallest reversible change and leave a `// TODO(stats):` comment rather than inventing behavior.
|
||
|
||
---
|
||
|
||
## 1. Guardrails (non-negotiable)
|
||
|
||
1. **Do not add Umami or Rybbit integration.** The dashboard is self-contained and durable. Remove/ignore any Rybbit references you touch.
|
||
2. **Do not query the DB on `/admin/stats` page load** except to (a) build a missing snapshot lazily, or (b) serve a `?mature=1` view. Normal loads read the stored snapshot only.
|
||
3. **Point-in-time correctness:** snapshot computation for date `D` may only use rows with `date <= D` (and `completed_at` before end of `D`).
|
||
4. **Plain language only.** No "velocity/acceleration/derivative" jargon in UI labels or captions. Every metric gets a one-sentence caption and a "what to do."
|
||
5. **Privacy:** `/admin/stats` is gated to `ADMIN_EMAIL`. Never render emails or raw anonymous IDs. The public widget must expose no retention/churn/signup/revenue data.
|
||
6. **Do not modify existing public routes** (`/`, `/progress`, `/stats`, `/imposter`, etc.) beyond the specific instrumentation hooks in Phase 0. Do not change the public `/global` until Phase 4.
|
||
7. **No new runtime dependency except `layerchart`.** (Email digest would require `nodemailer`; it is out of scope unless the owner approves — default to the Discord webhook.)
|
||
8. **Follow existing code style:** Svelte 5 runes (`$state`, `$derived`, `$effect`, `$props`, `$bindable`), Tailwind 4 utility classes, Drizzle query builder, `bun test` with `describe/test/expect`.
|
||
9. **Migrations:** never hand-edit generated SQL. Use `bun run db:generate` and review the diff. Be aware the `drizzle/` journal references `0001_loose_kree` but that `.sql` file is absent on disk — do not "fix" this; just add new migrations on top.
|
||
10. **Commit early and often**, one phase per logical commit set, message prefix `stats:`.
|
||
|
||
---
|
||
|
||
## 2. Environment & commands
|
||
|
||
```bash
|
||
bun install
|
||
bun run dev # dev server
|
||
bun run check # svelte-check / TS
|
||
bun test # all tests
|
||
bun test tests/<file>.test.ts
|
||
bun run db:generate # generate migration from schema.ts
|
||
bun run db:migrate # apply migrations to DATABASE_URL
|
||
```
|
||
|
||
- `DATABASE_URL` (e.g. `dev.db`) drives the app DB.
|
||
- `TEST_DATABASE_URL` drives `drizzle.test.config.ts`.
|
||
- Existing env: `CRON_SECRET`, `DISCORD_DAILY_WEBHOOK`, `ADMIN_EMAIL` (hardcoded in `src/lib/server/admin.ts`).
|
||
- **Add** `DISCORD_ADMIN_WEBHOOK` (digest target) to `.env.example` and `.env`.
|
||
|
||
**Verification gate after every phase:** `bun run check` clean, `bun test` green, `bun run build` succeeds.
|
||
|
||
---
|
||
|
||
## Phase 0 — Foundations & instrumentation
|
||
|
||
### T0.1 — Add tables and column to the Drizzle schema
|
||
- **Goal:** add `stats_snapshot`, `analytics_events`, `dashboard_settings`, and `user.created_at`.
|
||
- **Files:** `src/lib/server/db/schema.ts`
|
||
- **Steps:**
|
||
1. Add `user.createdAt = integer('created_at', { mode: 'timestamp' }).$defaultFn(() => new Date())`. (`$defaultFn` means new rows get a timestamp automatically; existing rows stay `NULL` = "unknown".)
|
||
2. Add `statsSnapshot` (`date` text PK; `generatedAt` integer timestamp; `buildKind` text; `configVersion` text; `payload` text; `playerState` text; `config` text).
|
||
3. Add `analyticsEvents` (`id` text PK; `name` text; `anonymousId` text; `userId` text nullable FK → user ON DELETE SET NULL; `date` text nullable; `createdAt` integer timestamp; `metadata` text). Indexes on `(name, date)` and `(anonymousId, date)`; unique index `analytics_events_serve_unique` on `(name, anonymousId, date)`.
|
||
4. Add `dashboardSettings` (`key` text PK; `value` text; `updatedAt` integer timestamp).
|
||
5. Export inferred types.
|
||
- **Steps:** run `bun run db:generate`, inspect SQL, then `bun run db:migrate`.
|
||
- **Acceptance:** fresh migration applies cleanly to `dev.db`; `sqlite3 dev.db ".schema user"` shows `created_at`; all three tables exist.
|
||
- **Tests:** none (schema only) — but run full suite.
|
||
|
||
### T0.2 — Settings module (thresholds with env defaults)
|
||
- **Goal:** a single source of truth for configurable thresholds.
|
||
- **Files:** new `src/lib/server/stats/settings.ts`; new `src/lib/types/stats.ts` (types).
|
||
- **Steps:**
|
||
1. Define keys with defaults: `core_plays_per_week` (4), `churn_days` (30), `resurrect_gap_days` (30), `at_risk_days` (8), `d1_target` (20), `return7_target` (25), `completion_target` (70), `first_try_min` (15), `first_try_max` (35), `share_target` (25), `active_window_days` (180), `anomaly_mad_threshold` (2).
|
||
2. `getSettings()` reads `dashboard_settings` and overlays env-var defaults (e.g. `STATS_D1_TARGET`). `updateSettings(partial)` upserts rows.
|
||
3. `configVersion()` = short stable hash of the resolved settings.
|
||
- **Acceptance:** `getSettings()` returns defaults with an empty table; `updateSettings` persists and is reflected on re-read.
|
||
- **Tests:** `tests/stats-settings.test.ts` — defaults, override, upsert, config hash stability.
|
||
|
||
### T0.3 — Event recording module + client helper
|
||
- **Goal:** a single, safe sink for analytics events (B/D/E/F).
|
||
- **Files:** new `src/lib/server/stats/events.ts`; new `src/routes/api/events/+server.ts`; new `src/lib/utils/analytics.ts`.
|
||
- **Steps:**
|
||
1. `recordEvent({ name, anonymousId, userId, date, metadata })` inserts into `analytics_events`. For `name === 'verse_served'`, use `INSERT OR IGNORE` (dedupe on unique index).
|
||
2. `POST /api/events` validates `name` against an allowlist (`verse_served`, `share`, `share_copy`, `first_visit`, `donate_click`, `discord_click`), caps metadata size, resolves `userId` from `locals.user` if present, and records. Return 204. Never throw to the client callers (fire-and-forget).
|
||
3. `src/lib/utils/analytics.ts` exports `track(name, metadata?)` that POSTs to `/api/events` with the lib's anonymous ID (read/create `bibdle-anonymous-id`), ignoring errors. Also export `getOrCreateAnonymousId()` and refactor `src/lib/stores/game-persistence.svelte.ts` to import it (avoid duplicated logic).
|
||
- **Acceptance:** calling `track('donate_click')` writes one row; repeated `verse_served` for the same `(anon, date)` writes only once.
|
||
- **Tests:** `tests/stats-events.test.ts` — allowlist rejection, metadata cap, verse_served dedupe, user attribution.
|
||
|
||
### T0.4 — Instrumentation A: guarantee guess sequences for everyone
|
||
- **Goal:** all completions persist `guesses`.
|
||
- **Files:** `src/routes/api/submit-completion/+server.ts`, `src/routes/+page.svelte` (verify), `src/routes/+page.server.ts` (legacy form action).
|
||
- **Steps:** confirm the client always sends `guesses`; ensure the API stores `JSON.stringify(guesses)` whenever the array is non-empty. Update the legacy `submitCompletion` form action to accept and store `guesses` too (or delete the dead action if unused). Do not backfill historical rows.
|
||
- **Acceptance:** a guest completion POSTed with `guesses` stores the JSON; `SELECT COUNT(*) FROM daily_completions WHERE guesses IS NOT NULL` grows for guest IDs in dev.
|
||
- **Tests:** extend `tests/stats.test.ts` or add `tests/stats-completion.test.ts`.
|
||
|
||
### T0.5 — Instrumentation B: log "verse served" (game started)
|
||
- **Goal:** measure started → completed.
|
||
- **Files:** `src/routes/+page.ts`, `src/routes/api/daily-verse/+server.ts`, `src/lib/utils/analytics.ts`.
|
||
- **Steps:** in `+page.ts`, read/create the anonymous ID before the fetch and include it in the `POST /api/daily-verse` body. In the endpoint, record `verse_served` (unique per anon+date). Keep the endpoint's response shape unchanged.
|
||
- **Acceptance:** loading `/` records exactly one `verse_served` per browser per day, even after reloads.
|
||
- **Tests:** `tests/stats-serve.test.ts` (endpoint logic with a test DB) + unit test for dedupe.
|
||
|
||
### T0.6 — Instrumentation D: share / copy events
|
||
- **Goal:** measure virality.
|
||
- **Files:** `src/lib/components/WinScreen.svelte`, `src/lib/utils/share.ts` (or call site).
|
||
- **Steps:** fire `track('share', { snippet })` when the native-share path is used and `track('share_copy', { snippet })` on clipboard copy. Do not double-count the copy that happens inside `shareResult` fallback. Keep existing Umami `data-umami-*` attributes untouched.
|
||
- **Acceptance:** clicking Share writes one `share` row; Copy writes one `share_copy` row; snippet flag correct.
|
||
- **Tests:** manual + a small unit test on the helper if logic is extracted.
|
||
|
||
### T0.7 — Instrumentation E: first-touch referrer / UTM
|
||
- **Goal:** acquisition source.
|
||
- **Files:** `src/lib/utils/analytics.ts`, root layout or `+page.ts`.
|
||
- **Steps:** on first visit per browser (guard with localStorage key), capture `document.referrer` and `?ref=` / `?utm_source=` / `?utm_medium` / `?utm_campaign`, and record a one-time `first_visit` event with that metadata.
|
||
- **Acceptance:** first visit with `?utm_source=bluesky` records one `first_visit`; subsequent visits do not.
|
||
- **Tests:** unit test on the parsing/guard logic.
|
||
|
||
### T0.8 — Instrumentation F: monetization clicks
|
||
- **Goal:** patron funnel.
|
||
- **Files:** `src/lib/components/WinScreen.svelte` (Donate, Discord), `src/lib/components/SocialLinks.svelte` if desired.
|
||
- **Steps:** fire `track('donate_click')` and `track('discord_click')` on click (keep the existing Umami attributes).
|
||
- **Acceptance:** clicks record events with the right names.
|
||
|
||
### T0.9 — Add `DISCORD_ADMIN_WEBHOOK` env
|
||
- **Files:** `.env.example`, `.env` (local only), `README.md` env list.
|
||
- **Acceptance:** documented; not required at boot (digest degrades gracefully when unset).
|
||
|
||
**Phase 0 exit criteria:** migrations applied; events flowing in dev; `bun run check` + `bun test` green.
|
||
|
||
---
|
||
|
||
## Phase 1 — Core console (snapshot + Pulse + Signals skeleton)
|
||
|
||
### T1.1 — Pure metric functions
|
||
- **Goal:** tested, deterministic metrics independent of Svelte/DB reads.
|
||
- **Files:** new `src/lib/server/stats/metrics.ts`, types in `src/lib/types/stats.ts`.
|
||
- **Steps:** implement, each taking plain rows and a cutoff date:
|
||
- `activePlayers(completions, end, windowDays)`
|
||
- `newVsReturning(completions, date)` (returning = first-ever completion `< date`)
|
||
- `netGrowth(completions, end, windowDays, settings)`
|
||
- `d1Return(completions, cohortDate)` and `returnWithin(completions, cohortDate, n)`
|
||
- `cohortTriangle(completions, { start, weeks })`
|
||
- `timeToSecondPlayHistogram(completions, buckets)`
|
||
- `guessHistogram(completions, maxBucket)` + `firstTryRate`
|
||
- `streakSurvival(completions)`
|
||
- `bookCoverage(verses, completions, start, end)`
|
||
- `retentionByBook(completions, verses, minPlays)` and `retentionByGuessBucket`
|
||
- `entropy(bookCounts)`
|
||
- `weekdayHourHeatmap(completions)`
|
||
- **Acceptance:** pure functions; no DB imports; correct on hand-built fixtures.
|
||
- **Tests:** `tests/stats-metrics.test.ts` — one test per function incl. edge cases (empty, single player, day boundaries in UTC).
|
||
|
||
### T1.2 — Segment derivation from stored player state
|
||
- **Goal:** render-time configurable segments.
|
||
- **Files:** new `src/lib/server/stats/segments.ts`.
|
||
- **Steps:** define `PlayerState = { id, firstDate, lastDate, totalPlays, plays7, plays30, streak }`. `deriveSegments(playerState[], asOfDate, settings)` returns `{ new, returning, core, atRisk, churned, resurrected }` counts/sets. Keep it pure.
|
||
- **Acceptance:** changing `core_plays_per_week` changes segmentation without recomputation from raw completions.
|
||
- **Tests:** `tests/stats-segments.test.ts`.
|
||
|
||
### T1.3 — Snapshot builder
|
||
- **Goal:** compute and persist one snapshot for a date.
|
||
- **Files:** new `src/lib/server/stats/snapshot.ts`.
|
||
- **Steps:**
|
||
1. `buildSnapshot(asOfDate, { kind })`: query completions/verses/users with `date <= asOfDate`, compute all Phase-1 (and later-phase) aggregates, build `playerState` for active players in the trailing `active_window_days`, assemble `payload` JSON + `config` + `configVersion`, and `INSERT OR REPLACE` into `stats_snapshot`.
|
||
2. `getSnapshot(date)` reads a row. `ensureSnapshot(date)` builds with `kind: 'lazy'` if missing (guard against races: `INSERT OR IGNORE` then re-read).
|
||
3. `isSnapshotStale(date)` compares stored `configVersion` to current `configVersion()` (used to warn in UI; do **not** auto-rebuild unless triggered).
|
||
- **Acceptance:** building 2026-09-14 twice is idempotent; the payload only reflects data ≤ 2026-09-14.
|
||
- **Tests:** `tests/stats-snapshot.test.ts` — cutoff enforcement, idempotency, lazy race handling.
|
||
|
||
### T1.4 — Snapshot endpoints
|
||
- **Goal:** protected build trigger + admin read.
|
||
- **Files:** new `src/routes/api/cron/build-stats/+server.ts`; optional `src/routes/api/admin/rebuild-stats/+server.ts`.
|
||
- **Steps:** `POST /api/cron/build-stats` checks `Authorization: Bearer ${CRON_SECRET}`, builds **yesterday** (server `America/New_York`, matching `send-daily-verse`), then triggers the digest (Phase 4; no-op if not built yet). The admin rebuild accepts `{ date }` and requires `isAdmin`.
|
||
- **Acceptance:** 401 without secret; builds yesterday with secret.
|
||
- **Tests:** `tests/stats-endpoints.test.ts` — auth + date selection.
|
||
|
||
### T1.5 — Admin route shell + auth gate + date navigator
|
||
- **Goal:** `/admin/stats` renders a snapshot and can navigate time.
|
||
- **Files:** new `src/routes/admin/stats/+page.server.ts`, `+page.svelte`, `src/lib/components/stats/DateNavigator.svelte`, `src/lib/components/stats/ConsoleChrome.svelte`.
|
||
- **Steps:**
|
||
1. `+page.server.ts`: if `!isAdmin(locals.user?.email)` → 404 (mirror `/scheduled-verses` behavior). Resolve `date` from `?date=`; default = yesterday. `ensureSnapshot(date)`. Load settings + snapshot + (if `?mature=1`) a matured computation. Return `{ date, generatedAt, buildKind, payload, segments, settings, matured }`.
|
||
2. Layout: sticky header with `DateNavigator` (prev/next/date input), "Mature this view" toggle (link with `?mature=1`), link to Settings, and an as-of banner: *"Showing {date} · snapshot built {generatedAt}"*.
|
||
- **Acceptance:** non-admin gets 404; admin sees the banner and can move between dates; days with no snapshot get lazily built (flagged `lazy`).
|
||
- **Tests:** endpoint/load unit test where feasible.
|
||
|
||
### T1.6 — Pulse lab
|
||
- **Goal:** the 10-second read.
|
||
- **Files:** `src/lib/components/stats/Pulse.svelte`, `src/lib/components/charts/Sparkline.svelte`, `src/lib/components/charts/StatusPill.svelte`, `src/lib/components/charts/Gauge.svelte` (LayerChart).
|
||
- **Steps:** render KPI cards (Players today, WAU, MAU, Core, Returning share, D1 return, Net growth) with same-weekday deltas and sparklines; a verdict banner from `signals.ts`.
|
||
- **Acceptance:** all cards render from the snapshot payload with no live DB query; numbers match a manual query for a test date.
|
||
|
||
### T1.7 — Signals engine skeleton
|
||
- **Goal:** plain-language verdicts + actions.
|
||
- **Files:** new `src/lib/server/stats/signals.ts`, `src/lib/components/stats/SignalsFeed.svelte`.
|
||
- **Steps:** implement `evaluateSignals(payload, settings)` returning `{ id, severity, title, detail, action }[]`. Implement the six target rules from the plan plus a generic anomaly detector (trailing-28-day median + MAD, weekday-aware). Thresholds from `settings`.
|
||
- **Acceptance:** synthesizing payloads produces expected severities; no jargon in strings.
|
||
- **Tests:** `tests/stats-signals.test.ts`.
|
||
|
||
### T1.8 — Settings menu
|
||
- **Goal:** edit thresholds; re-segment instantly.
|
||
- **Files:** `src/lib/components/stats/SettingsMenu.svelte`, `src/routes/admin/stats/settings/+page.server.ts` (form action), or an inline panel.
|
||
- **Steps:** form posts to `updateSettings`; on save, reload the same date (segments are re-derived from `player_state`, so no rebuild needed). Show a note: *"Changing these re-reads history; snapshots themselves are unchanged."* If `configVersion` differs from a snapshot's stored version, show an "outdated snapshot definitions" notice with a rebuild link.
|
||
- **Acceptance:** changing `core_plays_per_week` updates Core counts across multiple past dates immediately.
|
||
- **Tests:** settings module already covered; add a segment re-derivation test at two thresholds.
|
||
|
||
### T1.9 — Metric glossary panel
|
||
- **Goal:** plain-language definitions on the page.
|
||
- **Files:** `src/lib/components/stats/Glossary.svelte`.
|
||
- **Steps:** render the glossary table from the plan, driven by a constant map so labels stay consistent.
|
||
- **Acceptance:** every metric label used in charts has a glossary entry.
|
||
|
||
**Phase 1 exit criteria:** `/admin/stats` shows Pulse + Signals for any date, snapshot-only loads, settings work, LayerChart installed.
|
||
|
||
---
|
||
|
||
## Phase 2 — Retention Lab & Growth Loop
|
||
|
||
### T2.1 — Add LayerChart
|
||
- **Files:** `package.json` (`bun add layerchart`), lockfile.
|
||
- **Acceptance:** `bun run build` succeeds; a trivial `<LineChart>` renders.
|
||
|
||
### T2.2 — Retention Lab
|
||
- **Files:** `src/lib/components/stats/RetentionLab.svelte`, charts: `CohortHeatmap.svelte`, `Histogram.svelte`, `StackedArea.svelte`, `BoxPlot.svelte`.
|
||
- **Steps:** cohort triangle heatmap (weekly cohorts, toggle daily), D1/D7/D30 curves as small multiples, time-to-second-play histogram, segment-mix stacked area, retention-by-segment box/grouped bars. Captions per plan; segments from `player_state`.
|
||
- **Acceptance:** heatmap cells match a manual cohort calculation; captions plain-language.
|
||
|
||
### T2.3 — Growth Loop
|
||
- **Files:** `src/lib/components/stats/GrowthLoop.svelte`, charts: `AnnotatedArea.svelte`, `Waterfall.svelte`, `Funnel.svelte`.
|
||
- **Steps:** new-players/day area with 7-day MA; if manual annotations are added later, support marker pins (design the prop now, render none if absent); waterfall (new + resurrected − churned); lifecycle funnel (players reaching 1/2/5/10/25/50 plays); signup funnel using `user.created_at` + `verse_served`/completions.
|
||
- **Acceptance:** waterfall components sum to net growth; funnel is monotonic non-increasing.
|
||
|
||
### T2.4 — Mature view
|
||
- **Files:** `src/lib/server/stats/snapshot.ts` (add `buildMatured(date)`), `+page.server.ts`.
|
||
- **Steps:** implement on-demand matured computation (uses all data after `date` for cohort resolution) and clearly label it. Do not store matured results as snapshots.
|
||
- **Acceptance:** `?mature=1` resolves old D7/D30 where possible; banner says "Matured — uses data after this date."
|
||
|
||
**Phase 2 exit criteria:** Retention & Growth labs render from snapshots; mature toggle works.
|
||
|
||
---
|
||
|
||
## Phase 3 — Verse Mix Lab & Habit Lab
|
||
|
||
### T3.1 — Verse Mix Lab
|
||
- **Files:** `src/lib/components/stats/VerseMixLab.svelte`, charts reuse `StackedBar`, `BubbleScatter.svelte`, `CoverageHeatmap.svelte`, `StepLine.svelte`.
|
||
- **Steps:** mix balance (book/section/testament stack over time), 66-book coverage heatmap, freshness (repeat intervals + entropy line), retention-by-book bubble scatter, retention-by-guess-bucket, achievability band + step curve, regression alarm table, before/after curated-mix annotated comparison. Include the "giveaway detector" table (high first-try books/windows).
|
||
- **Acceptance:** Revelation/Genesis flags as easy outliers and Micah as hard on the current data; captions explain each.
|
||
|
||
### T3.2 — Habit Lab
|
||
- **Files:** `src/lib/components/stats/HabitLab.svelte`, charts: `SurvivalLine.svelte`, `Ridgeline.svelte`, `CalendarHeatmap.svelte`, `WeekdayHourHeatmap.svelte`.
|
||
- **Steps:** streak survival curve, streak length distribution, calendar heatmap, weekday×hour heatmap (note server-time caveat in the caption).
|
||
- **Acceptance:** survival curve is monotonic; calendar matches completion counts.
|
||
|
||
**Phase 3 exit criteria:** all five labs complete; page passes `bun run check`.
|
||
|
||
---
|
||
|
||
## Phase 4 — Delivery
|
||
|
||
### T4.1 — Daily digest
|
||
- **Files:** new `src/lib/server/stats/digest.ts`; call from `src/routes/api/cron/build-stats/+server.ts`.
|
||
- **Steps:** after a successful build, compose ~6 plain lines (as-of date; DAU/WAU/MAU; new vs returning; D1; top signal; link) and POST to `DISCORD_ADMIN_WEBHOOK`. No-op with a log if unset. Never fail the build on digest failure.
|
||
- **Acceptance:** running the cron locally posts the digest (or logs a skip); build still succeeds if the webhook errors.
|
||
- **Tests:** unit test message composition.
|
||
|
||
### T4.2 — Slim public community widget
|
||
- **Files:** rewrite `src/routes/global/+page.server.ts` and `+page.svelte` (or create a `CommunityStats.svelte` embedded on `/about`).
|
||
- **Steps:** read the **latest snapshot** and render only: all-time completions, players this week/month, yesterday's completions + average guesses, 30-day calendar heatmap. Label "as of {date}." No sensitive metrics. Mobile-friendly.
|
||
- **Acceptance:** public page contains no retention/churn/signup/revenue figures; loads without heavy queries (reads snapshot).
|
||
|
||
### T4.3 — Remove/replace any lingering `/global` references
|
||
- **Steps:** update README route table and any nav links. Keep old `+page.server.ts` logic out of the new console.
|
||
- **Acceptance:** README matches reality.
|
||
|
||
### T4.4 — Docs & env
|
||
- **Steps:** update `README.md` (routes, API, schema, env: `DISCORD_ADMIN_WEBHOOK`), and `todo.md` if relevant.
|
||
- **Acceptance:** a new contributor can find `/admin/stats` and the cron endpoint from the README.
|
||
|
||
**Phase 4 exit criteria:** digest + public widget live; docs current.
|
||
|
||
---
|
||
|
||
## Phase 5 — Deferred / roadmap (do not build unless approved)
|
||
|
||
- **C — feature-usage events:** emit `mode_enter` (imposter/practice), `hint_reveal`, `progress_view`, `submit_verse_*`; add the Feature Impact panel.
|
||
- **Manual marketing annotations:** small table + UI pins on the Growth Loop timeline.
|
||
- **CSV/JSON export** of snapshot payloads.
|
||
- **Email digest** (requires `nodemailer`).
|
||
|
||
---
|
||
|
||
## Cross-cutting requirements
|
||
|
||
### C1 — Testing
|
||
- All `src/lib/server/stats/*.ts` pure logic must have unit tests with fixtures (no real DB) where possible.
|
||
- DB-touching functions tested against `TEST_DATABASE_URL`.
|
||
- Maintain the phase verification gate: `bun run check && bun test && bun run build`.
|
||
|
||
### C2 — Accessibility & mobile
|
||
- Charts keyboard-navigable where interactive; provide text/table fallbacks for color-only heatmaps.
|
||
- Responsive: KPI grid 2-col on mobile, charts fluid width; tables horizontally scrollable.
|
||
- Respect the existing dark theme (`bg-white/10 dark:bg-white/5`, etc.) and the `Container`/`CollapsibleTable` components.
|
||
|
||
### C3 — Performance
|
||
- Snapshot build is a batch job; acceptable to scan tables. Page loads must not scan raw tables.
|
||
- Keep `player_state` bounded (~180-day active window); JSON may be gzipped if it grows.
|
||
|
||
### C4 — Copy & tone
|
||
- Every chart caption ≤ 2 short sentences.
|
||
- No metric named "rate of change," "velocity," or "acceleration."
|
||
- Always include a comparison and a "what to do."
|
||
|
||
---
|
||
|
||
## Definition of Done (overall)
|
||
|
||
1. `/admin/stats` is admin-only and serves a once-daily, as-of-yesterday snapshot without live DB reads.
|
||
2. Any past date can be viewed (point-in-time), with `—` where data didn't exist.
|
||
3. "Mature this view" works and is clearly labeled.
|
||
4. All five labs + Pulse + Signals + Glossary render from snapshots using LayerChart; layouts are mobile-friendly.
|
||
5. Segment thresholds are editable in-app and re-derive all history instantly.
|
||
6. Instrumentation A/B/D/E/F + `user.created_at` actively record data.
|
||
7. Daily digest posts; slim public widget shows only safe aggregates.
|
||
8. `bun run check`, `bun test`, `bun run build` all pass; README/env docs updated.
|
||
|
||
---
|
||
|
||
## Appendix A — Snapshot `payload` shape (extend per phase)
|
||
|
||
```ts
|
||
interface SnapshotPayload {
|
||
asOfDate: string;
|
||
activity: { dau: number; wau: number; mau: number; core: number; returningShare: number; newPlayers: number; resurrected: number; churned: number; netGrowth: number };
|
||
dailySeries: { date: string; completions: number; newPlayers: number; started: number }[]; // last ~90d
|
||
kpis: { d1Return: number | null; return7: number | null; overallReturnRate: number; completionRate: number | null; shareRate: number | null; firstTryRate: number; avgGuesses: number };
|
||
retention: { cohorts: { weekStart: string; sizes: number[]; rates: (number|null)[] }[]; curves: { d1: Series; d7: Series; d30: Series }; timeToSecondPlay: Bin[]; segmentMix: { date: string; new: number; returning: number; resurrected: number; atRisk: number; churned: number }[] };
|
||
verseMix: { coverage: { bookId: string; days: number }[]; sectionMix: Series[]; entropy: Series; repeatIntervals: number[]; firstTryByBook: { bookId: string; plays: number; firstTryPct: number; avgGuesses: number }[]; retentionByBook: { bookId: string; d1: number; plays: number }[]; retentionByGuessBucket: { bucket: string; d1: number }[] };
|
||
habit: { streakSurvival: { length: number; p: number }[]; streakLengths: Bin[]; calendar: { date: string; count: number }[]; weekdayHour: number[][] };
|
||
signals: Signal[];
|
||
}
|
||
```
|
||
|
||
## Appendix B — Event names (allowlisted)
|
||
|
||
`verse_served` · `share` · `share_copy` · `first_visit` · `donate_click` · `discord_click`
|
||
Metadata examples: share `{ snippet: boolean }`; first_visit `{ referrer, utm_source, utm_medium, utm_campaign }`.
|
||
|
||
## Appendix C — Settings keys (defaults)
|
||
|
||
| Key | Default | Meaning |
|
||
|---|---|---|
|
||
| `core_plays_per_week` | 4 | Core player threshold |
|
||
| `churn_days` | 30 | Days inactive → churned |
|
||
| `resurrect_gap_days` | 30 | Gap that makes a return "resurrected" |
|
||
| `at_risk_days` | 8 | Days inactive → at-risk |
|
||
| `active_window_days` | 180 | Player-state horizon stored per snapshot |
|
||
| `d1_target` | 20 | D1 return target (%) |
|
||
| `return7_target` | 25 | 7-day return target (%) |
|
||
| `completion_target` | 70 | Started→completed target (%) |
|
||
| `first_try_min` / `first_try_max` | 15 / 35 | Achievability band (%) |
|
||
| `share_target` | 25 | Share rate target (%) |
|
||
| `anomaly_mad_threshold` | 2 | Robust z-score cutoff |
|
||
|
||
## Appendix D — Signal rule template
|
||
|
||
```ts
|
||
{ id: 'd1_return_low', severity: 'warn',
|
||
title: 'New players are not coming back',
|
||
detail: `D1 return is ${v}% (target ${t}%).`,
|
||
action: 'Test a "come back tomorrow" prompt on the win screen.' }
|
||
``` |