bibdle/statistics-plan.md

# Bibdle Statistics Dashboard — Plan

**Status:** design agreed with the project owner · **Supersedes:** the public `/global` dashboard
**Deliverable:** a private, once-daily, point-in-time **Growth Console** at `/admin/stats`, plus a slim public community-stats widget.

> **Implementing this plan?** Use the agent-ready task list in [`statistics-implementation-checklist.md`](./statistics-implementation-checklist.md). It is ordered, dependency-aware, and includes acceptance criteria and tests per task.

---

## 1. Why we're replacing `/global`

`/global` was a good start but it is now a wall of aggregates and second derivatives ("completions velocity", "user acceleration") with **no verdict, no feature data, and no content data**. It answers *"did a number move?"* but not *"why, and what should I change?"*

Three structural problems:

1. **No "so what."** Every chart is descriptive. Nothing tells you what to improve, change, or add.
2. **No product surface.** There is no read on features (modes, hints, sharing), funnels (started → completed → shared), or content (which books/verses help or hurt retention).
3. **Wrong delivery.** It queries the DB on every load, shows partial "today" data, and offers no time machine.

**Goal of the new console:** a product review you can skim in 60 seconds, ordered around the questions that matter, that ends every section with a plain-language verdict and recommended action — and that can be scrolled back to any past day.

### Live baseline (pulled from bibdle.com/global, as of 2026-09-15)

| Metric | Value |
|---|---|
| All-time completions | 31,775 |
| Unique players (all-time) | 6,589 |
| Registered users | 98 |
| DAU / WAU / MAU | 102 / 363 / 852 |
| Net player growth (7d) | ~flat (−1: 175 new vs 176 churned) |
| Overall return rate (ever played 2×) | 22.8% |
| 7-day new-player return rate | 12.2% |
| Survival after 2 plays → 3rd | 67.2% |
| 7-day / 30-day retention | ~61% / ~51% |
| Difficulty | mean 4.13 guesses, median 3, **33.7% first-try** |
| Easiest / hardest books | Revelation 1.34 avg / 83% first-try · Micah 7.84 avg / 12% first-try |

**The core story:** acquisition is replacing churn but not growing. ~77% of all-time players played exactly once; the ~23% who return are very sticky. The leak is the **first session** (and possibly verse mix/achievability), not the endgame.

---

## 2. Decisions locked with the owner

| # | Decision | Choice |
|---|---|---|
| 1 | Audience | **Private founder dashboard**, replacing `/global` internally. Reuse `ADMIN_EMAIL` gate (as `/scheduled-verses` does). |
| 2 | Public presence | Keep a **slim public community-stats widget** (no sensitive/business data). |
| 3 | Freshness | **Once-daily snapshot, "as of yesterday."** Page load never queries the DB. |
| 4 | History | **Point-in-time by default, with a "Mature this view" toggle.** |
| 5 | Snapshot storage | **Store one immutable snapshot per day** (fidelity + speed + no-data fallbacks). |
| 6 | Config | Segment thresholds configurable at **render time** via a settings menu (env defaults). Requires storing compact per-player state in snapshots. |
| 7 | Data sources | **Existing SQLite only.** No Umami, no Rybbit, self-contained and durable. |
| 8 | North star | Lead with **DAU + MAU**, with **Core players (4+/7d)** as the durability signal. |
| 9 | Signals | **Both:** explicit targets for ~6 core metrics + anomaly detection everywhere else. |
| 10 | Charts | **LayerChart** (Svelte 5 native; verified `layerchart@2.5.0` peers `svelte ^5`). |
| 11 | Tone | **Plain language.** Short factual sentences. No jargon, no paragraphs next to charts. |
| 12 | Mobile | Must be **mobile-friendly**. |
| 13 | Digest | **Scheduled daily digest** of the snapshot (Discord admin webhook default; email optional). |
| 14 | Instrumentation approved | **A, B, D, E, F** + `user.created_at` (see §7). **C deferred** (roadmap). |

**Design principles (apply to every chart):**
- Every visual has a **plain-English caption** and, where useful, a **one-line "what to do."**
- Never show a number without a **comparison** (previous period, same weekday, or baseline).
- **Weekday-align** daily comparisons so weekends don't masquerade as growth.
- Prefer **shape** over derivatives: show "players per day" and a trend line, not "acceleration."

---

## 3. Information architecture

```
/admin/stats
├─ Global chrome: date navigator · "Mature this view" toggle · Settings · as-of banner
├─ 1. Pulse                (10-second read + verdict)
├─ 2. Growth Loop          (new · active · returning · churned · funnel)
├─ 3. Retention Lab        (cohorts, curves, time-to-return, segments)
├─ 4. Verse Mix Lab        (coverage, freshness, retention by book & difficulty)
├─ 5. Habit Lab            (streaks, calendar, timing)
├─ 6. Signals & Actions    (rule-based verdicts + recommended actions)
├─ 7. Feature Impact       (stub until C is instrumented)
└─ 8. Notes & Glossary     (plain-language metric definitions)
```

The header is a **time machine**: a date picker plus prev/next arrows, always labeled e.g. *"Showing 2026-09-14 (snapshot built 2026-09-15)."* Days before a metric existed show **"—"**.

---

## 4. The labs (what you see, why, and the chart type)

Chart types in **bold**. Every section ends in a one-sentence verdict from the Signals engine.

### 4.1 Pulse

| Element | Plain-English caption | Chart |
|---|---|---|
| Players today | "People who finished yesterday's verse." + same-weekday change | **KPI card + sparkline** |
| Players this week / month | "Distinct players in the last 7 / 30 days." | **KPI card + delta** |
| Core players | "Players who finished 4+ times this week — your habit-holders." | **KPI card + trend** |
| Returning share | "Of yesterday's players, how many had played before." | **KPI + mini stacked bar** |
| D1 return rate | "Of people who played for the first time, how many came back the next day." | **KPI + gauge vs target** |
| Net player growth (7d) | "New and returning players minus those who stopped." | **KPI + waterfall delta** |
| Verdict banner | e.g. "Stable. Biggest leak: 88% of new players never return." | **status pill** |

*Decision it informs:* which lab to open; whether the week was a win.

### 4.2 Growth Loop

| Element | Plain-English caption | Chart |
|---|---|---|
| New players per day | "First-time players each day." Annotated with HN/Bluesky/Discord posts. | **Annotated area + marker pins** |
| Acquisition vs churn | "New + resurrected − churned = net." | **Waterfall** |
| Lifecycle funnel | "Of everyone who ever played: how many returned 2×, 5×, 10×, 25×, 50×." | **Funnel** |
| Retention curves | D1 / D7 / D30 over time. | **Small-multiple line** |
| Signup funnel | "Of first-time players, how many created an account." (needs `user.created_at` + starts) | **Funnel / line** |

*Decision it informs:* is the problem acquisition or activation; did a marketing post actually move retention, not just traffic.

### 4.3 Retention Lab

| Element | Plain-English caption | Chart |
|---|---|---|
| Cohort retention triangle | "Each row is a starting week; each column is weeks later. Darker = more came back." | **Heatmap** |
| Time-to-second-play | "How many days until a first-timer plays again." | **Histogram** |
| Segment mix over time | New · Returning · Resurrected · At-risk · Churned. | **Stacked area** |
| Retention by segment | "Do account-holders / core players stick longer?" | **Box plot or grouped bars** |
| Overall return rate | "Share of all players who ever played more than once." | **KPI + target** |

*Decision it informs:* whether onboarding/notification changes bend retention; which segment is healthiest.

### 4.4 Verse Mix Lab

This lab treats **verse mix and appealability** (not raw difficulty) as the retention lever. Hard *and* easy days can both drive churn — one by frustration, one by boredom.

| Element | Plain-English caption | Chart |
|---|---|---|
| Mix balance over time | "Are we spreading across all 66 books / 9 sections / 2 testaments?" | **Stacked bars / treemap** |
| Coverage heatmap | 66 books × times featured. | **Heatmap** |
| Freshness | "How often the same book or window repeats." (repeat interval + entropy) | **Histogram + line** |
| Retention by book | For each book, the next-day return rate of players who got it. | **Bubble scatter** (x=plays, y=D1 return, size=first-try%) |
| Retention by difficulty | "Do easy or hard days make people leave?" | **Line/grouped bars by guess bucket** |
| Achievability band | First-try % and solve-by-N against a target band. | **Histogram + step line** |
| Regression alarms | Books/windows whose cohort churns above average. | **Table with inline bars** |
| Before/after the curated mix system | Same metrics pre- vs post-launch. | **Annotated line** |

*Decision it informs:* whether the new curated verse-mix system works; which books/windows to retire or feature more; submissions policy.

### 4.5 Habit Lab

| Element | Plain-English caption | Chart |
|---|---|---|
| Streak survival | "If you've played N days in a row, how likely are you to play day N+1." | **Survival line** |
| Streak lengths | Distribution of current streaks. | **Ridgeline / histogram** |
| Calendar heatmap | Daily completions, GitHub-style. | **Calendar heatmap** |
| Weekday × hour | When people play. (Note: `completedAt` is server time, approximate.) | **2-D heatmap** |

*Decision it informs:* whether to invest in streaks, reminders, and Discord/content timing.

### 4.6 Signals & Actions
Plain-language cards, each: **metric · current vs target · trend · interpretation · suggested action.**
> Example: *"D1 return is 12% (target 20%). Most first-timers never see day two. → Test a 'come back tomorrow' prompt on the win screen."*
> Example: *"Revelation averages 1.3 guesses (83% first-try). Likely self-revealing text. → Exclude or down-weight for the curated mix."*

### 4.7 Feature Impact (stub)
Since feature-usage events (C) are deferred, this ships as a **"needs data"** panel listing exactly what would be shown once C is added (Imposter entries, hint reveals, progress-page visits, submissions).

---

## 5. Plain-language metric glossary (shown in the page)

Each metric has a friendly name, a one-sentence definition, and a "what to do."

| Metric | Plain definition | What to do if it's off |
|---|---|---|
| Players today | Finished the puzzle yesterday | Investigate content or traffic |
| WAU / MAU | Played at least once in 7 / 30 days | Reach vs habit |
| Core players | 4+ finishes this week | If falling, retention problem |
| Returning share | Played before vs first time | Low = acquisition-heavy, leaky |
| D1 return | First-timers who came back next day | The #1 activation lever |
| Return rate (any) | Played again within N days | Onboarding quality |
| Overall return rate | Ever played more than once | Overall stickiness |
| Net growth | New + resurrected − churned | Is the base expanding? |
| Completion rate | Started → finished | Friction/confusion (needs starts) |
| Signup rate | Players who made an account | Account value |
| First-try rate | Solved with no wrong guesses | Too high = giveaway; too low = brutal |
| Share rate | Shares ÷ finishes | Virality (needs D) |
| Streak survival | Chance a streak continues | Streak feature ROI |

---

## 6. Data model & architecture

### 6.1 New tables (all additive)

**`stats_snapshot`** — one immutable row per as-of day.
| Column | Type | Notes |
|---|---|---|
| `date` | text PK | as-of date `YYYY-MM-DD` |
| `generated_at` | integer | build time |
| `build_kind` | text | `scheduled` \| `lazy` \| `reconstructed` |
| `config_version` | text | hash of thresholds used at build |
| `payload` | text (JSON) | all aggregate series/lab data |
| `player_state` | text (JSON, gzip) | compact per-player state as of that day, for render-time segmentation |
| `config` | text (JSON) | copy of thresholds used |

**`analytics_events`** — generic, extensible event log (unlocks B/D/E/F now, C later).
| Column | Type | Notes |
|---|---|---|
| `id` | text PK | |
| `name` | text | `verse_served`, `share`, `share_copy`, `first_visit`, `donate_click`, `discord_click`, … |
| `anonymous_id` | text | pseudonymous |
| `user_id` | text, nullable | |
| `date` | text, nullable | game date when relevant |
| `created_at` | integer | server timestamp |
| `metadata` | text (JSON) | referrer/utm, snippet flag, channel… |

Indexes: `(name, date)`, `(anonymous_id, date)`. **Unique** on `(name, anonymous_id, date)` for `verse_served` (dedupes reloads → "players who started").

**`dashboard_settings`** — key/value (e.g. `core_plays_per_week`, `churn_days`, `resurrect_gap_days`, `d1_target`, `first_try_band`, `share_target`). Env vars provide defaults; the settings menu writes here.

**`user.created_at`** — new timestamp column (see §7).

### 6.2 Snapshot build
- **Trigger:** daily cron hits a protected `POST /api/cron/build-stats` (bearer `CRON_SECRET`, same pattern as `/api/send-daily-verse`). It builds the most recent date whose global data window has closed and sends the digest.
- **Lazy fallback:** `/admin/stats` load builds a missing snapshot on demand (guarded against races, marked `lazy`).
- **Backfill:** an admin action/script can reconstruct a date range from raw data (marked `reconstructed`).
- **Computation cutoff:** data is grouped by the verse `date` (a player's local day), not a UTC 24-hour window. A date is final after `D+1 12:00Z` (UTC-12 has finished the day); until then a build is a non-persisted point-in-time view. Snapshots are only persisted once the window has closed.
- **Player state (for render-time config):** per player active in the trailing ~180 days, a compact tuple `[anonId, firstDate, lastDate, totalPlays, plays7, plays30, currentStreak]`. Thresholds applied at render time, so changing them re-segments **all history instantly** while the state stays frozen as of that day.
- **Retention/pruning:** keep all snapshots (small); optionally prune `player_state` older than a configurable horizon later.

### 6.3 "Mature this view"
A toggle (off by default). When on, the selected date's metrics are recomputed on demand using data *after* that date (so old cohorts have resolved D7/D30), clearly labeled **"Matured — uses data after this date."** This is the only mode allowed to query the DB directly.

### 6.4 Auth & privacy
- `/admin/stats` gated to `ADMIN_EMAIL` via existing `src/lib/server/admin.ts`.
- Aggregate-only rendering; never display emails or raw IDs. Anonymous IDs are pseudonymous.
- `/api/stats` name already exists (per-user stats) — the new snapshot endpoints will use distinct names (`/api/admin/stats/...`) to avoid collision.

### 6.5 Public community widget
Redesigned slim `/global` (or an embeddable component) showing only: all-time completions, players this week/month, yesterday's completions and average guesses, and a 30-day calendar heatmap. **No** retention, churn, signup, or revenue data. Labeled "as of yesterday."

---

## 7. Instrumentation plan (approved)

| ID | What | Where it wires in | Cost |
|---|---|---|---|
| **A** | Store guess sequences for **all** players | `src/routes/api/submit-completion/+server.ts` already persists `guesses` when provided; confirm the client always sends them and remove any conditional nulling. Historical rows stay null (correct point-in-time). | Tiny |
| **B** | Log "verse served" (game started) | Client generates/reads `bibdle-anonymous-id` in `src/routes/+page.ts` and sends it in the `POST /api/daily-verse` body; endpoint records a `verse_served` event, unique per `(anonymous_id, date)`. Gives start→completion funnel and bounce. | Small |
| **D** | Share / virality events | In `src/lib/utils/share.ts` / `WinScreen.svelte`, record `share` and `share_copy` events (with `snippet` flag) via a tiny client helper. Enables share rate + viral proxy. | Small |
| **E** | Referrer / UTM capture | On first visit (client), read `document.referrer` and `?ref=`/`?utm_source=`; store a `first_visit` event keyed to the anonymous ID. Enables acquisition source. | Small |
| **F** | Monetization clicks | Record `donate_click` and `discord_click` events from the win-screen/footer links. Enables the patron funnel. | Tiny |
| **G** | `user.created_at` | Add column + set on signup in `src/lib/server/auth.ts` (and OAuth paths). Existing users show "unknown." Enables signup-per-day, time-to-signup, and the visitor→signup funnel. | Small |

**Deferred (roadmap):** **C — feature-usage event log** (Imposter/hints/progress/submissions). The `analytics_events` table makes this a trivial follow-on; only the emit points need adding.

---

## 8. Signals engine (targets + anomaly detection)

**Explicit targets (defaults; editable in Settings):**

| Metric | Target | Red |
|---|---|---|
| D1 return rate | ≥ 20% | < 12% |
| 7-day return (any) | ≥ 25% | < 15% |
| Completion rate (started→finished) | ≥ 70% | < 55% |
| First-try rate (achievability band) | 15–35% | > 45% or < 8% |
| Share rate | ≥ 25% | < 15% |
| Core players (4+/7d) | non-declining WoW | down ≥ 2 weeks |

*Completion-rate and share-rate targets are placeholders until 2 weeks of new data exist.*

**Anomaly detection (everything else):** compare each metric to its own trailing 28-day median (weekday-aware), flag moves beyond a robust threshold (e.g. > 2× MAD). Output is a plain sentence, not a chart annotation.

Each rule = `{ metric, comparator, target, severity, interpretation, recommended_action }`. Stored as code (not user-editable), with thresholds from settings.

---

## 9. Chart inventory & LayerChart mapping

| Chart | Used in | LayerChart approach |
|---|---|---|
| Sparkline KPI | Pulse | minimal `<LineChart>` |
| Annotated area | Growth Loop | `<AreaChart>` + marker overlay |
| Waterfall | Growth Loop | floating `<Bar>` segments |
| Funnel | Growth Loop, signups | stacked `<Rect>`/`<Bar>` |
| Cohort heatmap | Retention | `<Chart>` + grid of `<Rect>` |
| Calendar heatmap | Habit, public widget | `<Chart>` + weekday×week `<Rect>` grid |
| 2-D heatmap | Habit | `<Chart>` + `<Rect>` grid |
| Bubble scatter | Verse Mix | `<ScatterChart>` with radius scale |
| Histogram | Retention, Verse Mix, Habit | `<BarChart>` from binned data |
| Step / survival line | Verse Mix, Habit | `<LineChart>` with step interpolation |
| Ridgeline | Habit | stacked `<AreaChart>`/density |
| Box plot | Retention | custom `<Rect>`+`<Line>` |
| Stacked area / bars | Retention, Verse Mix | `<AreaChart>`/`<BarChart>` stacked |
| Gauge / status pill | Pulse | custom `<Chart>` arc + badge |
| Table with inline bars | Verse Mix, Signals | existing `CollapsibleTable` |

**Rendering model:** the page fetches the day's snapshot once (already computed), then renders charts client-side. No live DB reads in the normal path → matches the "once per day" requirement.

---

## 10. Delivery: daily digest

- Runs right after the snapshot build (same cron endpoint), once per day.
- **Default channel:** a dedicated Discord admin webhook (`DISCORD_ADMIN_WEBHOOK`), reusing the existing webhook pattern. Optionally email later (note: `nodemailer` is **not** currently a dependency; adding email = adding a dependency).
- **Content (plain, ~6 lines):** as-of date · DAU/WAU/MAU · new vs returning · D1 · the single top signal · link to `/admin/stats`.

---

## 11. Phasing

| Phase | Scope |
|---|---|
| **0 — Plumbing** | Add `stats_snapshot`, `analytics_events`, `dashboard_settings`, `user.created_at`; wire instrumentation A/B/D/E/F; client event helper. |
| **1 — Core** | Snapshot builder + cron/lazy; date navigator; Pulse; Signals skeleton; Settings menu; plain-language glossary. |
| **2 — Retention & Growth** | Retention Lab (cohort triangle, curves, time-to-return, segments) + Growth Loop (waterfall, funnel, annotated area, signup funnel). |
| **3 — Content & Habit** | Verse Mix Lab (coverage, freshness, retention-by-book/difficulty, before/after) + Habit Lab (streaks, calendar, timing). |
| **4 — Delivery** | Daily digest; slim public community widget; "Mature this view". |
| **5 — Roadmap** | Feature-usage events (C) + Feature Impact panel; manual marketing-event annotations; CSV/JSON export; optional email digest. |

---

## 12. Open questions / to finalize during build

1. **Digest channel:** Discord admin webhook (default) vs. adding nodemailer for email.
2. **Target values** in §8 — confirm after ~2 weeks of instrumented data.
3. **Snapshot retention:** keep `player_state` forever vs. prune beyond a horizon.
4. **Public widget placement:** standalone `/global` vs. a section on `/about` vs. both.
5. **Manual marketing annotations** (nice-to-have, not in approved list): a tiny table to pin HN/Bluesky/Discord posts on the Growth Loop timeline.
6. **C (feature-usage events):** decide when to add — the table is ready, only emit points remain.

---

## 13. Success criteria for this dashboard

- You can answer, in under 60 seconds and without jargon: *are we growing, who's returning, and what's the one thing to fix?*
- The console tells you whether a **verse-mix** change helped retention, not just whether guesses changed.
- You can scroll to any past day and see what was true then, with honest "—" where data didn't exist.
- Adding a future metric (e.g. feature events) requires only emitting events — no rebuild of the dashboard architecture.