bibdle/statistics-plan.md

Bibdle Statistics Dashboard — Plan

Status: design agreed with the project owner · Supersedes: the public /global dashboard Deliverable: a private, once-daily, point-in-time Growth Console at /admin/stats, plus a slim public community-stats widget.

Implementing this plan? Use the agent-ready task list in statistics-implementation-checklist.md. It is ordered, dependency-aware, and includes acceptance criteria and tests per task.

---

1. Why we're replacing /global

/global was a good start but it is now a wall of aggregates and second derivatives ("completions velocity", "user acceleration") with no verdict, no feature data, and no content data. It answers "did a number move?" but not "why, and what should I change?"

Three structural problems:

1. No "so what." Every chart is descriptive. Nothing tells you what to improve, change, or add.
2. No product surface. There is no read on features (modes, hints, sharing), funnels (started → completed → shared), or content (which books/verses help or hurt retention).
3. Wrong delivery. It queries the DB on every load, shows partial "today" data, and offers no time machine.

Goal of the new console: a product review you can skim in 60 seconds, ordered around the questions that matter, that ends every section with a plain-language verdict and recommended action — and that can be scrolled back to any past day.

Live baseline (pulled from bibdle.com/global, as of 2026-09-15)

Metric	Value
All-time completions	31,775
Unique players (all-time)	6,589
Registered users	98
DAU / WAU / MAU	102 / 363 / 852
Net player growth (7d)	~flat (−1: 175 new vs 176 churned)
Overall return rate (ever played 2×)	22.8%
7-day new-player return rate	12.2%
Survival after 2 plays → 3rd	67.2%
7-day / 30-day retention	~61% / ~51%
Difficulty	mean 4.13 guesses, median 3, 33.7% first-try
Easiest / hardest books	Revelation 1.34 avg / 83% first-try · Micah 7.84 avg / 12% first-try

The core story: acquisition is replacing churn but not growing. ~77% of all-time players played exactly once; the ~23% who return are very sticky. The leak is the first session (and possibly verse mix/achievability), not the endgame.

---

2. Decisions locked with the owner

#	Decision	Choice
1	Audience	Private founder dashboard, replacing /global internally. Reuse ADMIN_EMAIL gate (as /scheduled-verses does).
2	Public presence	Keep a slim public community-stats widget (no sensitive/business data).
3	Freshness	Once-daily snapshot, "as of yesterday." Page load never queries the DB.
4	History	Point-in-time by default, with a "Mature this view" toggle.
5	Snapshot storage	Store one immutable snapshot per day (fidelity + speed + no-data fallbacks).
6	Config	Segment thresholds configurable at render time via a settings menu (env defaults). Requires storing compact per-player state in snapshots.
7	Data sources	Existing SQLite only. No Umami, no Rybbit, self-contained and durable.
8	North star	Lead with DAU + MAU, with Core players (4+/7d) as the durability signal.
9	Signals	Both: explicit targets for ~6 core metrics + anomaly detection everywhere else.
10	Charts	LayerChart (Svelte 5 native; verified layerchart@2.5.0 peers svelte ^5).
11	Tone	Plain language. Short factual sentences. No jargon, no paragraphs next to charts.
12	Mobile	Must be mobile-friendly.
13	Digest	Scheduled daily digest of the snapshot (Discord admin webhook default; email optional).
14	Instrumentation approved	A, B, D, E, F + user.created_at (see §7). C deferred (roadmap).

Design principles (apply to every chart):

• Every visual has a plain-English caption and, where useful, a one-line "what to do."
• Never show a number without a comparison (previous period, same weekday, or baseline).
• Weekday-align daily comparisons so weekends don't masquerade as growth.
• Prefer shape over derivatives: show "players per day" and a trend line, not "acceleration."

---

3. Information architecture

/admin/stats
├─ Global chrome: date navigator · "Mature this view" toggle · Settings · as-of banner
├─ 1. Pulse                (10-second read + verdict)
├─ 2. Growth Loop          (new · active · returning · churned · funnel)
├─ 3. Retention Lab        (cohorts, curves, time-to-return, segments)
├─ 4. Verse Mix Lab        (coverage, freshness, retention by book & difficulty)
├─ 5. Habit Lab            (streaks, calendar, timing)
├─ 6. Signals & Actions    (rule-based verdicts + recommended actions)
├─ 7. Feature Impact       (stub until C is instrumented)
└─ 8. Notes & Glossary     (plain-language metric definitions)

The header is a time machine: a date picker plus prev/next arrows, always labeled e.g. "Showing 2026-09-14 (snapshot built 2026-09-15)." Days before a metric existed show "—".

---

4. The labs (what you see, why, and the chart type)

Chart types in bold. Every section ends in a one-sentence verdict from the Signals engine.

4.1 Pulse

Element	Plain-English caption	Chart
Players today	"People who finished yesterday's verse." + same-weekday change	KPI card + sparkline
Players this week / month	"Distinct players in the last 7 / 30 days."	KPI card + delta
Core players	"Players who finished 4+ times this week — your habit-holders."	KPI card + trend
Returning share	"Of yesterday's players, how many had played before."	KPI + mini stacked bar
D1 return rate	"Of people who played for the first time, how many came back the next day."	KPI + gauge vs target
Net player growth (7d)	"New and returning players minus those who stopped."	KPI + waterfall delta
Verdict banner	e.g. "Stable. Biggest leak: 88% of new players never return."	status pill

Decision it informs: which lab to open; whether the week was a win.

4.2 Growth Loop

Element	Plain-English caption	Chart
New players per day	"First-time players each day." Annotated with HN/Bluesky/Discord posts.	Annotated area + marker pins
Acquisition vs churn	"New + resurrected − churned = net."	Waterfall
Lifecycle funnel	"Of everyone who ever played: how many returned 2×, 5×, 10×, 25×, 50×."	Funnel
Retention curves	D1 / D7 / D30 over time.	Small-multiple line
Signup funnel	"Of first-time players, how many created an account." (needs user.created_at + starts)	Funnel / line

Decision it informs: is the problem acquisition or activation; did a marketing post actually move retention, not just traffic.

4.3 Retention Lab

Element	Plain-English caption	Chart
Cohort retention triangle	"Each row is a starting week; each column is weeks later. Darker = more came back."	Heatmap
Time-to-second-play	"How many days until a first-timer plays again."	Histogram
Segment mix over time	New · Returning · Resurrected · At-risk · Churned.	Stacked area
Retention by segment	"Do account-holders / core players stick longer?"	Box plot or grouped bars
Overall return rate	"Share of all players who ever played more than once."	KPI + target

Decision it informs: whether onboarding/notification changes bend retention; which segment is healthiest.

4.4 Verse Mix Lab

This lab treats verse mix and appealability (not raw difficulty) as the retention lever. Hard and easy days can both drive churn — one by frustration, one by boredom.

Element	Plain-English caption	Chart
Mix balance over time	"Are we spreading across all 66 books / 9 sections / 2 testaments?"	Stacked bars / treemap
Coverage heatmap	66 books × times featured.	Heatmap
Freshness	"How often the same book or window repeats." (repeat interval + entropy)	Histogram + line
Retention by book	For each book, the next-day return rate of players who got it.	Bubble scatter (x=plays, y=D1 return, size=first-try%)
Retention by difficulty	"Do easy or hard days make people leave?"	Line/grouped bars by guess bucket
Achievability band	First-try % and solve-by-N against a target band.	Histogram + step line
Regression alarms	Books/windows whose cohort churns above average.	Table with inline bars
Before/after the curated mix system	Same metrics pre- vs post-launch.	Annotated line

Decision it informs: whether the new curated verse-mix system works; which books/windows to retire or feature more; submissions policy.

4.5 Habit Lab

Element	Plain-English caption	Chart
Streak survival	"If you've played N days in a row, how likely are you to play day N+1."	Survival line
Streak lengths	Distribution of current streaks.	Ridgeline / histogram
Calendar heatmap	Daily completions, GitHub-style.	Calendar heatmap
Weekday × hour	When people play. (Note: completedAt is server time, approximate.)	2-D heatmap

Decision it informs: whether to invest in streaks, reminders, and Discord/content timing.

4.6 Signals & Actions

Plain-language cards, each: metric · current vs target · trend · interpretation · suggested action.

Example: "D1 return is 12% (target 20%). Most first-timers never see day two. → Test a 'come back tomorrow' prompt on the win screen." Example: "Revelation averages 1.3 guesses (83% first-try). Likely self-revealing text. → Exclude or down-weight for the curated mix."

4.7 Feature Impact (stub)

Since feature-usage events (C) are deferred, this ships as a "needs data" panel listing exactly what would be shown once C is added (Imposter entries, hint reveals, progress-page visits, submissions).

---

5. Plain-language metric glossary (shown in the page)

Each metric has a friendly name, a one-sentence definition, and a "what to do."

Metric	Plain definition	What to do if it's off
Players today	Finished the puzzle yesterday	Investigate content or traffic
WAU / MAU	Played at least once in 7 / 30 days	Reach vs habit
Core players	4+ finishes this week	If falling, retention problem
Returning share	Played before vs first time	Low = acquisition-heavy, leaky
D1 return	First-timers who came back next day	The #1 activation lever
Return rate (any)	Played again within N days	Onboarding quality
Overall return rate	Ever played more than once	Overall stickiness
Net growth	New + resurrected − churned	Is the base expanding?
Completion rate	Started → finished	Friction/confusion (needs starts)
Signup rate	Players who made an account	Account value
First-try rate	Solved with no wrong guesses	Too high = giveaway; too low = brutal
Share rate	Shares ÷ finishes	Virality (needs D)
Streak survival	Chance a streak continues	Streak feature ROI

---

6. Data model & architecture

6.1 New tables (all additive)

stats_snapshot — one immutable row per as-of day.

Column	Type	Notes
date	text PK	as-of date YYYY-MM-DD
generated_at	integer	build time
build_kind	text	scheduled | lazy | reconstructed
config_version	text	hash of thresholds used at build
payload	text (JSON)	all aggregate series/lab data
player_state	text (JSON, gzip)	compact per-player state as of that day, for render-time segmentation
config	text (JSON)	copy of thresholds used

analytics_events — generic, extensible event log (unlocks B/D/E/F now, C later).

Column	Type	Notes
id	text PK
name	text	verse_served, share, share_copy, first_visit, donate_click, discord_click, …
anonymous_id	text	pseudonymous
user_id	text, nullable
date	text, nullable	game date when relevant
created_at	integer	server timestamp
metadata	text (JSON)	referrer/utm, snippet flag, channel…

Indexes: (name, date), (anonymous_id, date). Unique on (name, anonymous_id, date) for verse_served (dedupes reloads → "players who started").

dashboard_settings — key/value (e.g. core_plays_per_week, churn_days, resurrect_gap_days, d1_target, first_try_band, share_target). Env vars provide defaults; the settings menu writes here.

user.created_at — new timestamp column (see §7).

6.2 Snapshot build

• Trigger: daily cron hits a protected POST /api/cron/build-stats (bearer CRON_SECRET, same pattern as /api/send-daily-verse). It builds the most recent date whose global data window has closed and sends the digest.
• Lazy fallback: /admin/stats load builds a missing snapshot on demand (guarded against races, marked lazy).
• Backfill: an admin action/script can reconstruct a date range from raw data (marked reconstructed).
• Computation cutoff: data is grouped by the verse date (a player's local day), not a UTC 24-hour window. A date is final after D+1 12:00Z (UTC-12 has finished the day); until then a build is a non-persisted point-in-time view. Snapshots are only persisted once the window has closed.
• Player state (for render-time config): per player active in the trailing ~180 days, a compact tuple [anonId, firstDate, lastDate, totalPlays, plays7, plays30, currentStreak]. Thresholds applied at render time, so changing them re-segments all history instantly while the state stays frozen as of that day.
• Retention/pruning: keep all snapshots (small); optionally prune player_state older than a configurable horizon later.

6.3 "Mature this view"

A toggle (off by default). When on, the selected date's metrics are recomputed on demand using data after that date (so old cohorts have resolved D7/D30), clearly labeled "Matured — uses data after this date." This is the only mode allowed to query the DB directly.

6.4 Auth & privacy

• /admin/stats gated to ADMIN_EMAIL via existing src/lib/server/admin.ts.
• Aggregate-only rendering; never display emails or raw IDs. Anonymous IDs are pseudonymous.
• /api/stats name already exists (per-user stats) — the new snapshot endpoints will use distinct names (/api/admin/stats/...) to avoid collision.

6.5 Public community widget

Redesigned slim /global (or an embeddable component) showing only: all-time completions, players this week/month, yesterday's completions and average guesses, and a 30-day calendar heatmap. No retention, churn, signup, or revenue data. Labeled "as of yesterday."

---

7. Instrumentation plan (approved)

ID	What	Where it wires in	Cost
A	Store guess sequences for all players	src/routes/api/submit-completion/+server.ts already persists guesses when provided; confirm the client always sends them and remove any conditional nulling. Historical rows stay null (correct point-in-time).	Tiny
B	Log "verse served" (game started)	Client generates/reads bibdle-anonymous-id in src/routes/+page.ts and sends it in the POST /api/daily-verse body; endpoint records a verse_served event, unique per (anonymous_id, date). Gives start→completion funnel and bounce.	Small
D	Share / virality events	In src/lib/utils/share.ts / WinScreen.svelte, record share and share_copy events (with snippet flag) via a tiny client helper. Enables share rate + viral proxy.	Small
E	Referrer / UTM capture	On first visit (client), read document.referrer and ?ref=/?utm_source=; store a first_visit event keyed to the anonymous ID. Enables acquisition source.	Small
F	Monetization clicks	Record donate_click and discord_click events from the win-screen/footer links. Enables the patron funnel.	Tiny
G	user.created_at	Add column + set on signup in src/lib/server/auth.ts (and OAuth paths). Existing users show "unknown." Enables signup-per-day, time-to-signup, and the visitor→signup funnel.	Small

Deferred (roadmap): C — feature-usage event log (Imposter/hints/progress/submissions). The analytics_events table makes this a trivial follow-on; only the emit points need adding.

---

8. Signals engine (targets + anomaly detection)

Explicit targets (defaults; editable in Settings):

Metric	Target	Red
D1 return rate	≥ 20%	< 12%
7-day return (any)	≥ 25%	< 15%
Completion rate (started→finished)	≥ 70%	< 55%
First-try rate (achievability band)	15–35%	> 45% or < 8%
Share rate	≥ 25%	< 15%
Core players (4+/7d)	non-declining WoW	down ≥ 2 weeks

Completion-rate and share-rate targets are placeholders until 2 weeks of new data exist.

Anomaly detection (everything else): compare each metric to its own trailing 28-day median (weekday-aware), flag moves beyond a robust threshold (e.g. > 2× MAD). Output is a plain sentence, not a chart annotation.

Each rule = { metric, comparator, target, severity, interpretation, recommended_action }. Stored as code (not user-editable), with thresholds from settings.

---

9. Chart inventory & LayerChart mapping

Chart	Used in	LayerChart approach
Sparkline KPI	Pulse	minimal <LineChart>
Annotated area	Growth Loop	<AreaChart> + marker overlay
Waterfall	Growth Loop	floating <Bar> segments
Funnel	Growth Loop, signups	stacked <Rect>/<Bar>
Cohort heatmap	Retention	<Chart> + grid of <Rect>
Calendar heatmap	Habit, public widget	<Chart> + weekday×week <Rect> grid
2-D heatmap	Habit	<Chart> + <Rect> grid
Bubble scatter	Verse Mix	<ScatterChart> with radius scale
Histogram	Retention, Verse Mix, Habit	<BarChart> from binned data
Step / survival line	Verse Mix, Habit	<LineChart> with step interpolation
Ridgeline	Habit	stacked <AreaChart>/density
Box plot	Retention	custom <Rect>+<Line>
Stacked area / bars	Retention, Verse Mix	<AreaChart>/<BarChart> stacked
Gauge / status pill	Pulse	custom <Chart> arc + badge
Table with inline bars	Verse Mix, Signals	existing CollapsibleTable

Rendering model: the page fetches the day's snapshot once (already computed), then renders charts client-side. No live DB reads in the normal path → matches the "once per day" requirement.

---

10. Delivery: daily digest

• Runs right after the snapshot build (same cron endpoint), once per day.
• Default channel: a dedicated Discord admin webhook (DISCORD_ADMIN_WEBHOOK), reusing the existing webhook pattern. Optionally email later (note: nodemailer is not currently a dependency; adding email = adding a dependency).
• Content (plain, ~6 lines): as-of date · DAU/WAU/MAU · new vs returning · D1 · the single top signal · link to /admin/stats.

---

11. Phasing

Phase	Scope
0 — Plumbing	Add stats_snapshot, analytics_events, dashboard_settings, user.created_at; wire instrumentation A/B/D/E/F; client event helper.
1 — Core	Snapshot builder + cron/lazy; date navigator; Pulse; Signals skeleton; Settings menu; plain-language glossary.
2 — Retention & Growth	Retention Lab (cohort triangle, curves, time-to-return, segments) + Growth Loop (waterfall, funnel, annotated area, signup funnel).
3 — Content & Habit	Verse Mix Lab (coverage, freshness, retention-by-book/difficulty, before/after) + Habit Lab (streaks, calendar, timing).
4 — Delivery	Daily digest; slim public community widget; "Mature this view".
5 — Roadmap	Feature-usage events (C) + Feature Impact panel; manual marketing-event annotations; CSV/JSON export; optional email digest.

---

12. Open questions / to finalize during build

1. Digest channel: Discord admin webhook (default) vs. adding nodemailer for email.
2. Target values in §8 — confirm after ~2 weeks of instrumented data.
3. Snapshot retention: keep player_state forever vs. prune beyond a horizon.
4. Public widget placement: standalone /global vs. a section on /about vs. both.
5. Manual marketing annotations (nice-to-have, not in approved list): a tiny table to pin HN/Bluesky/Discord posts on the Growth Loop timeline.
6. C (feature-usage events): decide when to add — the table is ready, only emit points remain.

---

13. Success criteria for this dashboard

• You can answer, in under 60 seconds and without jargon: are we growing, who's returning, and what's the one thing to fix?
• The console tells you whether a verse-mix change helped retention, not just whether guesses changed.
• You can scroll to any past day and see what was true then, with honest "—" where data didn't exist.
• Adding a future metric (e.g. feature events) requires only emitting events — no rebuild of the dashboard architecture.