george/bibdle
1
0 Fork 0
mirror of https://github.com/pupperpowell/bibdle.git synced 2026-10-22 05:54:10 -04:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
bibdle/statistics-implementation-checklist.md
T

26 KiB
Raw Permalink Blame History

Bibdle Statistics Dashboard — Implementation Checklist

For: an AI coding agent (or human) implementing the plan in statistics-plan.md. Repository: Bibdle — SvelteKit 2 · Svelte 5 (runes) · Tailwind 4 · Bun · SQLite + Drizzle ORM. Rule: read statistics-plan.md and README.md fully before starting. This checklist is ordered and dependency-aware; do tasks top to bottom unless a task says otherwise.

0. How to use this document

  • Each task has an ID, Goal, Files, Steps, Acceptance, and Tests.
  • Do not skip the Phase 0 foundations — every later phase depends on the schema and event log.
  • Prefer pure, testable functions in src/lib/server/stats/*; keep Svelte components presentational.
  • When a task is ambiguous, prefer the smallest reversible change and leave a // TODO(stats): comment rather than inventing behavior.

1. Guardrails (non-negotiable)

  1. Do not add Umami or Rybbit integration. The dashboard is self-contained and durable. Remove/ignore any Rybbit references you touch.
  2. Do not query the DB on /admin/stats page load except to (a) build a missing snapshot lazily, or (b) serve a ?mature=1 view. Normal loads read the stored snapshot only.
  3. Point-in-time correctness: snapshot computation for date D may only use rows with date <= D (and completed_at before end of D).
  4. Plain language only. No "velocity/acceleration/derivative" jargon in UI labels or captions. Every metric gets a one-sentence caption and a "what to do."
  5. Privacy: /admin/stats is gated to ADMIN_EMAIL. Never render emails or raw anonymous IDs. The public widget must expose no retention/churn/signup/revenue data.
  6. Do not modify existing public routes (/, /progress, /stats, /imposter, etc.) beyond the specific instrumentation hooks in Phase 0. Do not change the public /global until Phase 4.
  7. No new runtime dependency except layerchart. (Email digest would require nodemailer; it is out of scope unless the owner approves — default to the Discord webhook.)
  8. Follow existing code style: Svelte 5 runes ($state, $derived, $effect, $props, $bindable), Tailwind 4 utility classes, Drizzle query builder, bun test with describe/test/expect.
  9. Migrations: never hand-edit generated SQL. Use bun run db:generate and review the diff. Be aware the drizzle/ journal references 0001_loose_kree but that .sql file is absent on disk — do not "fix" this; just add new migrations on top.
  10. Commit early and often, one phase per logical commit set, message prefix stats:.

2. Environment & commands

bun install
bun run dev                 # dev server
bun run check               # svelte-check / TS
bun test                    # all tests
bun test tests/<file>.test.ts
bun run db:generate         # generate migration from schema.ts
bun run db:migrate          # apply migrations to DATABASE_URL
  • DATABASE_URL (e.g. dev.db) drives the app DB.
  • TEST_DATABASE_URL drives drizzle.test.config.ts.
  • Existing env: CRON_SECRET, DISCORD_DAILY_WEBHOOK, ADMIN_EMAIL (hardcoded in src/lib/server/admin.ts).
  • Add DISCORD_ADMIN_WEBHOOK (digest target) to .env.example and .env.

Verification gate after every phase: bun run check clean, bun test green, bun run build succeeds.

Phase 0 — Foundations & instrumentation

T0.1 — Add tables and column to the Drizzle schema

  • Goal: add stats_snapshot, analytics_events, dashboard_settings, and user.created_at.
  • Files: src/lib/server/db/schema.ts
  • Steps:
    1. Add user.createdAt = integer('created_at', { mode: 'timestamp' }).$defaultFn(() => new Date()). ($defaultFn means new rows get a timestamp automatically; existing rows stay NULL = "unknown".)
    2. Add statsSnapshot (date text PK; generatedAt integer timestamp; buildKind text; configVersion text; payload text; playerState text; config text).
    3. Add analyticsEvents (id text PK; name text; anonymousId text; userId text nullable FK → user ON DELETE SET NULL; date text nullable; createdAt integer timestamp; metadata text). Indexes on (name, date) and (anonymousId, date); unique index analytics_events_serve_unique on (name, anonymousId, date).
    4. Add dashboardSettings (key text PK; value text; updatedAt integer timestamp).
    5. Export inferred types.
  • Steps: run bun run db:generate, inspect SQL, then bun run db:migrate.
  • Acceptance: fresh migration applies cleanly to dev.db; sqlite3 dev.db ".schema user" shows created_at; all three tables exist.
  • Tests: none (schema only) — but run full suite.

T0.2 — Settings module (thresholds with env defaults)

  • Goal: a single source of truth for configurable thresholds.
  • Files: new src/lib/server/stats/settings.ts; new src/lib/types/stats.ts (types).
  • Steps:
    1. Define keys with defaults: core_plays_per_week (4), churn_days (30), resurrect_gap_days (30), at_risk_days (8), d1_target (20), return7_target (25), completion_target (70), first_try_min (15), first_try_max (35), share_target (25), active_window_days (180), anomaly_mad_threshold (2).
    2. getSettings() reads dashboard_settings and overlays env-var defaults (e.g. STATS_D1_TARGET). updateSettings(partial) upserts rows.
    3. configVersion() = short stable hash of the resolved settings.
  • Acceptance: getSettings() returns defaults with an empty table; updateSettings persists and is reflected on re-read.
  • Tests: tests/stats-settings.test.ts — defaults, override, upsert, config hash stability.

T0.3 — Event recording module + client helper

  • Goal: a single, safe sink for analytics events (B/D/E/F).
  • Files: new src/lib/server/stats/events.ts; new src/routes/api/events/+server.ts; new src/lib/utils/analytics.ts.
  • Steps:
    1. recordEvent({ name, anonymousId, userId, date, metadata }) inserts into analytics_events. For name === 'verse_served', use INSERT OR IGNORE (dedupe on unique index).
    2. POST /api/events validates name against an allowlist (verse_served, share, share_copy, first_visit, donate_click, discord_click), caps metadata size, resolves userId from locals.user if present, and records. Return 204. Never throw to the client callers (fire-and-forget).
    3. src/lib/utils/analytics.ts exports track(name, metadata?) that POSTs to /api/events with the lib's anonymous ID (read/create bibdle-anonymous-id), ignoring errors. Also export getOrCreateAnonymousId() and refactor src/lib/stores/game-persistence.svelte.ts to import it (avoid duplicated logic).
  • Acceptance: calling track('donate_click') writes one row; repeated verse_served for the same (anon, date) writes only once.
  • Tests: tests/stats-events.test.ts — allowlist rejection, metadata cap, verse_served dedupe, user attribution.

T0.4 — Instrumentation A: guarantee guess sequences for everyone

  • Goal: all completions persist guesses.
  • Files: src/routes/api/submit-completion/+server.ts, src/routes/+page.svelte (verify), src/routes/+page.server.ts (legacy form action).
  • Steps: confirm the client always sends guesses; ensure the API stores JSON.stringify(guesses) whenever the array is non-empty. Update the legacy submitCompletion form action to accept and store guesses too (or delete the dead action if unused). Do not backfill historical rows.
  • Acceptance: a guest completion POSTed with guesses stores the JSON; SELECT COUNT(*) FROM daily_completions WHERE guesses IS NOT NULL grows for guest IDs in dev.
  • Tests: extend tests/stats.test.ts or add tests/stats-completion.test.ts.

T0.5 — Instrumentation B: log "verse served" (game started)

  • Goal: measure started → completed.
  • Files: src/routes/+page.ts, src/routes/api/daily-verse/+server.ts, src/lib/utils/analytics.ts.
  • Steps: in +page.ts, read/create the anonymous ID before the fetch and include it in the POST /api/daily-verse body. In the endpoint, record verse_served (unique per anon+date). Keep the endpoint's response shape unchanged.
  • Acceptance: loading / records exactly one verse_served per browser per day, even after reloads.
  • Tests: tests/stats-serve.test.ts (endpoint logic with a test DB) + unit test for dedupe.

T0.6 — Instrumentation D: share / copy events

  • Goal: measure virality.
  • Files: src/lib/components/WinScreen.svelte, src/lib/utils/share.ts (or call site).
  • Steps: fire track('share', { snippet }) when the native-share path is used and track('share_copy', { snippet }) on clipboard copy. Do not double-count the copy that happens inside shareResult fallback. Keep existing Umami data-umami-* attributes untouched.
  • Acceptance: clicking Share writes one share row; Copy writes one share_copy row; snippet flag correct.
  • Tests: manual + a small unit test on the helper if logic is extracted.

T0.7 — Instrumentation E: first-touch referrer / UTM

  • Goal: acquisition source.
  • Files: src/lib/utils/analytics.ts, root layout or +page.ts.
  • Steps: on first visit per browser (guard with localStorage key), capture document.referrer and ?ref= / ?utm_source= / ?utm_medium / ?utm_campaign, and record a one-time first_visit event with that metadata.
  • Acceptance: first visit with ?utm_source=bluesky records one first_visit; subsequent visits do not.
  • Tests: unit test on the parsing/guard logic.

T0.8 — Instrumentation F: monetization clicks

  • Goal: patron funnel.
  • Files: src/lib/components/WinScreen.svelte (Donate, Discord), src/lib/components/SocialLinks.svelte if desired.
  • Steps: fire track('donate_click') and track('discord_click') on click (keep the existing Umami attributes).
  • Acceptance: clicks record events with the right names.

T0.9 — Add DISCORD_ADMIN_WEBHOOK env

  • Files: .env.example, .env (local only), README.md env list.
  • Acceptance: documented; not required at boot (digest degrades gracefully when unset).

Phase 0 exit criteria: migrations applied; events flowing in dev; bun run check + bun test green.

Phase 1 — Core console (snapshot + Pulse + Signals skeleton)

T1.1 — Pure metric functions

  • Goal: tested, deterministic metrics independent of Svelte/DB reads.
  • Files: new src/lib/server/stats/metrics.ts, types in src/lib/types/stats.ts.
  • Steps: implement, each taking plain rows and a cutoff date:
    • activePlayers(completions, end, windowDays)
    • newVsReturning(completions, date) (returning = first-ever completion < date)
    • netGrowth(completions, end, windowDays, settings)
    • d1Return(completions, cohortDate) and returnWithin(completions, cohortDate, n)
    • cohortTriangle(completions, { start, weeks })
    • timeToSecondPlayHistogram(completions, buckets)
    • guessHistogram(completions, maxBucket) + firstTryRate
    • streakSurvival(completions)
    • bookCoverage(verses, completions, start, end)
    • retentionByBook(completions, verses, minPlays) and retentionByGuessBucket
    • entropy(bookCounts)
    • weekdayHourHeatmap(completions)
  • Acceptance: pure functions; no DB imports; correct on hand-built fixtures.
  • Tests: tests/stats-metrics.test.ts — one test per function incl. edge cases (empty, single player, day boundaries in UTC).

T1.2 — Segment derivation from stored player state

  • Goal: render-time configurable segments.
  • Files: new src/lib/server/stats/segments.ts.
  • Steps: define PlayerState = { id, firstDate, lastDate, totalPlays, plays7, plays30, streak }. deriveSegments(playerState[], asOfDate, settings) returns { new, returning, core, atRisk, churned, resurrected } counts/sets. Keep it pure.
  • Acceptance: changing core_plays_per_week changes segmentation without recomputation from raw completions.
  • Tests: tests/stats-segments.test.ts.

T1.3 — Snapshot builder

  • Goal: compute and persist one snapshot for a date.
  • Files: new src/lib/server/stats/snapshot.ts.
  • Steps:
    1. buildSnapshot(asOfDate, { kind }): query completions/verses/users with date <= asOfDate, compute all Phase-1 (and later-phase) aggregates, build playerState for active players in the trailing active_window_days, assemble payload JSON + config + configVersion, and INSERT OR REPLACE into stats_snapshot.
    2. getSnapshot(date) reads a row. ensureSnapshot(date) builds with kind: 'lazy' if missing (guard against races: INSERT OR IGNORE then re-read).
    3. isSnapshotStale(date) compares stored configVersion to current configVersion() (used to warn in UI; do not auto-rebuild unless triggered).
  • Acceptance: building 2026-09-14 twice is idempotent; the payload only reflects data ≤ 2026-09-14.
  • Tests: tests/stats-snapshot.test.ts — cutoff enforcement, idempotency, lazy race handling.

T1.4 — Snapshot endpoints

  • Goal: protected build trigger + admin read.
  • Files: new src/routes/api/cron/build-stats/+server.ts; optional src/routes/api/admin/rebuild-stats/+server.ts.
  • Steps: POST /api/cron/build-stats checks Authorization: Bearer ${CRON_SECRET}, builds yesterday (server America/New_York, matching send-daily-verse), then triggers the digest (Phase 4; no-op if not built yet). The admin rebuild accepts { date } and requires isAdmin.
  • Acceptance: 401 without secret; builds yesterday with secret.
  • Tests: tests/stats-endpoints.test.ts — auth + date selection.

T1.5 — Admin route shell + auth gate + date navigator

  • Goal: /admin/stats renders a snapshot and can navigate time.
  • Files: new src/routes/admin/stats/+page.server.ts, +page.svelte, src/lib/components/stats/DateNavigator.svelte, src/lib/components/stats/ConsoleChrome.svelte.
  • Steps:
    1. +page.server.ts: if !isAdmin(locals.user?.email) → 404 (mirror /scheduled-verses behavior). Resolve date from ?date=; default = yesterday. ensureSnapshot(date). Load settings + snapshot + (if ?mature=1) a matured computation. Return { date, generatedAt, buildKind, payload, segments, settings, matured }.
    2. Layout: sticky header with DateNavigator (prev/next/date input), "Mature this view" toggle (link with ?mature=1), link to Settings, and an as-of banner: "Showing {date} · snapshot built {generatedAt}".
  • Acceptance: non-admin gets 404; admin sees the banner and can move between dates; days with no snapshot get lazily built (flagged lazy).
  • Tests: endpoint/load unit test where feasible.

T1.6 — Pulse lab

  • Goal: the 10-second read.
  • Files: src/lib/components/stats/Pulse.svelte, src/lib/components/charts/Sparkline.svelte, src/lib/components/charts/StatusPill.svelte, src/lib/components/charts/Gauge.svelte (LayerChart).
  • Steps: render KPI cards (Players today, WAU, MAU, Core, Returning share, D1 return, Net growth) with same-weekday deltas and sparklines; a verdict banner from signals.ts.
  • Acceptance: all cards render from the snapshot payload with no live DB query; numbers match a manual query for a test date.

T1.7 — Signals engine skeleton

  • Goal: plain-language verdicts + actions.
  • Files: new src/lib/server/stats/signals.ts, src/lib/components/stats/SignalsFeed.svelte.
  • Steps: implement evaluateSignals(payload, settings) returning { id, severity, title, detail, action }[]. Implement the six target rules from the plan plus a generic anomaly detector (trailing-28-day median + MAD, weekday-aware). Thresholds from settings.
  • Acceptance: synthesizing payloads produces expected severities; no jargon in strings.
  • Tests: tests/stats-signals.test.ts.

T1.8 — Settings menu

  • Goal: edit thresholds; re-segment instantly.
  • Files: src/lib/components/stats/SettingsMenu.svelte, src/routes/admin/stats/settings/+page.server.ts (form action), or an inline panel.
  • Steps: form posts to updateSettings; on save, reload the same date (segments are re-derived from player_state, so no rebuild needed). Show a note: "Changing these re-reads history; snapshots themselves are unchanged." If configVersion differs from a snapshot's stored version, show an "outdated snapshot definitions" notice with a rebuild link.
  • Acceptance: changing core_plays_per_week updates Core counts across multiple past dates immediately.
  • Tests: settings module already covered; add a segment re-derivation test at two thresholds.

T1.9 — Metric glossary panel

  • Goal: plain-language definitions on the page.
  • Files: src/lib/components/stats/Glossary.svelte.
  • Steps: render the glossary table from the plan, driven by a constant map so labels stay consistent.
  • Acceptance: every metric label used in charts has a glossary entry.

Phase 1 exit criteria: /admin/stats shows Pulse + Signals for any date, snapshot-only loads, settings work, LayerChart installed.

Phase 2 — Retention Lab & Growth Loop

T2.1 — Add LayerChart

  • Files: package.json (bun add layerchart), lockfile.
  • Acceptance: bun run build succeeds; a trivial <LineChart> renders.

T2.2 — Retention Lab

  • Files: src/lib/components/stats/RetentionLab.svelte, charts: CohortHeatmap.svelte, Histogram.svelte, StackedArea.svelte, BoxPlot.svelte.
  • Steps: cohort triangle heatmap (weekly cohorts, toggle daily), D1/D7/D30 curves as small multiples, time-to-second-play histogram, segment-mix stacked area, retention-by-segment box/grouped bars. Captions per plan; segments from player_state.
  • Acceptance: heatmap cells match a manual cohort calculation; captions plain-language.

T2.3 — Growth Loop

  • Files: src/lib/components/stats/GrowthLoop.svelte, charts: AnnotatedArea.svelte, Waterfall.svelte, Funnel.svelte.
  • Steps: new-players/day area with 7-day MA; if manual annotations are added later, support marker pins (design the prop now, render none if absent); waterfall (new + resurrected − churned); lifecycle funnel (players reaching 1/2/5/10/25/50 plays); signup funnel using user.created_at + verse_served/completions.
  • Acceptance: waterfall components sum to net growth; funnel is monotonic non-increasing.

T2.4 — Mature view

  • Files: src/lib/server/stats/snapshot.ts (add buildMatured(date)), +page.server.ts.
  • Steps: implement on-demand matured computation (uses all data after date for cohort resolution) and clearly label it. Do not store matured results as snapshots.
  • Acceptance: ?mature=1 resolves old D7/D30 where possible; banner says "Matured — uses data after this date."

Phase 2 exit criteria: Retention & Growth labs render from snapshots; mature toggle works.

Phase 3 — Verse Mix Lab & Habit Lab

T3.1 — Verse Mix Lab

  • Files: src/lib/components/stats/VerseMixLab.svelte, charts reuse StackedBar, BubbleScatter.svelte, CoverageHeatmap.svelte, StepLine.svelte.
  • Steps: mix balance (book/section/testament stack over time), 66-book coverage heatmap, freshness (repeat intervals + entropy line), retention-by-book bubble scatter, retention-by-guess-bucket, achievability band + step curve, regression alarm table, before/after curated-mix annotated comparison. Include the "giveaway detector" table (high first-try books/windows).
  • Acceptance: Revelation/Genesis flags as easy outliers and Micah as hard on the current data; captions explain each.

T3.2 — Habit Lab

  • Files: src/lib/components/stats/HabitLab.svelte, charts: SurvivalLine.svelte, Ridgeline.svelte, CalendarHeatmap.svelte, WeekdayHourHeatmap.svelte.
  • Steps: streak survival curve, streak length distribution, calendar heatmap, weekday×hour heatmap (note server-time caveat in the caption).
  • Acceptance: survival curve is monotonic; calendar matches completion counts.

Phase 3 exit criteria: all five labs complete; page passes bun run check.

Phase 4 — Delivery

T4.1 — Daily digest

  • Files: new src/lib/server/stats/digest.ts; call from src/routes/api/cron/build-stats/+server.ts.
  • Steps: after a successful build, compose ~6 plain lines (as-of date; DAU/WAU/MAU; new vs returning; D1; top signal; link) and POST to DISCORD_ADMIN_WEBHOOK. No-op with a log if unset. Never fail the build on digest failure.
  • Acceptance: running the cron locally posts the digest (or logs a skip); build still succeeds if the webhook errors.
  • Tests: unit test message composition.

T4.2 — Slim public community widget

  • Files: rewrite src/routes/global/+page.server.ts and +page.svelte (or create a CommunityStats.svelte embedded on /about).
  • Steps: read the latest snapshot and render only: all-time completions, players this week/month, yesterday's completions + average guesses, 30-day calendar heatmap. Label "as of {date}." No sensitive metrics. Mobile-friendly.
  • Acceptance: public page contains no retention/churn/signup/revenue figures; loads without heavy queries (reads snapshot).

T4.3 — Remove/replace any lingering /global references

  • Steps: update README route table and any nav links. Keep old +page.server.ts logic out of the new console.
  • Acceptance: README matches reality.

T4.4 — Docs & env

  • Steps: update README.md (routes, API, schema, env: DISCORD_ADMIN_WEBHOOK), and todo.md if relevant.
  • Acceptance: a new contributor can find /admin/stats and the cron endpoint from the README.

Phase 4 exit criteria: digest + public widget live; docs current.

Phase 5 — Deferred / roadmap (do not build unless approved)

  • C — feature-usage events: emit mode_enter (imposter/practice), hint_reveal, progress_view, submit_verse_*; add the Feature Impact panel.
  • Manual marketing annotations: small table + UI pins on the Growth Loop timeline.
  • CSV/JSON export of snapshot payloads.
  • Email digest (requires nodemailer).

Cross-cutting requirements

C1 — Testing

  • All src/lib/server/stats/*.ts pure logic must have unit tests with fixtures (no real DB) where possible.
  • DB-touching functions tested against TEST_DATABASE_URL.
  • Maintain the phase verification gate: bun run check && bun test && bun run build.

C2 — Accessibility & mobile

  • Charts keyboard-navigable where interactive; provide text/table fallbacks for color-only heatmaps.
  • Responsive: KPI grid 2-col on mobile, charts fluid width; tables horizontally scrollable.
  • Respect the existing dark theme (bg-white/10 dark:bg-white/5, etc.) and the Container/CollapsibleTable components.

C3 — Performance

  • Snapshot build is a batch job; acceptable to scan tables. Page loads must not scan raw tables.
  • Keep player_state bounded (~180-day active window); JSON may be gzipped if it grows.

C4 — Copy & tone

  • Every chart caption ≤ 2 short sentences.
  • No metric named "rate of change," "velocity," or "acceleration."
  • Always include a comparison and a "what to do."

Definition of Done (overall)

  1. /admin/stats is admin-only and serves a once-daily, as-of-yesterday snapshot without live DB reads.
  2. Any past date can be viewed (point-in-time), with — where data didn't exist.
  3. "Mature this view" works and is clearly labeled.
  4. All five labs + Pulse + Signals + Glossary render from snapshots using LayerChart; layouts are mobile-friendly.
  5. Segment thresholds are editable in-app and re-derive all history instantly.
  6. Instrumentation A/B/D/E/F + user.created_at actively record data.
  7. Daily digest posts; slim public widget shows only safe aggregates.
  8. bun run check, bun test, bun run build all pass; README/env docs updated.

Appendix A — Snapshot payload shape (extend per phase)

interface SnapshotPayload {
  asOfDate: string;
  activity: { dau: number; wau: number; mau: number; core: number; returningShare: number; newPlayers: number; resurrected: number; churned: number; netGrowth: number };
  dailySeries: { date: string; completions: number; newPlayers: number; started: number }[]; // last ~90d
  kpis: { d1Return: number | null; return7: number | null; overallReturnRate: number; completionRate: number | null; shareRate: number | null; firstTryRate: number; avgGuesses: number };
  retention: { cohorts: { weekStart: string; sizes: number[]; rates: (number|null)[] }[]; curves: { d1: Series; d7: Series; d30: Series }; timeToSecondPlay: Bin[]; segmentMix: { date: string; new: number; returning: number; resurrected: number; atRisk: number; churned: number }[] };
  verseMix: { coverage: { bookId: string; days: number }[]; sectionMix: Series[]; entropy: Series; repeatIntervals: number[]; firstTryByBook: { bookId: string; plays: number; firstTryPct: number; avgGuesses: number }[]; retentionByBook: { bookId: string; d1: number; plays: number }[]; retentionByGuessBucket: { bucket: string; d1: number }[] };
  habit: { streakSurvival: { length: number; p: number }[]; streakLengths: Bin[]; calendar: { date: string; count: number }[]; weekdayHour: number[][] };
  signals: Signal[];
}

Appendix B — Event names (allowlisted)

verse_served · share · share_copy · first_visit · donate_click · discord_click Metadata examples: share { snippet: boolean }; first_visit { referrer, utm_source, utm_medium, utm_campaign }.

Appendix C — Settings keys (defaults)

Key Default Meaning
core_plays_per_week 4 Core player threshold
churn_days 30 Days inactive → churned
resurrect_gap_days 30 Gap that makes a return "resurrected"
at_risk_days 8 Days inactive → at-risk
active_window_days 180 Player-state horizon stored per snapshot
d1_target 20 D1 return target (%)
return7_target 25 7-day return target (%)
completion_target 70 Started→completed target (%)
first_try_min / first_try_max 15 / 35 Achievability band (%)
share_target 25 Share rate target (%)
anomaly_mad_threshold 2 Robust z-score cutoff

Appendix D — Signal rule template

{ id: 'd1_return_low', severity: 'warn',
  title: 'New players are not coming back',
  detail: `D1 return is ${v}% (target ${t}%).`,
  action: 'Test a "come back tomorrow" prompt on the win screen.' }
Reference in new issue View Git Blame Copy Permalink