first pass analytics redo

README.md

@@ -44,14 +44,18 @@ bun run db:migrate   # run migrations
 bun run db:studio    # open Drizzle Studio GUI
 ```
 
+The statistics dashboard migration (`drizzle/0004_slim_nomad.sql`) is **purely additive**: it creates `stats_snapshot`, `analytics_events` and `dashboard_settings`, and adds a nullable `user.created_at`. It never alters or drops historical rows. To replicate to production, apply that SQL to `prod.db` (e.g. `sqlite3 prod.db < drizzle/0004_slim_nomad.sql`) **before** deploying the new code. Applying it twice is safe for the new tables only if you skip the existing objects; run it once.
+
 ### Environment Variables
 
 See `.env.example`. Required/used variables:
 
 - `DATABASE_URL` — path to the SQLite database file (e.g. `prod.db`)
 - `PUBLIC_SITE_URL` — canonical site URL
-- `CRON_SECRET` — bearer token protecting the cron-only `/api/send-daily-verse` endpoint
+- `CRON_SECRET` — bearer token protecting the cron-only `/api/send-daily-verse` and `/api/cron/build-stats` endpoints
 - `DISCORD_DAILY_WEBHOOK` — webhook for posting the daily verse to Discord
+- `DISCORD_ADMIN_WEBHOOK` — optional webhook for the daily statistics digest (digest is skipped when unset)
+- `STATS_*` — optional overrides for dashboard thresholds (`STATS_D1_TARGET`, `STATS_CORE_PLAYS_PER_WEEK`, …); see `src/lib/types/stats.ts` for defaults
 - `AUTH_SECRET` — secret for Apple Sign-In
 - `APPLE_ID` / `APPLE_TEAM_ID` / `APPLE_KEY_ID` / `APPLE_PRIVATE_KEY` — Apple Sign-In credentials
 - `SMTP_USERNAME` / `SMTP_TOKEN` / `SMTP_SERVER` / `SMTP_PORT` — email (nodemailer)
@@ -60,11 +64,14 @@ See `.env.example`. Required/used variables:
 
 ### Database Schema (`src/lib/server/db/schema.ts`)
 
-- **user** — `id`, `firstName`, `lastName`, `email` (unique), `passwordHash`, `appleId` (unique), `googleId` (unique), `isPrivate`
+- **user** — `id`, `firstName`, `lastName`, `email` (unique), `passwordHash`, `appleId` (unique), `googleId` (unique), `isPrivate`, `createdAt` (nullable; existing accounts show "unknown")
 - **session** — `id` (SHA-256 hash of token), `userId` (FK), `expiresAt`
 - **dailyVerses** — cached daily verse: `date` (unique), `bookId`, `verseText`, `reference`, `createdAt`
 - **dailyCompletions** — one row per player/date: `anonymousId`, `date`, `guessCount`, `guesses` (JSON of book IDs, nullable), `completedAt`. Unique on `(anonymousId, date)` to prevent duplicate submissions.
 - **verseSubmissions** — a log of community-submitted future verses: `id`, `userId` (FK → `user.id` ON DELETE SET NULL), `scheduledDate` (unique `YYYY-MM-DD` matching a `dailyVerses` row), `selectedBookId`, `selectedChapter`, `selectedVerse` (the anchor the user picked), `submittedAt` (server UTC millis, drives the 7-day cooldown). Indexed on `userId` (cooldown lookup) and `scheduledDate` (unique). The canonical verse text/reference lives on `dailyVerses`, joined by `scheduledDate`.
+- **statsSnapshot** — one immutable JSON aggregate row per as-of day (`date` PK, `generatedAt`, `buildKind`, `configVersion`, `payload`, `playerState`, `config`). Powers `/admin/stats` with point-in-time reads.
+- **analyticsEvents** — generic event log (`id`, `name`, `anonymousId`, `userId`, `date`, `createdAt`, `metadata`) with a partial unique index deduping `verse_served` per player/day.
+- **dashboardSettings** — key/value overrides for dashboard thresholds.
 
 Sessions expire after 30 days and auto-renew when fewer than 15 days remain.
 
@@ -113,17 +120,21 @@ Authenticated users who have solved today's puzzle can submit a verse for schedu
 | `/greek-random` | Debug page showing a random verse in parallel Greek (1904) / English (NKJV). |
 | `/similarity` | Search a sentence and return the most similar Bible verses via the (currently disabled) embeddings model. |
 | `/about` | About page with the project's backstory and social links. |
-| `/global` | Public stats dashboard: completions today/all-time, unique & weekly & monthly players, active streak distribution, 14-day completions trend, retention/return-rate metrics. |
+| `/global` | Slim public community-stats widget: all-time finishes/players, weekly/monthly players, yesterday's finishes + average guesses, and a 30-day finish calendar. Reads the latest daily snapshot; no retention/churn/signup data. |
 | `/progress` | Personal progress page (requires auth): activity calendar, 66-book grid with mastery tiers, insights, and achievements/milestones. |
 | `/stats` | Personal stats page (requires auth); returns `requiresAuth: true` for unauthenticated visitors and renders a sign-in modal. |
 | `/dev` | Local-time / countdown debug page. |
 | `/scheduled-verses` | Admin-only (gated to `ADMIN_EMAIL` in `src/lib/server/admin.ts`). Full historical + future log of `verseSubmissions` joined to `dailyVerses` and `user`, sorted by scheduled date. The sole surface where submitter identity is shown. Not linked from the UI. |
+| `/admin/stats` | Admin-only private **Growth Console** (gated to `ADMIN_EMAIL`). Once-daily, point-in-time snapshot; date navigator, “Mature this view”, editable thresholds, Pulse, Growth Loop, Retention, Verse Mix, Habit, Signals and Glossary. Set `?date=YYYY-MM-DD`; `?mature=1` recomputes old cohorts using later data. |
 
 ### API Endpoints
 
 | Endpoint | Description |
 | --- | --- |
-| `POST /api/daily-verse` | Fetch (and cache) the verse for a given `YYYY-MM-DD` date. |
+| `POST /api/daily-verse` | Fetch (and cache) the verse for a given `YYYY-MM-DD` date. Also records a deduped `verse_served` event for the supplied `anonymousId` (started→finished funnel). |
+| `POST /api/events` | Fire-and-forget analytics sink. Allowlisted names: `verse_served`, `share`, `share_copy`, `first_visit`, `donate_click`, `discord_click`; `userId` resolved from the session. Always returns 204. |
+| `POST /api/cron/build-stats` | Cron-only (bearer `CRON_SECRET`). Builds yesterday's snapshot (`America/New_York`) and posts the digest to `DISCORD_ADMIN_WEBHOOK`. |
+| `POST /api/admin/rebuild-stats` | Admin-only. Rebuilds one snapshot: `{ date: 'YYYY-MM-DD' }`. |
 | `POST /api/submit-completion` | Submit a game result (`anonymousId`, `date`, `guessCount`, `guesses`); returns solve rank, guess rank, total solves, average guesses, ties, and percentile. Unique on `(anonymousId, date)`. |
 | `GET /api/streak?anonymousId=X&localDate=Y` | Current streak: walks backwards from the client's local date through completed dates, counting consecutive days. Single-day streaks are reported as 0 (minimum displayed streak is 2). |
 | `GET /api/streak-percentile?streak=N&localDate=Y` | Streak percentile ranking computed across all players' current streaks. |
@@ -184,7 +195,10 @@ A streak counts consecutive calendar days (in the player's local timezone) on wh
 | `src/lib/stores/game-persistence.svelte.ts` | Reactive localStorage-backed game state |
 | `src/lib/server/db/schema.ts` | Drizzle ORM schema |
 | `src/hooks.server.ts` | Session validation hook; (commented-out) embeddings init |
-| `tests/` | Bun test suites: timezone, game, bible, stats, share, sign-in migration |
+| `tests/` | Bun test suites: timezone, game, bible, stats, share, sign-in migration, and `stats-*` (metrics, segments, settings, events, signals, snapshot, digest) |
+| `src/lib/server/stats/*` | Statistics engine: `metrics.ts`, `segments.ts`, `snapshot.ts`, `signals.ts`, `settings.ts`, `events.ts`, `digest.ts`, `dates.ts`. |
+| `src/lib/components/stats/*` | Growth Console UI (Pulse, Growth Loop, Retention/Verse Mix/Habit labs, Signals, Settings, Glossary). |
+| `src/lib/utils/analytics.ts` | Client event helper (`track`, anonymous ID, first-visit capture). |
 
 ## Deployment