4 Commits
Author SHA1 Message Date
George Powell f9aded56f8 feat(stats): add activity buckets, WAU projection, and shared community stats
- Add a weekly vs monthly players chart to Pulse, counted over whole
  completed weeks (Mon-Sun) and calendar months instead of rolling windows.
- Add a Projection section below Pulse that estimates future weekly players
  three ways (flow, regression, seasonal) with a shaded likely range.
- Pull the public community stats into one shared component used by both the
  public page and the admin console, and show yesterday's verse on the
  public page.
- Show "Resurrected" instead of "Churned" in the segment mix chart, since
  churned only ever grows and flattens the rest.
- Make full-width charts fill their container and line the hover cursor up
  correctly; show 3 months of new players per day.
- Hide the verse mix lab for now.
2026-09-30 17:00:48 -04:00
George Powell 4d5893ce14 chore(docs): note community stats and admin graph todos 2026-09-28 20:21:49 -04:00
George Powell da29ee7196 feat(stats): quantile-scaled, annotated calendar heatmap
Replace the linear count/max alpha with quantile buckets (GitHub-style)
so a single outlier day no longer flattens the rest of the graph, and
render best/median/worst day labels inside their cells.

- Add buildCalendarScale utility with empty-bucket handling.
- Make CalendarHeatmap size, gap, and annotations configurable; align
  weeks to Monday.
- Expand the community calendar to 90 days, add play totals/averages and
  all-time best day, and format large numbers with locale separators.
2026-09-28 20:21:48 -04:00
George Powell a562ee3cc9 fix(stats): finalize snapshots on the verse-day window
A verse date is a player's local calendar day, spanning D-1 10:00Z
(UTC+14) through D+1 12:00Z (UTC-12). The old UTC end-of-day cutoff both
dropped late-night finishes in negative timezones and could persist a
partial day.

- Scope snapshot loads by the verse date column; use completedBefore only
  to pin an in-progress point-in-time build.
- Gate persistence on isDayClosed (D+1 12:00Z); open days are computed
  live and never written.
- Build the cron snapshot for latestClosedDate instead of EST yesterday,
  and read the same date for the community page.
- Add all-time best single day to snapshot totals.
- Bump PAYLOAD_SCHEMA_VERSION to 4 and extend snapshot/window tests.
2026-09-28 20:21:47 -04:00
25 changed files with 1726 additions and 147 deletions

No files matched your search

+1 -1
View File
@@ -133,7 +133,7 @@ Authenticated users who have solved today's puzzle can submit a verse for schedu
| --- | --- |
| `POST /api/daily-verse` | Fetch (and cache) the verse for a given `YYYY-MM-DD` date. Also records a deduped `verse_served` event for the supplied `anonymousId` (started→finished funnel). |
| `POST /api/events` | Fire-and-forget analytics sink. Allowlisted names: `verse_served`, `share`, `share_copy`, `first_visit`, `donate_click`, `discord_click`; `userId` resolved from the session. Always returns 204. |
| `POST /api/cron/build-stats` | Cron-only (bearer `CRON_SECRET`). Builds yesterday's snapshot (`America/New_York`) and posts the digest to `DISCORD_ADMIN_WEBHOOK`. |
| `POST /api/cron/build-stats` | Cron-only (bearer `CRON_SECRET`). Builds the most recent snapshot whose global data window has closed (`D+1 12:00Z`) and posts the digest to `DISCORD_ADMIN_WEBHOOK`. |
| `POST /api/admin/rebuild-stats` | Admin-only. Rebuilds one snapshot: `{ date: 'YYYY-MM-DD' }`. |
| `POST /api/submit-completion` | Submit a game result (`anonymousId`, `date`, `guessCount`, `guesses`); returns solve rank, guess rank, total solves, average guesses, ties, and percentile. Unique on `(anonymousId, date)`. |
| `GET /api/streak?anonymousId=X&localDate=Y` | Current streak: walks backwards from the client's local date through completed dates, counting consecutive days. Single-day streaks are reported as 0 (minimum displayed streak is 2). |
@@ -1,22 +1,39 @@
<script lang="ts">
import { buildCalendarScale } from '$lib/utils/calendar-scale';
interface Props {
data: { date: string; count: number }[];
color?: string;
/** Pixel size of each day square. */
cellSize?: number;
/** Gap between squares in pixels. */
gap?: number;
/** Optional date → short label rendered inside the square. */
annotations?: Record<string, string>;
}
let { data, color = '52,211,153' }: Props = $props();
let {
data,
color = '52,211,153',
cellSize = 12,
gap = 3,
annotations = {}
}: Props = $props();
const values = $derived(new Map(data.map((d) => [d.date, d.count])));
const max = $derived(Math.max(1, ...data.map((d) => d.count)));
const scale = $derived(buildCalendarScale(data.map((d) => d.count)));
const fontSize = $derived(Math.max(8, Math.round(cellSize * 0.42)));
const radius = $derived(Math.max(2, Math.round(cellSize * 0.16)));
// Build Sunday-aligned weeks ending on the last date.
// Build Monday-aligned weeks ending on the last date (weekend at the bottom).
const weeks = $derived.by(() => {
if (data.length === 0) return [] as { date: string; inRange: boolean }[][];
const mondayOffset = (date: Date) => (date.getUTCDay() + 6) % 7;
const first = new Date(data[0].date + 'T00:00:00Z');
const start = new Date(first);
start.setUTCDate(start.getUTCDate() - start.getUTCDay());
start.setUTCDate(start.getUTCDate() - mondayOffset(start));
const last = new Date(data[data.length - 1].date + 'T00:00:00Z');
const end = new Date(last);
end.setUTCDate(end.getUTCDate() + (6 - end.getUTCDay()));
end.setUTCDate(end.getUTCDate() + (6 - mondayOffset(end)));
const range = new Set(data.map((d) => d.date));
const out: { date: string; inRange: boolean }[][] = [];
let cursor = new Date(start);
@@ -34,25 +51,55 @@
</script>
<div class="overflow-x-auto">
<div class="flex gap-1">
<div class="flex flex-col justify-between py-0.5 text-[9px] text-gray-500">
<span>Sun</span><span>Wed</span><span>Sat</span>
<div class="flex" style={`gap:${Math.max(4, gap)}px`}>
<div
class="flex flex-col justify-between py-0.5 text-gray-500"
style={`font-size:${fontSize}px`}
>
<span>Mon</span><span>Thu</span><span>Sun</span>
</div>
<div class="flex gap-[3px]">
<div class="flex" style={`gap:${gap}px`}>
{#each weeks as week, wi (wi)}
<div class="flex flex-col gap-[3px]">
<div class="flex flex-col" style={`gap:${gap}px`}>
{#each week as cell (cell.date)}
{@const count = values.get(cell.date) ?? 0}
{@const alpha = scale.alphaFor(count) || 0.06}
{@const label = annotations[cell.date]}
<div
class="h-3 w-3 rounded-[2px]"
style={cell.inRange
? `background:rgba(${color},${(0.12 + 0.88 * (count / max)).toFixed(3)})`
: 'background:transparent'}
title={cell.inRange ? `${cell.date}: ${count} finishes` : cell.date}
></div>
class="cell-label flex items-center justify-center font-semibold tabular-nums"
data-pale={label && alpha < 0.4 ? 'true' : undefined}
style={`width:${cellSize}px;height:${cellSize}px;border-radius:${radius}px;font-size:${fontSize}px;line-height:1;${
cell.inRange ? `background:rgba(${color},${alpha})` : 'background:transparent'
}`}
title={cell.inRange
? `${cell.date}: ${count.toLocaleString('en-US')} finishes`
: cell.date}
>{label ?? ''}</div>
{/each}
</div>
{/each}
</div>
</div>
</div>
</div>
<style>
/*
* Annotated numbers need to read on both light and dark cell backgrounds.
* Intensity is the dominant factor: the bright/high buckets are light enough
* for dark ink in either theme, while the lowest bucket is a dark green on a
* dark page and needs light ink there.
*/
.cell-label {
color: rgb(17 24 39 / 0.95);
}
:global(html.dark) .cell-label[data-pale='true'] {
color: rgb(249 250 251 / 0.95);
}
@media (prefers-color-scheme: dark) {
:global(html:not(.light)) .cell-label[data-pale='true'] {
color: rgb(249 250 251 / 0.95);
}
}
</style>
+138 -23
View File
@@ -4,6 +4,14 @@
color: string;
points: { date: string; value: number | null }[];
step?: boolean;
/** Draw the line dashed (e.g. a projection). */
dash?: boolean;
}
/** Optional shaded uncertainty band, aligned index-for-index with the series. */
interface Band {
lower: (number | null)[];
upper: (number | null)[];
color: string;
}
interface Props {
series: Series[];
@@ -12,6 +20,13 @@
yMin?: number;
yMax?: number;
showLegend?: boolean;
/**
* Axis labels for calendar buckets, indexed like the points. Falls back
* to each point's date when omitted.
*/
xLabels?: string[];
/** Shaded range drawn behind the lines (p10–p90 style). */
band?: Band;
}
let {
series,
@@ -19,16 +34,42 @@
ySuffix = '',
yMin,
yMax,
showLegend = true
showLegend = true,
xLabels,
band
}: Props = $props();
const W = 600;
/**
* The viewBox tracks the element's pixel width so the plot genuinely fills
* a full-width container. A fixed viewBox would be letterboxed and centred
* by the svg's "meet" scaling, leaving dead space at the sides and breaking
* the pointer -> index mapping. Falls back to 600 before first measurement.
*/
const FALLBACK_W = 600;
const MIN_W = 280;
let wrapperEl: HTMLDivElement | undefined = $state();
let measuredWidth = $state(0);
const W = $derived(Math.max(MIN_W, measuredWidth || FALLBACK_W));
$effect(() => {
const el = wrapperEl;
if (!el) return;
const observer = new ResizeObserver((entries) => {
measuredWidth = Math.round(entries[0]?.contentRect.width ?? 0);
});
observer.observe(el);
return () => observer.disconnect();
});
const pad = { l: 36, r: 10, t: 10, b: 22 };
const innerW = W - pad.l - pad.r;
const innerW = $derived(W - pad.l - pad.r);
const innerH = $derived(height - pad.t - pad.b);
const allValues = $derived(
series.flatMap((s) => s.points.map((p) => p.value)).filter((v): v is number => v !== null)
[
...series.flatMap((s) => s.points.map((p) => p.value)),
...(band ? [...band.lower, ...band.upper] : [])
].filter((v): v is number => v !== null && v !== undefined)
);
const domain = $derived.by(() => {
const lo = yMin ?? (allValues.length ? Math.min(...allValues) : 0);
@@ -64,25 +105,81 @@
current.push(`${xAt(i).toFixed(1)},${yAt(p.value).toFixed(1)}`);
});
if (current.length) segments.push(current.join(' '));
return { name: s.name, color: s.color, segments, step: s.step };
return { name: s.name, color: s.color, segments, step: s.step, dash: s.dash };
})
);
const firstLabel = $derived(series[0]?.points[0]?.date?.slice(5) ?? '');
const lastLabel = $derived(
series[0]?.points[series[0].points.length - 1]?.date?.slice(5) ?? ''
);
/**
* Filled band path: for each contiguous run where both bounds exist, trace the
* upper edge left-to-right and the lower edge right-to-left, then close.
*/
const bandPath = $derived.by(() => {
if (!band) return '';
const n = Math.max(band.lower.length, band.upper.length);
const parts: string[] = [];
let run: number[] = [];
const flush = () => {
if (run.length < 2) {
run = [];
return;
}
const top = run.map((i) => `${xAt(i).toFixed(1)},${yAt(band.upper[i] as number).toFixed(1)}`);
const bottom = [...run]
.reverse()
.map((i) => `${xAt(i).toFixed(1)},${yAt(band.lower[i] as number).toFixed(1)}`);
const edge = [top[0], ...top.slice(1).map((p) => `L${p}`), ...bottom.map((p) => `L${p}`)];
parts.push(`M${edge.join(' ')} Z`);
run = [];
};
for (let i = 0; i < n; i++) {
const lo = band.lower[i];
const hi = band.upper[i];
if (lo === null || lo === undefined || hi === null || hi === undefined) flush();
else run.push(i);
}
flush();
return parts.join(' ');
});
function labelAt(i: number): string {
return xLabels?.[i] ?? series[0]?.points[i]?.date ?? '';
}
/** Axis ticks fall back to MM-DD so a 90-day chart's labels stay compact. */
function axisLabelAt(i: number): string {
const label = xLabels?.[i];
if (label !== undefined) return label;
return (series[0]?.points[i]?.date ?? '').slice(5);
}
const firstLabel = $derived.by(() => (pointCount > 0 ? axisLabelAt(0) : ''));
const lastLabel = $derived.by(() => (pointCount > 0 ? axisLabelAt(pointCount - 1) : ''));
// A middle tick only once the plot is wide enough that the three labels
// cannot collide (roughly 70px per label at this font size).
const midIndex = $derived(Math.floor((pointCount - 1) / 2));
const showMidLabel = $derived(innerW > 240 && pointCount > 4);
const midLabel = $derived(showMidLabel ? axisLabelAt(midIndex) : '');
// --- Hover / tooltip state -------------------------------------------------
let wrapperEl: HTMLDivElement | undefined = $state();
let svgEl: SVGSVGElement | undefined = $state();
let hoverIndex = $state<number | null>(null);
/** Letterbox transform of the viewBox inside the element, refreshed on move. */
let box = $state({ left: 0, scale: 1, width: 1 });
function handlePointerMove(event: PointerEvent) {
if (!wrapperEl || pointCount <= 0) return;
const rect = wrapperEl.getBoundingClientRect();
if (rect.width === 0) return;
const scale = rect.width / W;
const svgX = (event.clientX - rect.left) / scale;
if (!svgEl || pointCount <= 0) return;
const rect = svgEl.getBoundingClientRect();
if (rect.width === 0 || rect.height === 0) return;
// The svg scales its viewBox to fit the element ("meet"), so on a
// container wider than the viewBox aspect ratio the drawing is
// letterboxed and centred. Map the pointer through that same transform,
// otherwise the right half of a full-width chart reports the wrong index.
const scale = Math.min(rect.width / W, rect.height / height);
const offsetX = (rect.width - W * scale) / 2;
box = { left: offsetX, scale, width: rect.width };
const svgX = (event.clientX - rect.left - offsetX) / scale;
if (pointCount <= 1) {
hoverIndex = 0;
return;
@@ -109,18 +206,18 @@
const hoveredDate = $derived.by(() => {
if (hoverIndex === null) return null;
for (const s of series) {
const point = s.points[hoverIndex];
if (point) return point.date;
}
return null;
return labelAt(hoverIndex) || null;
});
const hasHoverValues = $derived(hoverRows.some((r) => r.value !== null));
const tooltipLeft = $derived(
hoverIndex === null ? 50 : Math.min(85, Math.max(15, (xAt(hoverIndex) / W) * 100))
);
/** Tooltip position in wrapper-relative %, following the letterboxed plot. */
const tooltipLeft = $derived.by(() => {
if (hoverIndex === null || box.width === 0) return 50;
const px = box.left + xAt(hoverIndex) * box.scale;
const pctLeft = (px / box.width) * 100;
return Math.min(88, Math.max(12, pctLeft));
});
function formatValue(value: number | null): string {
if (value === null) return '—';
@@ -138,6 +235,7 @@
onpointercancel={clearHover}
>
<svg
bind:this={svgEl}
viewBox={`0 0 ${W} ${height}`}
class="w-full text-gray-400"
style={`height:${height}px`}
@@ -162,6 +260,10 @@
{domain.lo.toFixed(0)}{ySuffix}
</text>
{#if band && bandPath}
<path d={bandPath} fill={band.color} opacity="0.16" stroke="none" />
{/if}
{#each paths as s (s.name)}
{#each s.segments as d}
<polyline
@@ -171,6 +273,7 @@
stroke-width="1.75"
stroke-linejoin="round"
stroke-linecap="round"
stroke-dasharray={s.dash ? '4 3' : undefined}
vector-effect="non-scaling-stroke"
opacity={hoverIndex === null ? 1 : 0.45}
style="transition:opacity 120ms ease"
@@ -209,6 +312,18 @@
<text x={pad.l} y={height - 6} fill="currentColor" fill-opacity="0.8" font-size="9">
{firstLabel}
</text>
{#if showMidLabel && midLabel}
<text
x={xAt(midIndex)}
y={height - 6}
text-anchor="middle"
fill="currentColor"
fill-opacity="0.6"
font-size="9"
>
{midLabel}
</text>
{/if}
<text
x={W - pad.r}
y={height - 6}
@@ -0,0 +1,141 @@
<script lang="ts">
import Container from '$lib/components/Container.svelte';
import CalendarHeatmap from '$lib/components/charts/CalendarHeatmap.svelte';
import { formatDateLabel, numOrDash } from '$lib/utils/stats-format';
import type { CommunityStatsData } from '$lib/types/stats';
interface Props {
data: CommunityStatsData;
/**
* `auto` follows the site light/dark theme (public page); `dark` forces
* the dark palette for the always-dark admin console.
*/
tone?: 'auto' | 'dark';
}
let { data, tone = 'auto' }: Props = $props();
const calendar = $derived(data.calendar);
const active = $derived(calendar.filter((d) => d.count > 0));
const total = $derived(calendar.reduce((sum, d) => sum + d.count, 0));
const avgPerDay = $derived(calendar.length ? total / calendar.length : 0);
const bestDay = $derived(
active.reduce<{ date: string; count: number } | null>(
(best, d) => (best === null || d.count > best.count ? d : best),
null
)
);
const worstDay = $derived(
active.reduce<{ date: string; count: number } | null>(
(worst, d) => (worst === null || d.count < worst.count ? d : worst),
null
)
);
const medianDay = $derived.by(() => {
if (active.length === 0) return null;
const sorted = [...active].sort((a, b) => a.count - b.count);
return sorted[Math.floor((sorted.length - 1) / 2)];
});
/** Compact enough to sit inside a day square. */
function compactCount(n: number): string {
if (n >= 10000) return `${Math.round(n / 1000)}k`;
if (n >= 1000) return `${(n / 1000).toFixed(1)}k`;
return String(n);
}
const annotations = $derived.by(() => {
const out: Record<string, string> = {};
for (const day of [bestDay, medianDay, worstDay]) {
if (day) out[day.date] = compactCount(day.count);
}
return out;
});
const palette = $derived(
tone === 'dark'
? {
heading: 'text-gray-100',
sub: 'text-gray-400',
label: 'text-gray-400',
value: 'text-gray-100',
muted: 'text-gray-500'
}
: {
heading: 'text-gray-900 dark:text-gray-100',
sub: 'text-gray-600 dark:text-gray-400',
label: 'text-gray-600 dark:text-gray-400',
value: 'text-gray-900 dark:text-gray-100',
muted: 'text-gray-500 dark:text-gray-400'
}
);
</script>
<div>
<div class="mb-6 text-center">
<h1 class="text-3xl font-bold {palette.heading}">Community Stats</h1>
<p class="mt-1 text-sm {palette.sub}">As of {formatDateLabel(data.asOfDate)}</p>
</div>
<div class="grid grid-cols-2 gap-3 md:grid-cols-3">
{#each [
{ label: 'All-time plays', value: numOrDash(data.completions) },
{ label: 'All-time players', value: numOrDash(data.players) },
{ label: 'Players this week', value: numOrDash(data.weeklyPlayers) },
{ label: 'Players this month', value: numOrDash(data.monthlyPlayers) },
{ label: "Latest day's plays", value: numOrDash(data.yesterdayCompletions) },
{ label: 'Avg guesses that day', value: numOrDash(data.yesterdayAvgGuesses) }
] as card (card.label)}
<Container class="p-4 w-full items-start text-left">
<p class="text-[11px] font-semibold uppercase tracking-wider {palette.label}">{card.label}</p>
<p class="text-2xl font-bold {palette.value}">{card.value}</p>
</Container>
{/each}
</div>
<Container class="mt-3 p-4 w-full items-stretch text-left gap-3">
<p class="font-semibold {palette.heading}">Daily plays (last {calendar.length} days)</p>
<div class="flex flex-col gap-4 sm:flex-row sm:items-start">
<div class="flex flex-col gap-1">
<CalendarHeatmap data={calendar} cellSize={22} annotations={annotations} />
<p class="text-[10px] {palette.muted}">
Numbers mark the best, median, and least popular days.
</p>
</div>
<div class="flex min-w-0 flex-1 flex-col gap-4">
<dl class="grid grid-cols-2 gap-x-4 gap-y-3 text-sm">
<div>
<dt class="text-[11px] uppercase tracking-wider {palette.muted}">{calendar.length}-day plays</dt>
<dd class="font-semibold tabular-nums {palette.value}">{numOrDash(total)}</dd>
</div>
<div>
<dt class="text-[11px] uppercase tracking-wider {palette.muted}">Daily average</dt>
<dd class="font-semibold tabular-nums {palette.value}">
{avgPerDay.toLocaleString('en-US', { maximumFractionDigits: 1 })}
</dd>
</div>
{#if bestDay}
<div>
<dt class="text-[11px] uppercase tracking-wider {palette.muted}">Best day ({calendar.length}d)</dt>
<dd class="font-semibold tabular-nums {palette.value}">
{numOrDash(bestDay.count)}
<span class="font-normal {palette.muted}">on {formatDateLabel(bestDay.date)}</span>
</dd>
</div>
{/if}
{#if data.bestDayAllTime}
<div>
<dt class="text-[11px] uppercase tracking-wider {palette.muted}">Best day · all-time</dt>
<dd class="font-semibold tabular-nums {palette.value}">
{numOrDash(data.bestDayAllTime.count)}
<span class="font-normal {palette.muted}">on {formatDateLabel(data.bestDayAllTime.date)}</span>
</dd>
</div>
{/if}
</dl>
</div>
</div>
</Container>
</div>
+2
View File
@@ -11,6 +11,8 @@
const GLOSSARY: GlossaryEntry[] = [
{ name: 'Players today', definition: 'Finished the puzzle yesterday.', action: 'Investigate content or traffic.' },
{ name: 'WAU / MAU', definition: 'Played at least once in the last 7 / 30 days.', action: 'Reach vs habit.' },
{ name: 'Weekly vs monthly players', definition: 'Every completed ISO week (Mon–Sun) or calendar month, up to the last fully closed one. Week buckets count the whole week; month buckets count the whole month.', action: 'A shrinking gap means fewer monthly players play weekly.' },
{ name: 'Resurrected', definition: `Came back after a break of 30+ days (see Settings).`, action: 'Did something bring them back?' },
{ name: 'Core players', definition: 'Finished 4+ times this week.', action: 'If falling, it is a retention problem.' },
{ name: 'Returning share', definition: 'Played before today vs. first time.', action: 'Low = acquisition-heavy and leaky.' },
{ name: 'D1 return', definition: 'First-timers who came back the next day.', action: 'The number one activation lever.' },
+3 -2
View File
@@ -10,7 +10,8 @@
}
let { payload }: Props = $props();
const series = $derived(payload.dailySeries.slice(-60));
// The full 90-day window: 3 months of daily points.
const series = $derived(payload.dailySeries);
const hasStarted = $derived(payload.meta.hasStartedData);
const signupTotal = $derived(payload.growth.signups.reduce((a, b) => a + b.count, 0));
</script>
@@ -20,7 +21,7 @@
<Container class="p-4 w-full items-stretch text-left gap-2">
<p class="font-semibold text-gray-100">New players per day</p>
<p class="text-xs text-gray-400">First-time players each day. Hover later for post annotations.</p>
<p class="text-xs text-gray-400">First-time players each day, last 3 months. Hover later for post annotations.</p>
<MiniLineChart
series={[{ name: 'New players', color: '#60a5fa', points: series.map((d) => ({ date: d.date, value: d.newPlayers })) }]}
showLegend={false}
+147
View File
@@ -0,0 +1,147 @@
<script lang="ts">
import Container from '$lib/components/Container.svelte';
import MiniLineChart from '$lib/components/charts/MiniLineChart.svelte';
import type { ProjectionPoint, SnapshotPayload } from '$lib/types/stats';
import { numOrDash } from '$lib/utils/stats-format';
interface Props {
payload: SnapshotPayload;
}
let { payload }: Props = $props();
type Method = 'flow' | 'regression' | 'seasonal';
let method: Method = $state('flow');
/** Only the most recent stretch of history is plotted; the models use all of it. */
const HISTORY_WEEKS = 52;
const weeks = $derived(payload.activity.stickiness?.week ?? []);
const projection = $derived(payload.activity.projection ?? null);
const methods: { value: Method; label: string; hint: string }[] = [
{
value: 'flow',
label: 'Flow',
hint: 'Retention × current base + new/returning inflow, fitted over all weeks. Converges to a saturation level; shaded band is a p10–p90 bootstrap.'
},
{
value: 'regression',
label: 'Regression',
hint: 'OLS on lagged WAU and this week’s new/resurrected inflow. Band from residual bootstrap.'
},
{
value: 'seasonal',
label: 'Seasonal',
hint: 'One annual harmonic plus a damped linear trend. Meaningful once there is close to a full year of history; band from residual bootstrap.'
}
];
const chart = $derived.by(() => {
if (!projection || weeks.length === 0) return null;
const points: ProjectionPoint[] = projection[method];
if (points.length === 0) return null;
const recent = weeks.slice(-HISTORY_WEEKS);
const n = recent.length;
const total = n + points.length;
const actual: (number | null)[] = Array(total).fill(null);
const projected: (number | null)[] = Array(total).fill(null);
const lower: (number | null)[] = Array(total).fill(null);
const upper: (number | null)[] = Array(total).fill(null);
const labels: string[] = Array(total).fill('');
const dates: string[] = Array(total).fill('');
for (let i = 0; i < n; i++) {
actual[i] = recent[i].wau;
labels[i] = recent[i].label;
dates[i] = recent[i].date;
}
// Anchor the projection to the last actual point so the lines connect.
projected[n - 1] = recent[n - 1].wau;
lower[n - 1] = recent[n - 1].wau;
upper[n - 1] = recent[n - 1].wau;
for (let k = 0; k < points.length; k++) {
const idx = n + k;
projected[idx] = points[k].value;
lower[idx] = points[k].low;
upper[idx] = points[k].high;
labels[idx] = points[k].label;
dates[idx] = points[k].date;
}
const toLine = (values: (number | null)[]) =>
values.map((value, i) => ({ date: dates[i], value }));
return {
series: [
{ name: 'Actual WAU', color: '#60a5fa', points: toLine(actual) },
{ name: 'Projected', color: '#a78bfa', dash: true, points: toLine(projected) }
],
band: { lower, upper, color: '#a78bfa' },
xLabels: labels
};
});
const activeHint = $derived(methods.find((m) => m.value === method)?.hint ?? '');
const lastWau = $derived(weeks.at(-1)?.wau ?? null);
const projectedLast = $derived(projection?.[method].at(-1) ?? null);
function toggleClass(value: Method): string {
return method === value
? 'bg-white/15 text-gray-100'
: 'text-gray-400 hover:text-gray-200';
}
</script>
<section class="flex flex-col gap-3">
<div class="flex flex-wrap items-center justify-between gap-2">
<h2 class="text-lg font-bold text-gray-100">Projection · Weekly active users</h2>
<div class="flex shrink-0 gap-1 rounded-lg border border-white/10 p-0.5">
{#each methods as m (m.value)}
<button
type="button"
onclick={() => (method = m.value)}
aria-pressed={method === m.value}
class="rounded-md px-2.5 py-1 text-xs font-medium transition-colors {toggleClass(m.value)}"
>
{m.label}
</button>
{/each}
</div>
</div>
<Container class="p-4 w-full items-stretch text-left gap-2">
{#if chart && projection}
<p class="text-xs text-gray-400">{activeHint}</p>
<div class="flex flex-wrap gap-x-6 gap-y-1 text-xs text-gray-400">
<span>Latest actual: <span class="font-semibold text-gray-200">{numOrDash(lastWau)}</span></span>
<span>
Retention:
<span class="font-semibold text-gray-200">{Math.round(projection.retention * 100)}%</span>/wk
</span>
<span>Weekly inflow: <span class="font-semibold text-gray-200">{numOrDash(projection.inflow)}</span></span>
<span>History: <span class="font-semibold text-gray-200">{projection.sampleWeeks}w</span></span>
<span>Horizon: <span class="font-semibold text-gray-200">{projection.horizonWeeks}w</span></span>
</div>
<MiniLineChart series={chart.series} xLabels={chart.xLabels} band={chart.band} yMin={0} />
<p class="text-xs text-gray-400">
Projected WAU in {projectedLast?.label}:
<span class="font-semibold text-gray-200">{numOrDash(projectedLast?.value ?? null)}</span>
{#if projectedLast}
<span class="text-gray-500">
(80% range {numOrDash(projectedLast.low)}–{numOrDash(projectedLast.high)})</span
>
{/if}
· solid = actual, dashed = projection.
</p>
<p class="text-[11px] text-gray-500">
Models are fitted on all {projection.sampleWeeks} completed weeks. Bands are model uncertainty from
resampling, not real-world coverage; treat long horizons with caution.
</p>
{:else}
<p class="text-sm text-gray-400">Not enough completed weeks of history to project yet.</p>
{/if}
</Container>
</section>
+96 -1
View File
@@ -1,9 +1,17 @@
<script lang="ts">
import KpiCard from './KpiCard.svelte';
import StatusPill from '$lib/components/charts/StatusPill.svelte';
import type { Signal, SnapshotPayload, StatsSettings } from '$lib/types/stats';
import MiniLineChart from '$lib/components/charts/MiniLineChart.svelte';
import type {
ActivityBucket,
BucketGranularity,
Signal,
SnapshotPayload,
StatsSettings
} from '$lib/types/stats';
import {
delta,
formatDateLabel,
numOrDash,
pctOrDash,
signed
@@ -30,6 +38,49 @@
? 'bad'
: 'warn'
);
// WAU vs MAU, bucketed by calendar period rather than a rolling window.
let granularity: BucketGranularity = $state('week');
const buckets: ActivityBucket[] = $derived(payload.activity.stickiness?.[granularity] ?? []);
const latestBucket = $derived(buckets.at(-1) ?? null);
const priorBucket = $derived(buckets.at(-2) ?? null);
const stickiness = $derived(
latestBucket && latestBucket.mau > 0
? Math.round((latestBucket.wau / latestBucket.mau) * 100)
: null
);
// In week mode WAU is the headline series; in month mode it is MAU.
const comparisonMetric = $derived(granularity === 'week' ? ('wau' as const) : ('mau' as const));
const comparisonLabel = $derived(granularity === 'week' ? 'Weekly players' : 'Monthly players');
const stickinessDelta = $derived(
latestBucket && priorBucket
? delta(latestBucket[comparisonMetric], priorBucket[comparisonMetric])
: null
);
const granularityOptions: { value: BucketGranularity; label: string }[] = [
{ value: 'week', label: 'Weeks' },
{ value: 'month', label: 'Months' }
];
const bucketSeries = $derived([
{
name: 'Weekly players',
color: '#60a5fa',
points: buckets.map((b) => ({ date: b.date, value: b.wau }))
},
{
name: 'Monthly players',
color: '#a78bfa',
points: buckets.map((b) => ({ date: b.date, value: b.mau }))
}
]);
function toggleClass(value: BucketGranularity): string {
return granularity === value
? 'bg-white/15 text-gray-100'
: 'text-gray-400 hover:text-gray-200';
}
</script>
<section class="flex flex-col gap-3">
@@ -100,4 +151,48 @@
caption={`New + returning − churned (${payload.activity.newPlayers} + ${payload.activity.resurrected} − ${payload.activity.churned}).`}
/>
</div>
<div
class="flex flex-col gap-3 rounded-2xl border border-white/10 bg-white/5 p-3 text-sm text-gray-200"
>
<div class="flex flex-wrap items-center justify-between gap-2">
<div>
<p class="font-semibold text-gray-100">Weekly vs monthly players</p>
<p class="text-xs text-gray-400">
{#if latestBucket}
Latest completed {granularity === 'week' ? 'week' : 'month'}
({latestBucket.label}) ended {formatDateLabel(latestBucket.date)}: {numOrDash(latestBucket.wau)}
weekly vs {numOrDash(latestBucket.mau)} monthly
{#if stickiness !== null}
— {stickiness}% stickiness
{/if}.
{:else}
Not enough history yet.
{/if}
</p>
</div>
<div class="flex shrink-0 gap-1 rounded-lg border border-white/10 p-0.5">
{#each granularityOptions as opt (opt.value)}
<button
type="button"
onclick={() => (granularity = opt.value)}
aria-pressed={granularity === opt.value}
class="rounded-md px-2.5 py-1 text-xs font-medium transition-colors {toggleClass(opt.value)}"
>
{opt.label}
</button>
{/each}
</div>
</div>
<MiniLineChart series={bucketSeries} xLabels={buckets.map((b) => b.label)} yMin={0} />
{#if stickinessDelta && priorBucket}
<p class="text-xs text-gray-400">
{comparisonLabel} {stickinessDelta.text} vs the previous
{granularity === 'week' ? 'week' : 'month'}
({priorBucket.label}).
</p>
{/if}
</div>
</section>
+12 -4
View File
@@ -13,7 +13,10 @@
let { payload, settings }: Props = $props();
const mix = $derived(payload.retention.segmentMix);
const colors = ['#60a5fa', '#34d399', '#a78bfa', '#f59e0b', '#fb7185'];
// New / Core / Returning / Resurrected (recently returning) / At risk.
// "Churned" is deliberately not plotted: it counts every player idle for
// churn_days+, so it only ever grows and flattens the other series.
const colors = ['#60a5fa', '#34d399', '#a78bfa', '#f59e0b'];
</script>
<section class="flex flex-col gap-3">
@@ -67,7 +70,12 @@
<Container class="p-4 w-full items-stretch text-left gap-2">
<p class="font-semibold text-gray-100">Segment mix over time</p>
<p class="text-xs text-gray-400">
Active players split into new, core, returning, at-risk and churned. Total tracked:
Players re-segmented on each of the past 30 days, over the last
{settings.active_window_days} days of history.
New = first week. Core = {settings.core_plays_per_week}+ plays this week. Returning = active
again, recently. Resurrected = recently returning after a break of
{settings.resurrect_gap_days}+ days. At risk = {settings.at_risk_days}+ days idle; churned
({settings.churn_days}+ days idle) is counted in the tiles above instead. Total tracked:
{numOrDash(payload.activity.mau)} recently active.
</p>
<MiniLineChart
@@ -75,8 +83,8 @@
{ name: 'New', color: colors[0], points: mix.map((m) => ({ date: m.date, value: m.new })) },
{ name: 'Core', color: colors[1], points: mix.map((m) => ({ date: m.date, value: m.core })) },
{ name: 'Returning', color: colors[2], points: mix.map((m) => ({ date: m.date, value: m.returning })) },
{ name: 'At risk', color: colors[3], points: mix.map((m) => ({ date: m.date, value: m.atRisk })) },
{ name: 'Churned', color: colors[4], points: mix.map((m) => ({ date: m.date, value: m.churned })) }
{ name: 'Resurrected', color: colors[3], points: mix.map((m) => ({ date: m.date, value: m.resurrected })) },
{ name: 'At risk', color: colors[4], points: mix.map((m) => ({ date: m.date, value: m.atRisk })) }
]}
/>
</Container>
+47
View File
@@ -21,6 +21,32 @@ export function startOfWeek(dateStr: string): string {
return addDays(dateStr, -day);
}
/** The Monday on or before `dateStr` (ISO weeks run Monday–Sunday). */
export function startOfIsoWeek(dateStr: string): string {
const day = new Date(dateStr + 'T00:00:00Z').getUTCDay(); // 0 = Sunday
return addDays(dateStr, day === 0 ? -6 : 1 - day);
}
/**
* ISO-8601 week number and week-year for `dateStr`.
*
* ISO weeks start on Monday and week 1 is the week containing the first
* Thursday of the year, so a date in late December can belong to week 1 of the
* following week-year and early January can belong to week 52/53 of the
* previous one.
*/
export function isoWeek(dateStr: string): { week: number; year: number } {
const d = new Date(dateStr + 'T00:00:00Z');
const day = d.getUTCDay() || 7; // Monday = 1 … Sunday = 7
// Move to the Thursday of the same ISO week; its calendar year is the
// ISO week-year and the number of weeks since Jan 1 gives the week number.
d.setUTCDate(d.getUTCDate() + 4 - day);
const year = d.getUTCFullYear();
const yearStart = Date.UTC(year, 0, 1);
const week = Math.ceil(((d.getTime() - yearStart) / DAY_MS + 1) / 7);
return { week, year };
}
/** All dates from `start` to `end` inclusive. */
export function dateRange(start: string, end: string): string[] {
const out: string[] = [];
@@ -32,6 +58,27 @@ export function todayUtc(): string {
return new Date().toISOString().slice(0, 10);
}
// A verse `date` is the player's *local* calendar date, not a fixed UTC window.
// Globally the same verse date is played from D-1 10:00Z (UTC+14) through
// D+1 12:00Z (UTC-12), so a day's data is only final after D+1 12:00Z.
const HOUR_MS = 60 * 60 * 1000;
const VERSE_WINDOW_MS = 36 * HOUR_MS; // D+1 12:00Z, relative to D 00:00Z
/** Instant after which no further plays for verse `dateStr` are expected. */
export function endOfVerseWindow(dateStr: string): Date {
return new Date(new Date(dateStr + 'T00:00:00Z').getTime() + VERSE_WINDOW_MS);
}
/** True once the global data window for `dateStr` has closed. */
export function isDayClosed(dateStr: string, now: Date = new Date()): boolean {
return now.getTime() >= endOfVerseWindow(dateStr).getTime();
}
/** Most recent verse date whose global data window has closed. */
export function latestClosedDate(now: Date = new Date()): string {
return new Date(now.getTime() - VERSE_WINDOW_MS).toISOString().slice(0, 10);
}
export function isValidDate(dateStr: string): boolean {
return /^\d{4}-\d{2}-\d{2}$/.test(dateStr) && !Number.isNaN(new Date(dateStr + 'T00:00:00Z').getTime());
}
+446 -2
View File
@@ -1,12 +1,16 @@
import type {
ActivityBucket,
BucketGranularity,
BookStat,
CohortRow,
CompletionRow,
HistogramBin,
VerseRow
ProjectionPoint,
VerseRow,
WauProjection
} from '$lib/types/stats';
import { bibleBooks } from '$lib/types/bible';
import { addDays, dateRange, dayDiff, startOfWeek } from './dates';
import { addDays, dateRange, dayDiff, isoWeek, startOfIsoWeek, startOfWeek } from './dates';
// ── internal helpers ─────────────────────────────────────────────────────────
@@ -133,6 +137,446 @@ export function netGrowth(
return { newPlayers, resurrected, churned, netGrowth: newPlayers + resurrected - churned };
}
/** Last day of the calendar month containing `dateStr`. */
function endOfMonth(dateStr: string): string {
const d = new Date(dateStr + 'T00:00:00Z');
return new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth() + 1, 0))
.toISOString()
.slice(0, 10);
}
/** First day of the calendar month containing `dateStr`. */
function startOfMonth(dateStr: string): string {
return `${dateStr.slice(0, 7)}-01`;
}
/**
* WAU vs MAU bucketed by whole calendar periods, from the first period that
* contains data up to the most recent *completed* one.
*
* Only fully completed periods are emitted, so the in-progress week/month never
* drags the series down. Weeks are ISO (Monday–Sunday); months are calendar
* months.
*
* Each bucket is anchored on its last day:
* - week buckets: `wau` is the whole calendar week, `mau` the 30 days ending it
* - month buckets: `mau` is the whole calendar month, `wau` the 7 days ending it
*/
export function activityBuckets(
completions: CompletionRow[],
asOfDate: string,
granularity: BucketGranularity
): ActivityBucket[] {
let earliest: string | null = null;
for (const c of completions) {
if (earliest === null || c.date < earliest) earliest = c.date;
}
if (earliest === null) return [];
// Period ends, most recent first while building, then reversed.
const ends: string[] = [];
const starts: string[] = [];
if (granularity === 'week') {
// The most recent completed ISO week is the one whose Sunday is on or
// before `asOfDate` (a week is complete once its Sunday has closed).
let end = addDays(startOfIsoWeek(asOfDate), 6);
while (end > asOfDate) end = addDays(end, -7);
const firstStart = startOfIsoWeek(earliest);
while (end >= firstStart) {
ends.push(end);
starts.push(addDays(end, -6));
end = addDays(end, -7);
}
} else {
// The most recent completed month is the one whose last day is on or
// before `asOfDate`; otherwise it is the previous month.
let end = endOfMonth(asOfDate);
if (end > asOfDate) end = endOfMonth(addDays(startOfMonth(asOfDate), -1));
const firstStart = startOfMonth(earliest);
while (end >= firstStart) {
ends.push(end);
starts.push(startOfMonth(end));
end = endOfMonth(addDays(startOfMonth(end), -1));
}
}
ends.reverse(); // oldest first, most recent last
starts.reverse();
// Widest (earliest) window start per bucket, used to skip rows that are too
// old. Both arrays are ascending, so a completion touches a contiguous run.
const wauStart =
granularity === 'week' ? starts : ends.map((e) => addDays(e, -6));
const mauStart =
granularity === 'week' ? ends.map((e) => addDays(e, -29)) : starts;
// One pass over the rows instead of one pass per bucket. `ends` is
// ascending, so binary-search the first bucket that has not already ended;
// from there a completion touches a contiguous run until the widest window
// starts after it.
const wau = ends.map(() => new Set<string>());
const mau = ends.map(() => new Set<string>());
for (const c of completions) {
let lo = 0;
let hi = ends.length;
while (lo < hi) {
const mid = (lo + hi) >> 1;
if (ends[mid] < c.date) lo = mid + 1;
else hi = mid;
}
for (let i = lo; i < ends.length; i++) {
if (c.date < mauStart[i]) break; // too early for this and every later bucket
if (c.date >= wauStart[i]) wau[i].add(c.anonymousId);
mau[i].add(c.anonymousId);
}
}
const weeks = ends.map((e) => isoWeek(e));
const months = ends.map((e) => new Date(e + 'T00:00:00Z').getUTCMonth());
const years = ends.map((e) => new Date(e + 'T00:00:00Z').getUTCFullYear());
const spansYears =
granularity === 'week'
? new Set(weeks.map((w) => w.year)).size > 1
: new Set(years).size > 1;
return ends.map((date, i) => {
let label: string;
if (granularity === 'week') {
label = `W${weeks[i].week}${
spansYears ? ` '${String(weeks[i].year).slice(2)}` : ''
}`;
} else {
label =
new Date(date + 'T00:00:00Z').toLocaleDateString('en-US', {
month: 'short',
timeZone: 'UTC'
}) + (spansYears || months[i] === 0 ? ` ${String(years[i]).slice(2)}` : '');
}
return { date, label, wau: wau[i].size, mau: mau[i].size };
});
}
// ── projection ───────────────────────────────────────────────────────────────
export interface WeeklyFlow {
/** Last day (Sunday) of the ISO week. */
end: string;
wau: number;
/** Players active both this week and the previous one. */
retained: number;
/** Players active this week but not the previous one (new + resurrected). */
inflow: number;
/** First-ever players this week. */
newPlayers: number;
/** Players returning after a gap this week (inflow − newPlayers). */
resurrected: number;
}
/**
* Week-over-week flow accounting for every completed ISO week: how much of the
* previous week's active base returned, and how many active players were new to
* the active set. This is the input to the WAU projection.
*/
export function weeklyFlows(
completions: CompletionRow[],
asOfDate: string
): WeeklyFlow[] {
const weeks = activityBuckets(completions, asOfDate, 'week');
if (weeks.length === 0) return [];
const start0 = addDays(weeks[0].date, -6);
const sets = weeks.map(() => new Set<string>());
const firsts = firstPlayDates(completions);
const newCounts = weeks.map(() => 0);
for (const c of completions) {
const idx = Math.floor(dayDiff(start0, c.date) / 7);
if (idx >= 0 && idx < sets.length) sets[idx].add(c.anonymousId);
}
for (const first of firsts.values()) {
const idx = Math.floor(dayDiff(start0, first) / 7);
if (idx >= 0 && idx < newCounts.length) newCounts[idx]++;
}
const out: WeeklyFlow[] = [];
for (let i = 0; i < weeks.length; i++) {
const current = sets[i];
const previous = i > 0 ? sets[i - 1] : null;
let retained = 0;
if (previous) {
for (const id of current) if (previous.has(id)) retained++;
}
const newPlayers = newCounts[i];
const inflow = current.size - retained;
out.push({
end: weeks[i].date,
wau: current.size,
retained,
inflow,
newPlayers,
resurrected: Math.max(0, inflow - newPlayers)
});
}
return out;
}
function meanOf(xs: number[]): number {
return xs.length ? xs.reduce((sum, x) => sum + x, 0) / xs.length : 0;
}
/** Ordinary least squares with a tiny ridge term for collinear features. */
function fitOls(X: number[][], y: number[], ridge = 1e-6): number[] | null {
const p = X[0]?.length ?? 0;
if (p === 0 || X.length < p) return null;
const A: number[][] = Array.from({ length: p }, () => Array(p).fill(0));
const b: number[] = Array(p).fill(0);
for (let i = 0; i < X.length; i++) {
for (let a = 0; a < p; a++) {
b[a] += X[i][a] * y[i];
for (let c = 0; c <= a; c++) A[a][c] += X[i][a] * X[i][c];
}
}
for (let a = 0; a < p; a++) {
for (let c = a + 1; c < p; c++) A[a][c] = A[c][a];
A[a][a] += ridge;
}
return solveLinear(A, b);
}
/** Gaussian elimination with partial pivoting. Mutates `A`/`b`. */
function solveLinear(A: number[][], b: number[]): number[] | null {
const n = A.length;
for (let col = 0; col < n; col++) {
let pivot = col;
for (let r = col + 1; r < n; r++) {
if (Math.abs(A[r][col]) > Math.abs(A[pivot][col])) pivot = r;
}
if (Math.abs(A[pivot][col]) < 1e-9) return null;
[A[col], A[pivot]] = [A[pivot], A[col]];
[b[col], b[pivot]] = [b[pivot], b[col]];
const d = A[col][col];
for (let c = col; c < n; c++) A[col][c] /= d;
b[col] /= d;
for (let r = 0; r < n; r++) {
if (r === col) continue;
const f = A[r][col];
if (f === 0) continue;
for (let c = col; c < n; c++) A[r][c] -= f * A[col][c];
b[r] -= f * b[col];
}
}
return b;
}
function dot(beta: number[], x: number[]): number {
let sum = 0;
for (let i = 0; i < beta.length; i++) sum += beta[i] * x[i];
return sum;
}
/** Weeks in a year, for the annual harmonic terms. */
const SEASONAL_PERIOD = 365.25 / 7;
/** One annual harmonic (a single sine/cosine pair) keeps the fit stable on <2y. */
function seasonalFeatures(t: number): number[] {
const w = (2 * Math.PI * t) / SEASONAL_PERIOD;
return [1, t, Math.sin(w), Math.cos(w)];
}
/** How quickly extrapolated trend is damped (per projected week). */
const TREND_DAMPING = 0.85;
function percentile(sorted: number[], p: number): number {
return sorted[Math.min(sorted.length - 1, Math.round(p * (sorted.length - 1)))];
}
function summarize(paths: number[][], horizonWeeks: number) {
const lows: number[] = [];
const highs: number[] = [];
for (let k = 0; k < horizonWeeks; k++) {
const column = paths.map((path) => path[k]).sort((a, b) => a - b);
lows.push(percentile(column, 0.1));
highs.push(percentile(column, 0.9));
}
return { lows, highs };
}
/**
* Project weekly active users forward using three methods, all fitted over the
* full completed-week history:
*
* - flow: `WAU(t+1) = retention × WAU(t) + inflow`, with a bootstrap band
* - regression: distributed lag on lagged WAU and contemporaneous inflow
* - seasonal: harmonic (annual Fourier) regression with a damped trend
*
* The flow model is the structural guard against the classic linear overshoot:
* retention < 1 makes it converge to `inflow / (1 - retention)`.
*/
export function projectWau(
completions: CompletionRow[],
asOfDate: string,
horizonWeeks = 8
): WauProjection | null {
const flows = weeklyFlows(completions, asOfDate);
const n = flows.length;
if (n < 3) return null;
const last = flows[n - 1];
const SIMS = 1000;
let seed = 0x2f6e2b1;
const rand = () => {
// xorshift32: deterministic and good enough for a UI band.
seed ^= seed << 13;
seed ^= seed >>> 17;
seed ^= seed << 5;
return (seed >>> 0) / 0xffffffff;
};
// Recency-weighted flow parameters over the whole history (13-week half-life).
const HALF_LIFE = 13;
let wsum = 0;
let retSum = 0;
let inSum = 0;
let newSum = 0;
let resSum = 0;
const samples: { retention: number; inflow: number }[] = [];
for (let i = 1; i < n; i++) {
const w = Math.pow(0.5, (n - 1 - i) / HALF_LIFE);
const prev = flows[i - 1];
const ret = prev.wau > 0 ? flows[i].retained / prev.wau : 0;
wsum += w;
retSum += w * ret;
inSum += w * flows[i].inflow;
newSum += w * flows[i].newPlayers;
resSum += w * flows[i].resurrected;
samples.push({ retention: ret, inflow: flows[i].inflow });
}
const retention = wsum > 0 ? retSum / wsum : 0;
const inflow = wsum > 0 ? inSum / wsum : 0;
const meanNew = wsum > 0 ? newSum / wsum : 0;
const meanRes = wsum > 0 ? resSum / wsum : 0;
// Future ISO-week dates, with the year only when it actually changes.
const futureDates = Array.from({ length: horizonWeeks }, (_, k) =>
addDays(last.end, 7 * (k + 1))
);
const lastYear = isoWeek(last.end).year;
const needsYear = futureDates.some((d) => isoWeek(d).year !== lastYear);
const labelFor = (date: string) => {
const { week, year } = isoWeek(date);
return `W${week}${needsYear ? ` '${String(year).slice(2)}` : ''}`;
};
const toPoints = (values: number[], lows: number[], highs: number[]): ProjectionPoint[] =>
values.map((v, k) => {
const value = Math.round(v);
return {
date: futureDates[k],
label: labelFor(futureDates[k]),
value,
low: Math.round(Math.min(lows[k], value)),
high: Math.round(Math.max(highs[k], value))
};
});
const wauY = flows.map((f) => f.wau);
// ── 1. Flow model + bootstrap band ──────────────────────────────────────
const flowPoint: number[] = [];
let wau = last.wau;
for (let k = 0; k < horizonWeeks; k++) {
wau = retention * wau + inflow;
flowPoint.push(Math.max(0, wau));
}
const flowPaths = Array.from({ length: SIMS }, () => {
let value = last.wau;
const path: number[] = [];
for (let k = 0; k < horizonWeeks; k++) {
const picked = samples[Math.floor(rand() * samples.length)];
value = picked.retention * value + picked.inflow;
path.push(Math.max(0, value));
}
return path;
});
// ── 2. Distributed-lag regression ───────────────────────────────────────
// Uses contemporaneous new/resurrected inflow, not lagged: WAU(t) is itself
// retained(t) + new(t) + resurrected(t). With lagged inflow (an earlier
// version) the autoregressive coefficient was the only real driver, so the
// projection decayed geometrically toward zero.
const regX: number[][] = [];
const regY: number[] = [];
for (let i = 1; i < n; i++) {
regX.push([1, flows[i - 1].wau, flows[i].newPlayers, flows[i].resurrected]);
regY.push(flows[i].wau);
}
const regBeta = fitOls(regX, regY);
const regResiduals = regBeta ? regY.map((y, i) => y - dot(regBeta, regX[i])) : [];
const runRegression = (sampled: boolean): number[] => {
const path: number[] = [];
let prev = last.wau;
for (let k = 0; k < horizonWeeks; k++) {
let pred = regBeta
? dot(regBeta, [1, prev, meanNew, meanRes])
: retention * prev + meanNew + meanRes;
if (sampled && regResiduals.length) {
pred += regResiduals[Math.floor(rand() * regResiduals.length)];
}
pred = Math.max(0, pred);
path.push(pred);
prev = pred;
}
return path;
};
const regressionPoint = runRegression(false);
const regressionPaths = Array.from({ length: SIMS }, () => runRegression(true));
// ── 3. Seasonality-aware harmonic regression ────────────────────────────
// A partial year cannot separate growth from seasonality, so only one annual
// harmonic is used and the linear trend is damped during projection. Without
// that, trend and annual cycle double-count the same rise and the forecast
// marches upward.
const seasonX = flows.map((_, i) => seasonalFeatures(i));
const seasonBeta = fitOls(seasonX, wauY);
const seasonResiduals = seasonBeta ? wauY.map((y, i) => y - dot(seasonBeta, seasonX[i])) : [0];
const seasonalBase: number[] = [];
let dampedTime = n - 1;
for (let k = 0; k < horizonWeeks; k++) {
dampedTime += Math.pow(TREND_DAMPING, k + 1);
if (seasonBeta) {
const w = (2 * Math.PI * (n + k)) / SEASONAL_PERIOD;
seasonalBase.push(
seasonBeta[0] +
seasonBeta[1] * dampedTime +
seasonBeta[2] * Math.sin(w) +
seasonBeta[3] * Math.cos(w)
);
} else {
seasonalBase.push(meanOf(wauY));
}
}
const seasonalPoint = seasonalBase.map((v) => Math.max(0, v));
const seasonalPaths = Array.from({ length: SIMS }, () => {
const path: number[] = [];
for (let k = 0; k < horizonWeeks; k++) {
const resid = seasonResiduals[Math.floor(rand() * seasonResiduals.length)];
path.push(Math.max(0, seasonalBase[k] + resid));
}
return path;
});
const flowBand = summarize(flowPaths, horizonWeeks);
const regressionBand = summarize(regressionPaths, horizonWeeks);
const seasonalBand = summarize(seasonalPaths, horizonWeeks);
return {
horizonWeeks,
retention: Math.round(retention * 1000) / 1000,
inflow: Math.round(inflow),
sampleWeeks: n,
flow: toPoints(flowPoint, flowBand.lows, flowBand.highs),
regression: toPoints(regressionPoint, regressionBand.lows, regressionBand.highs),
seasonal: toPoints(seasonalPoint, seasonalBand.lows, seasonalBand.highs)
};
}
// ── retention ────────────────────────────────────────────────────────────────
/**
+97 -34
View File
@@ -27,6 +27,7 @@ import { deriveSegments, countSegments } from './segments';
import { evaluateSignals } from './signals';
import {
activePlayers,
activityBuckets,
averageGuesses,
bookCoverage,
cohortTriangle,
@@ -39,13 +40,14 @@ import {
netGrowth,
newVsReturning,
overallReturnRate,
projectWau,
repeatIntervals,
retentionByGuessBucket,
streakSurvival,
timeToSecondPlayHistogram,
weekdayHourHeatmap
} from './metrics';
import { addDays, dateRange, dayDiff, startOfWeek, todayUtc } from './dates';
import { addDays, dateRange, dayDiff, endOfVerseWindow, isDayClosed, startOfWeek, todayUtc } from './dates';
export type BuildKind = 'scheduled' | 'lazy' | 'reconstructed';
@@ -53,16 +55,12 @@ export type BuildKind = 'scheduled' | 'lazy' | 'reconstructed';
* Bump whenever the snapshot payload shape changes. Older snapshots are
* rebuilt lazily so the UI never reads a stale shape.
*/
export const PAYLOAD_SCHEMA_VERSION = 2;
export const PAYLOAD_SCHEMA_VERSION = 9;
const RETENTION_WINDOW_DAYS = 180;
const SERIES_DAYS = 90;
const RECENT_KPI_DAYS = 30;
function endOfDay(dateStr: string): Date {
return new Date(dateStr + 'T23:59:59.999Z');
}
function round1(n: number): number {
return Math.round(n * 10) / 10;
}
@@ -76,12 +74,28 @@ export interface LoadedData {
signupDates: string[];
}
/** Load all rows needed to build a snapshot as of `cutoff` (inclusive). */
export interface LoadStatsOptions {
/**
* Point-in-time cutoff for time-stamped rows. When omitted, the full data
* window is loaded and rows are grouped purely by their verse `date`.
*/
completedBefore?: Date;
}
/**
* Load all rows needed to build a snapshot as of `cutoff` (inclusive).
*
* Rows are scoped by the verse `date` column, not by a UTC 24-hour window:
* a verse date is a player's *local* day, so a single date legitimately spans
* D-1 10:00Z through D+1 12:00Z. `completedBefore` is only for point-in-time
* builds of a day whose window is still open.
*/
export async function loadStatsData(
cutoff: string,
database: StatsDb = defaultDb
database: StatsDb = defaultDb,
opts: LoadStatsOptions = {}
): Promise<LoadedData> {
const cutoffDate = endOfDay(cutoff);
const { completedBefore } = opts;
const completionRows = await database
.select({
@@ -92,13 +106,23 @@ export async function loadStatsData(
completedAt: dailyCompletions.completedAt
})
.from(dailyCompletions)
.where(and(lte(dailyCompletions.date, cutoff), lte(dailyCompletions.completedAt, cutoffDate)));
.where(
completedBefore
? and(
lte(dailyCompletions.date, cutoff),
lte(dailyCompletions.completedAt, completedBefore)
)
: lte(dailyCompletions.date, cutoff)
);
const verseRows = await database
.select({ date: dailyVerses.date, bookId: dailyVerses.bookId })
.from(dailyVerses)
.where(lte(dailyVerses.date, cutoff));
// Events (and signups) are time-stamped rather than date-scoped. For a final
// build the window end is exact; an open-day build pins to `completedBefore`.
const rowBefore = completedBefore ?? endOfVerseWindow(cutoff);
const eventRows = await database
.select({
name: analyticsEvents.name,
@@ -106,18 +130,19 @@ export async function loadStatsData(
createdAt: analyticsEvents.createdAt
})
.from(analyticsEvents)
.where(lte(analyticsEvents.createdAt, cutoffDate));
.where(lte(analyticsEvents.createdAt, rowBefore));
const startedByDate = new Map<string, number>();
let shareEvents = 0;
let firstEventDate: string | null = null;
for (const e of eventRows) {
const eventDate = e.date ?? new Date(e.createdAt).toISOString().slice(0, 10);
if (eventDate > cutoff) continue;
if (!firstEventDate || eventDate < firstEventDate) firstEventDate = eventDate;
if (e.name === 'verse_served' && e.date && e.date <= cutoff) {
startedByDate.set(e.date, (startedByDate.get(e.date) ?? 0) + 1);
}
if ((e.name === 'share' || e.name === 'share_copy') && eventDate <= cutoff) {
if (e.name === 'share' || e.name === 'share_copy') {
shareEvents++;
}
}
@@ -125,7 +150,7 @@ export async function loadStatsData(
const userRows = await database
.select({ id: user.id, createdAt: user.createdAt })
.from(user)
.where(lte(user.createdAt, cutoffDate));
.where(lte(user.createdAt, rowBefore));
return {
completions: completionRows as CompletionRow[],
@@ -248,7 +273,12 @@ function buildActivity(
churned: growth.churned,
netGrowth: growth.netGrowth,
corePrevious,
previous
previous,
stickiness: {
week: activityBuckets(pointCompletions, asOfDate, 'week'),
month: activityBuckets(pointCompletions, asOfDate, 'month')
},
projection: projectWau(pointCompletions, asOfDate)
};
}
@@ -521,11 +551,25 @@ export function computePayload(
const verseMix = buildVerseMix(pointCompletions, data.verses, asOfDate);
const habit = buildHabit(pointCompletions, players, asOfDate);
const yesterday = pointCompletions.filter((c) => c.date === asOfDate);
// All-time best single day (earliest date wins ties).
const completionsByDate = new Map<string, number>();
for (const c of pointCompletions) {
completionsByDate.set(c.date, (completionsByDate.get(c.date) ?? 0) + 1);
}
let bestDay: { date: string; count: number } | null = null;
for (const [date, count] of completionsByDate) {
if (!bestDay || count > bestDay.count || (count === bestDay.count && date < bestDay.date)) {
bestDay = { date, count };
}
}
const totals = {
completions: pointCompletions.length,
players: firstPlayDates(pointCompletions).size,
yesterdayCompletions: yesterday.length,
yesterdayAvgGuesses: averageGuesses(yesterday)
yesterdayAvgGuesses: averageGuesses(yesterday),
bestDay
};
const payload: SnapshotPayload = {
@@ -589,8 +633,12 @@ export interface BuildResult {
}
/**
* Build and persist a snapshot. `replace=false` never overwrites an existing
* (immutable) snapshot; the caller re-reads instead.
* Build a snapshot. `replace=false` never overwrites an existing (immutable)
* snapshot; the caller re-reads instead.
*
* A snapshot is only persisted once its global data window has closed: an
* in-progress day is computed as a point-in-time view and never written, so
* partial early-morning data can't become the permanent record.
*/
export async function buildSnapshot(
asOfDate: string,
@@ -600,7 +648,10 @@ export async function buildSnapshot(
const settings = await getSettings(database);
const version = hashSettings(settings);
const cutoff = opts.matureThrough ?? asOfDate;
const data = await loadStatsData(cutoff, database);
const closed = isDayClosed(asOfDate);
const data = await loadStatsData(cutoff, database, {
completedBefore: closed ? undefined : new Date()
});
const payload = computePayload(asOfDate, settings, data, {
matureThrough: opts.matureThrough
});
@@ -620,7 +671,7 @@ export async function buildSnapshot(
config: JSON.stringify(settings)
};
if (opts.persist !== false) {
if (opts.persist !== false && closed) {
if (opts.replace) {
await database
.insert(statsSnapshot)
@@ -647,28 +698,40 @@ export async function buildSnapshot(
}
/**
* Return the snapshot for `date`, building it lazily if missing. Race-safe via
* `INSERT OR IGNORE` followed by a re-read.
* Return the snapshot for `date`, building it lazily if missing.
*
* Closed days are persisted and reused (race-safe via `INSERT OR IGNORE`),
* while an in-progress day is always computed on demand and never written.
*/
export async function ensureSnapshot(
date: string,
opts: { matureThrough?: string } = {},
database: StatsDb = defaultDb
): Promise<StoredSnapshot> {
const existing = await getSnapshot(date, database);
if (existing && existing.payload.meta?.schemaVersion === PAYLOAD_SCHEMA_VERSION) {
return existing;
if (isDayClosed(date)) {
const existing = await getSnapshot(date, database);
if (existing && existing.payload.meta?.schemaVersion === PAYLOAD_SCHEMA_VERSION) {
return existing;
}
// Missing or built with an older payload shape: (re)build the final view.
await buildSnapshot(
date,
{ kind: 'lazy', matureThrough: opts.matureThrough, replace: !!existing },
database
);
const stored = await getSnapshot(date, database);
if (!stored) throw new Error(`Failed to build snapshot for ${date}`);
return stored;
}
// Missing or built with an older payload shape: (re)build deterministically
// for this point in time. The point-in-time cutoff keeps it faithful.
await buildSnapshot(
date,
{ kind: 'lazy', matureThrough: opts.matureThrough, replace: !!existing },
database
);
const stored = await getSnapshot(date, database);
if (!stored) throw new Error(`Failed to build snapshot for ${date}`);
return stored;
// Open day: compute a live point-in-time view without persisting it.
return (
await buildSnapshot(
date,
{ kind: 'lazy', matureThrough: opts.matureThrough, persist: false },
database
)
).snapshot;
}
/** True when a stored snapshot was built with different thresholds or shape. */
+76
View File
@@ -136,6 +136,60 @@ export interface BookStat {
d1Return: number | null;
}
/** Calendar period used to bucket the WAU/MAU comparison. */
export type BucketGranularity = 'week' | 'month';
export interface ActivityBucket {
/** Last day of the calendar period, inclusive. */
date: string;
/** Axis label: "W36 '26" for ISO weeks, or a month name for months. */
label: string;
/**
* Distinct players in the bucket's weekly window: the whole ISO week for
* week buckets, or the 7 days ending on `date` for month buckets.
*/
wau: number;
/**
* Distinct players in the bucket's monthly window: the 30 days ending on
* `date` for week buckets, or the whole calendar month for month buckets.
*/
mau: number;
}
/** One future week in a WAU projection. */
export interface ProjectionPoint {
date: string;
label: string;
/** Point estimate (or p50 for the range view). */
value: number;
/** p10 lower bound (equals `value` for the point models). */
low: number;
/** p90 upper bound (equals `value` for the point models). */
high: number;
}
/**
* Forward view of weekly active users. Three methods are produced so the UI can
* toggle between them: a flow/retention model (with a bootstrap p10–p90 band),
* a distributed-lag regression, and a seasonality-aware harmonic regression.
*/
export interface WauProjection {
/** Number of future weeks projected. */
horizonWeeks: number;
/** Recency-weighted week-over-week retention used by the flow model (0..1). */
retention: number;
/** Recency-weighted weekly inflow: players active who were not active before. */
inflow: number;
/** Completed weeks of history available to the models. */
sampleWeeks: number;
/** Flow model with a bootstrap p10–p90 band. */
flow: ProjectionPoint[];
/** Distributed-lag regression on lagged WAU and contemporaneous new/resurrected. */
regression: ProjectionPoint[];
/** Harmonic (annual Fourier) regression with a damped linear trend. */
seasonal: ProjectionPoint[];
}
export interface Activity {
dau: number;
wau: number;
@@ -155,6 +209,13 @@ export interface Activity {
mau: number;
core: number;
} | null;
/**
* WAU vs MAU by completed calendar period, most recent last. Optional so
* payloads stored before this field existed still typecheck.
*/
stickiness?: Record<BucketGranularity, ActivityBucket[]>;
/** Forward WAU projection (trend / flow / range). Optional for old payloads. */
projection?: WauProjection | null;
}
export interface Kpis {
@@ -204,6 +265,19 @@ export interface HabitData {
weekdayHour: number[][];
}
/** Safe, all-time aggregates rendered by the public community stats widget. */
export interface CommunityStatsData {
asOfDate: string;
completions: number;
players: number;
weeklyPlayers: number;
monthlyPlayers: number;
yesterdayCompletions: number;
yesterdayAvgGuesses: number | null;
bestDayAllTime: { date: string; count: number } | null;
calendar: { date: string; count: number }[];
}
export interface SnapshotPayload {
asOfDate: string;
activity: Activity;
@@ -222,6 +296,8 @@ export interface SnapshotPayload {
players: number;
yesterdayCompletions: number;
yesterdayAvgGuesses: number | null;
/** Highest single-day finish count across all time (earliest date wins ties). */
bestDay: { date: string; count: number } | null;
};
signals: Signal[];
/** Metadata about data availability, so the UI can show honest em-dashes. */
+96
View File
@@ -0,0 +1,96 @@
/**
* Quantile-bucketed color scale for calendar/heatmap charts.
*
* A naive `count / max` alpha collapses most cells into the same pale shade when
* a single day is an outlier. Bucketing by quantiles of the non-zero values
* (like GitHub's contribution graph) keeps typical variation visible and never
* changes when a new all-time-high day appears.
*/
export interface CalendarBucket {
/** Lowest count in this bucket. */
min: number;
/** Highest count in this bucket. */
max: number;
/** Alpha (0..1) applied to the base color. */
alpha: number;
/** How many non-zero days fall in this bucket. */
days: number;
}
export interface CalendarScale {
min: number;
max: number;
/** Non-empty intensity buckets, low → high. Zero is rendered separately. */
buckets: CalendarBucket[];
/** Alpha for a raw count (0 for a zero/empty day). */
alphaFor: (count: number) => number;
}
/** Linear-interpolated quantile over an already-sorted ascending array. */
export function quantile(sorted: number[], q: number): number {
if (sorted.length === 0) return 0;
if (sorted.length === 1) return sorted[0];
const pos = (sorted.length - 1) * q;
const base = Math.floor(pos);
const rest = pos - base;
const next = sorted[base + 1];
return next === undefined ? sorted[base] : sorted[base] + rest * (next - sorted[base]);
}
/**
* Build a quantile scale from a list of daily counts.
*
* The non-zero counts are split into `steps` intensity buckets at evenly spaced
* quantiles; duplicate-heavy data is tolerated because empty buckets are dropped.
*/
export function buildCalendarScale(counts: number[], steps = 4): CalendarScale {
const nonzero = counts.filter((c) => c > 0).sort((a, b) => a - b);
const min = counts.length ? Math.min(...counts) : 0;
const max = counts.length ? Math.max(...counts) : 0;
if (nonzero.length === 0) {
return { min, max, buckets: [], alphaFor: () => 0 };
}
const alphas =
steps <= 1
? [0.9]
: Array.from({ length: steps }, (_, i) => 0.28 + (0.72 * i) / (steps - 1));
const thresholds: number[] = [];
for (let i = 1; i < steps; i++) thresholds.push(quantile(nonzero, i / steps));
const bucketIndex = (count: number): number => {
if (count <= 0) return -1;
let index = 0;
for (const threshold of thresholds) {
if (count > threshold) index++;
}
return index;
};
const buckets: CalendarBucket[] = alphas.map((alpha) => ({
alpha,
min: Number.POSITIVE_INFINITY,
max: Number.NEGATIVE_INFINITY,
days: 0
}));
for (const count of nonzero) {
const bucket = buckets[bucketIndex(count)];
bucket.min = Math.min(bucket.min, count);
bucket.max = Math.max(bucket.max, count);
bucket.days++;
}
return {
min,
max,
buckets: buckets.filter((bucket) => bucket.days > 0),
alphaFor: (count: number) => {
const index = bucketIndex(count);
return index < 0 ? 0 : alphas[index];
}
};
}
+1 -1
View File
@@ -1,7 +1,7 @@
/** Small formatting helpers shared by the stats dashboard components. */
export function numOrDash(n: number | null | undefined): string {
return n === null || n === undefined ? '—' : String(n);
return n === null || n === undefined ? '—' : n.toLocaleString('en-US');
}
export function pctOrDash(n: number | null | undefined): string {
+27 -2
View File
@@ -2,19 +2,39 @@
import type { PageData } from './$types';
import ConsoleChrome from '$lib/components/stats/ConsoleChrome.svelte';
import Pulse from '$lib/components/stats/Pulse.svelte';
import Projection from '$lib/components/stats/Projection.svelte';
import GrowthLoop from '$lib/components/stats/GrowthLoop.svelte';
import RetentionLab from '$lib/components/stats/RetentionLab.svelte';
import VerseMixLab from '$lib/components/stats/VerseMixLab.svelte';
// VerseMixLab hidden for now.
// import VerseMixLab from '$lib/components/stats/VerseMixLab.svelte';
import HabitLab from '$lib/components/stats/HabitLab.svelte';
import SignalsFeed from '$lib/components/stats/SignalsFeed.svelte';
import FeatureImpact from '$lib/components/stats/FeatureImpact.svelte';
import Glossary from '$lib/components/stats/Glossary.svelte';
import CommunityStats from '$lib/components/stats/CommunityStats.svelte';
import Container from '$lib/components/Container.svelte';
import { numOrDash } from '$lib/utils/stats-format';
import type { CommunityStatsData } from '$lib/types/stats';
let { data }: { data: PageData } = $props();
const segments = $derived(data.segmentCounts);
// The public community view lives at the top of the console too. The verse
// card is intentionally omitted here.
const community = $derived.by(
(): CommunityStatsData => ({
asOfDate: data.payload.asOfDate,
completions: data.payload.totals.completions,
players: data.payload.totals.players,
weeklyPlayers: data.payload.activity.wau,
monthlyPlayers: data.payload.activity.mau,
yesterdayCompletions: data.payload.totals.yesterdayCompletions,
yesterdayAvgGuesses: data.payload.totals.yesterdayAvgGuesses,
bestDayAllTime: data.payload.totals.bestDay,
calendar: data.payload.habit.calendar
})
);
</script>
<svelte:head>
@@ -24,6 +44,8 @@
<div class="min-h-screen bg-gray-950 p-4 md:p-6">
<div class="mx-auto flex max-w-6xl flex-col gap-8 pb-16">
<CommunityStats data={community} tone="dark" />
<ConsoleChrome
date={data.date}
today={data.today}
@@ -41,6 +63,8 @@
topSignal={data.topSignal}
/>
<Projection payload={data.payload} />
<section class="flex flex-col gap-3">
<h2 class="text-lg font-bold text-gray-100">Segment mix (as of this date)</h2>
<div class="grid grid-cols-2 gap-3 sm:grid-cols-3 lg:grid-cols-6">
@@ -66,7 +90,8 @@
<GrowthLoop payload={data.payload} />
<RetentionLab payload={data.payload} settings={data.settings} />
<VerseMixLab payload={data.payload} />
<!-- VerseMixLab hidden for now. -->
<!-- <VerseMixLab payload={data.payload} /> -->
<HabitLab payload={data.payload} />
<SignalsFeed signals={data.payload.signals} />
<FeatureImpact />
+6 -7
View File
@@ -1,13 +1,13 @@
import { json } from '@sveltejs/kit';
import type { RequestHandler } from './$types';
import { addDays } from '$lib/server/stats/dates';
import { latestClosedDate } from '$lib/server/stats/dates';
import { buildSnapshot } from '$lib/server/stats/snapshot';
import { sendDigest } from '$lib/server/stats/digest';
/**
* Daily snapshot build trigger. Protected by `Authorization: Bearer $CRON_SECRET`
* (same pattern as `/api/send-daily-verse`). Builds yesterday in `America/New_York`
* and then sends the digest.
* (same pattern as `/api/send-daily-verse`). Builds the most recent date whose
* global data window has closed, then sends the digest.
*/
export const POST: RequestHandler = async ({ request }) => {
const cronSecret = Bun.env.CRON_SECRET;
@@ -16,10 +16,9 @@ export const POST: RequestHandler = async ({ request }) => {
return json({ error: 'Unauthorized' }, { status: 401 });
}
const todayEst = new Date().toLocaleDateString('en-CA', { timeZone: 'America/New_York' });
const yesterday = addDays(todayEst, -1);
const date = latestClosedDate();
const { snapshot } = await buildSnapshot(yesterday, {
const { snapshot } = await buildSnapshot(date, {
kind: 'scheduled',
replace: true
});
@@ -27,5 +26,5 @@ export const POST: RequestHandler = async ({ request }) => {
// Digest failure must never fail the build.
await sendDigest(snapshot.payload);
return json({ ok: true, date: yesterday, buildKind: 'scheduled' });
return json({ ok: true, date, buildKind: 'scheduled' });
};
+28 -8
View File
@@ -1,25 +1,45 @@
import type { PageServerLoad } from './$types';
import { ensureSnapshot } from '$lib/server/stats/snapshot';
import { addDays } from '$lib/server/stats/dates';
import { latestClosedDate } from '$lib/server/stats/dates';
import { db } from '$lib/server/db';
import { dailyVerses } from '$lib/server/db/schema';
import { eq } from 'drizzle-orm';
/**
* Slim public community widget. Reads the latest daily snapshot and exposes
* only safe aggregates (no retention, churn, signup or revenue data).
* Slim public community widget. Reads the latest completed daily snapshot and
* exposes only safe aggregates (no retention, churn, signup or revenue data).
*/
export const load: PageServerLoad = async () => {
const todayEst = new Date().toLocaleDateString('en-CA', { timeZone: 'America/New_York' });
const yesterday = addDays(todayEst, -1);
const snapshot = await ensureSnapshot(yesterday);
const snapshot = await ensureSnapshot(latestClosedDate());
const payload = snapshot.payload;
// The verse itself is public content (it was served to everyone), so showing
// it here is safe. A missing row or lookup failure just hides the card.
let verse: { reference: string; verseText: string } | null = null;
try {
const [row] = await db
.select({
reference: dailyVerses.reference,
verseText: dailyVerses.verseText
})
.from(dailyVerses)
.where(eq(dailyVerses.date, payload.asOfDate))
.limit(1);
verse = row ?? null;
} catch (err) {
console.error('Failed to load community verse', err);
}
return {
asOfDate: payload.asOfDate,
verse: verse ?? null,
completions: payload.totals.completions,
players: payload.totals.players,
weeklyPlayers: payload.activity.wau,
monthlyPlayers: payload.activity.mau,
yesterdayCompletions: payload.totals.yesterdayCompletions,
yesterdayAvgGuesses: payload.totals.yesterdayAvgGuesses,
calendar: payload.habit.calendar.slice(-30)
bestDayAllTime: payload.totals.bestDay,
calendar: payload.habit.calendar
};
};
};
+20 -39
View File
@@ -1,56 +1,37 @@
<script lang="ts">
import type { PageData } from './$types';
import Container from '$lib/components/Container.svelte';
import CalendarHeatmap from '$lib/components/charts/CalendarHeatmap.svelte';
import { formatDateLabel, numOrDash } from '$lib/utils/stats-format';
import CommunityStats from '$lib/components/stats/CommunityStats.svelte';
import { formatDateLabel } from '$lib/utils/stats-format';
let { data }: { data: PageData } = $props();
</script>
<svelte:head>
<title>Community Stats | Bibdle</title>
<meta name="description" content="Bibdle community statistics, as of yesterday." />
<title>Community Stats | BIBDLE</title>
<meta name="description" content="BIBDLE community statistics, as of the latest completed day." />
</svelte:head>
<div class="mx-auto max-w-3xl px-4 py-8">
<div class="mb-6 text-center">
<h1 class="text-3xl font-bold text-gray-900 dark:text-gray-100">Community Stats</h1>
<p class="mt-1 text-sm text-gray-600 dark:text-gray-400">As of {formatDateLabel(data.asOfDate)}</p>
</div>
<div class="grid grid-cols-2 gap-3 md:grid-cols-3">
<Container class="p-4 w-full items-start text-left">
<p class="text-[11px] font-semibold uppercase tracking-wider text-gray-600 dark:text-gray-400">All-time finishes</p>
<p class="text-2xl font-bold text-gray-900 dark:text-gray-100">{numOrDash(data.completions)}</p>
</Container>
<Container class="p-4 w-full items-start text-left">
<p class="text-[11px] font-semibold uppercase tracking-wider text-gray-600 dark:text-gray-400">All-time players</p>
<p class="text-2xl font-bold text-gray-900 dark:text-gray-100">{numOrDash(data.players)}</p>
</Container>
<Container class="p-4 w-full items-start text-left">
<p class="text-[11px] font-semibold uppercase tracking-wider text-gray-600 dark:text-gray-400">Players this week</p>
<p class="text-2xl font-bold text-gray-900 dark:text-gray-100">{numOrDash(data.weeklyPlayers)}</p>
</Container>
<Container class="p-4 w-full items-start text-left">
<p class="text-[11px] font-semibold uppercase tracking-wider text-gray-600 dark:text-gray-400">Players this month</p>
<p class="text-2xl font-bold text-gray-900 dark:text-gray-100">{numOrDash(data.monthlyPlayers)}</p>
</Container>
<Container class="p-4 w-full items-start text-left">
<p class="text-[11px] font-semibold uppercase tracking-wider text-gray-600 dark:text-gray-400">Yesterday's finishes</p>
<p class="text-2xl font-bold text-gray-900 dark:text-gray-100">{numOrDash(data.yesterdayCompletions)}</p>
</Container>
<Container class="p-4 w-full items-start text-left">
<p class="text-[11px] font-semibold uppercase tracking-wider text-gray-600 dark:text-gray-400">Avg guesses yesterday</p>
<p class="text-2xl font-bold text-gray-900 dark:text-gray-100">{numOrDash(data.yesterdayAvgGuesses)}</p>
</Container>
</div>
<CommunityStats data={data} />
<Container class="mt-3 p-4 w-full items-stretch text-left gap-2">
<p class="font-semibold text-gray-900 dark:text-gray-100">Daily finishes (last 30 days)</p>
<CalendarHeatmap data={data.calendar} />
<p class="font-semibold text-gray-900 dark:text-gray-100">Latest verse</p>
{#if data.verse}
<blockquote class="text-sm leading-relaxed text-gray-700 dark:text-gray-200">
“{data.verse.verseText}”
<cite class="mt-1 block text-xs not-italic text-gray-500 dark:text-gray-400">
— {data.verse.reference}
</cite>
</blockquote>
{:else}
<p class="text-sm text-gray-500 dark:text-gray-400">
Verse text unavailable for {formatDateLabel(data.asOfDate)}.
</p>
{/if}
</Container>
<p class="mt-4 text-center text-xs text-gray-500 dark:text-gray-500">
Updated once a day from completed games. <a class="underline hover:text-gray-300" href="/">Play today's verse →</a>
Updated once a day from completed games.<br /> <a class="underline hover:text-gray-300" href="/">Play today's verse →</a>
</p>
</div>
</div>
+2 -2
View File
@@ -220,10 +220,10 @@ Indexes: `(name, date)`, `(anonymous_id, date)`. **Unique** on `(name, anonymous
**`user.created_at`** — new timestamp column (see §7).
### 6.2 Snapshot build
- **Trigger:** daily cron hits a protected `POST /api/cron/build-stats` (bearer `CRON_SECRET`, same pattern as `/api/send-daily-verse`). It builds yesterday's snapshot and sends the digest.
- **Trigger:** daily cron hits a protected `POST /api/cron/build-stats` (bearer `CRON_SECRET`, same pattern as `/api/send-daily-verse`). It builds the most recent date whose global data window has closed and sends the digest.
- **Lazy fallback:** `/admin/stats` load builds a missing snapshot on demand (guarded against races, marked `lazy`).
- **Backfill:** an admin action/script can reconstruct a date range from raw data (marked `reconstructed`).
- **Computation cutoff:** only data with `date <= asOfDate` (and `completedAt` before end of that day) — guarantees point-in-time.
- **Computation cutoff:** data is grouped by the verse `date` (a player's local day), not a UTC 24-hour window. A date is final after `D+1 12:00Z` (UTC-12 has finished the day); until then a build is a non-persisted point-in-time view. Snapshots are only persisted once the window has closed.
- **Player state (for render-time config):** per player active in the trailing ~180 days, a compact tuple `[anonId, firstDate, lastDate, totalPlays, plays7, plays30, currentStreak]`. Thresholds applied at render time, so changing them re-segments **all history instantly** while the state stays frozen as of that day.
- **Retention/pruning:** keep all snapshots (small); optionally prune `player_state` older than a configurable horizon later.
+1 -1
View File
@@ -40,7 +40,7 @@ const payload: SnapshotPayload = {
},
habit: { streakSurvival: [], streakLengths: [], calendar: [], weekdayHour: [] },
growth: { lifecycleFunnel: [], signups: [] },
totals: { completions: 31775, players: 6589, yesterdayCompletions: 102, yesterdayAvgGuesses: 4.1 },
totals: { completions: 31775, players: 6589, yesterdayCompletions: 102, yesterdayAvgGuesses: 4.1, bestDay: null },
signals: [
{
id: 'd1_return',
+190 -1
View File
@@ -17,7 +17,10 @@ import {
retentionByGuessBucket,
weekdayHourHeatmap,
cohortTriangle,
dailyCompletionsSeries
dailyCompletionsSeries,
activityBuckets,
weeklyFlows,
projectWau
} from '$lib/server/stats/metrics';
import type { CompletionRow } from '$lib/types/stats';
@@ -191,3 +194,189 @@ describe('dailyCompletionsSeries', () => {
expect(series[2]).toEqual({ date: '2026-09-03', completions: 2, newPlayers: 0 });
});
});
describe('activityBuckets', () => {
// 2026-09-10 is a Thursday, so its ISO week (Mon 09-07 .. Sun 09-13) is
// still in progress: the most recent completed week ends 2026-09-06 (W36).
const asOf = '2026-09-10';
test('weeks are ISO Monday–Sunday and stop at the last completed week', () => {
const rows: CompletionRow[] = [
c('mon', '2026-08-31'), // first day of the week
c('sun', '2026-09-06') // last day of the week
];
const weeks = activityBuckets(rows, asOf, 'week');
expect(weeks).toHaveLength(1);
expect(weeks[0].date).toBe('2026-09-06');
expect(weeks[0].label).toBe('W36');
expect(weeks[0].wau).toBe(2);
});
test('spans every completed week back to the first week with data', () => {
const rows: CompletionRow[] = [
c('p1', '2026-08-20'),
c('p2', '2026-08-27'),
c('p3', '2026-09-03')
];
const weeks = activityBuckets(rows, asOf, 'week');
expect(weeks.map((b) => b.date)).toEqual(['2026-08-23', '2026-08-30', '2026-09-06']);
expect(weeks.map((b) => b.label)).toEqual(['W34', 'W35', 'W36']);
expect(weeks.map((b) => b.wau)).toEqual([1, 1, 1]);
// The trailing 30-day window reaches further back, so it grows per bucket.
expect(weeks.map((b) => b.mau)).toEqual([1, 2, 3]);
});
test('excludes the in-progress week so a partial period cannot drag the series down', () => {
const rows: CompletionRow[] = [c('current', '2026-09-08'), c('done', '2026-09-04')];
const weeks = activityBuckets(rows, asOf, 'week');
expect(weeks).toHaveLength(1);
expect(weeks[0].date).toBe('2026-09-06');
expect(weeks[0].wau).toBe(1);
});
test('monthly buckets are whole calendar months up to the last completed one', () => {
const rows: CompletionRow[] = [
c('jun', '2026-06-01'),
c('jul', '2026-07-15'),
c('aug', '2026-08-31')
];
const months = activityBuckets(rows, asOf, 'month');
expect(months.map((b) => b.date)).toEqual(['2026-06-30', '2026-07-31', '2026-08-31']);
expect(months.map((b) => b.label)).toEqual(['Jun', 'Jul', 'Aug']);
// A whole calendar month, not a trailing 30-day window: Jun 1 is included.
expect(months[0].mau).toBe(1);
});
test('a month ending on the as-of day counts as completed', () => {
const rows: CompletionRow[] = [c('a', '2026-09-01'), c('b', '2026-09-30')];
const months = activityBuckets(rows, '2026-09-30', 'month');
expect(months.map((b) => b.date)).toEqual(['2026-09-30']);
expect(months[0].mau).toBe(2);
});
test('adds the week-year only when the range spans week-years', () => {
const rows: CompletionRow[] = [c('a', '2025-12-01'), c('b', '2026-01-15')];
const weeks = activityBuckets(rows, '2026-02-10', 'week');
expect(weeks[0].label).toBe("W49 '25");
expect(weeks.at(-1)?.label).toBe("W6 '26");
});
test('adds the year to month labels when the range spans years', () => {
const rows: CompletionRow[] = [c('a', '2025-11-01'), c('b', '2026-01-15')];
const months = activityBuckets(rows, '2026-02-10', 'month');
expect(months.map((b) => b.label)).toEqual(['Nov 25', 'Dec 25', 'Jan 26']);
});
test('labels January with its year even inside a single-year range', () => {
const rows: CompletionRow[] = [c('a', '2026-01-01'), c('b', '2026-03-15')];
const months = activityBuckets(rows, '2026-04-15', 'month');
expect(months.map((b) => b.label)).toEqual(['Jan 26', 'Feb', 'Mar']);
});
test('returns nothing when every period with data is still in progress', () => {
const rows: CompletionRow[] = [c('a', '2026-09-08')];
expect(activityBuckets(rows, asOf, 'month')).toEqual([]);
});
});
describe('weeklyFlows / projectWau', () => {
// Three consecutive completed ISO weeks ending Sun 08-23, 08-30, 09-06.
// p2/p3 return, p4/p5 arrive: 2 retained and 1 inflow each week.
const rows: CompletionRow[] = [
c('p1', '2026-08-18'),
c('p2', '2026-08-18'),
c('p3', '2026-08-18'),
c('p2', '2026-08-25'),
c('p3', '2026-08-25'),
c('p4', '2026-08-25'),
c('p3', '2026-09-01'),
c('p4', '2026-09-01'),
c('p5', '2026-09-01')
];
const asOf = '2026-09-10';
test('weeklyFlows accounts for retained vs inflow players', () => {
const flows = weeklyFlows(rows, asOf);
expect(flows.map((f) => f.end)).toEqual(['2026-08-23', '2026-08-30', '2026-09-06']);
expect(flows.map((f) => f.wau)).toEqual([3, 3, 3]);
expect(flows.map((f) => f.retained)).toEqual([0, 2, 2]);
expect(flows.map((f) => f.inflow)).toEqual([3, 1, 1]);
expect(flows.map((f) => f.newPlayers)).toEqual([3, 1, 1]);
expect(flows.map((f) => f.resurrected)).toEqual([0, 0, 0]);
});
test('the flow model holds a flat base steady instead of overshooting', () => {
const projection = projectWau(rows, asOf, 4);
expect(projection).not.toBeNull();
expect(projection?.sampleWeeks).toBe(3);
expect(projection?.retention).toBeCloseTo(0.667, 3);
expect(projection?.inflow).toBe(1);
// retention 2/3 * 3 + 1 = 3 every week.
expect(projection?.flow.map((p) => p.value)).toEqual([3, 3, 3, 3]);
expect(projection?.flow).toHaveLength(4);
expect(projection?.flow[0].label).toBe('W37');
});
test('every model returns a bracketed band', () => {
const projection = projectWau(rows, asOf, 4);
for (const key of ['flow', 'regression', 'seasonal'] as const) {
expect(projection?.[key]).toHaveLength(4);
for (const point of projection?.[key] ?? []) {
expect(point.low).toBeLessThanOrEqual(point.value);
expect(point.value).toBeLessThanOrEqual(point.high);
}
}
});
test('fits regression and seasonal models once there is enough history', () => {
const start = new Date('2026-01-05T00:00:00Z'); // Monday
const rows: CompletionRow[] = [];
for (let w = 0; w < 30; w++) {
const weekStart = new Date(start);
weekStart.setUTCDate(weekStart.getUTCDate() + w * 7);
for (let p = 0; p < 3; p++) {
const d = new Date(weekStart);
d.setUTCDate(d.getUTCDate() + ((w + p) % 7));
rows.push(c(`core-${p}`, d.toISOString().slice(0, 10)));
}
for (let p = 0; p < 2; p++) {
const d = new Date(weekStart);
d.setUTCDate(d.getUTCDate() + ((w + p + 3) % 7));
rows.push(c(`new-${w}-${p}`, d.toISOString().slice(0, 10)));
}
}
const after = new Date(start);
after.setUTCDate(after.getUTCDate() + 30 * 7);
const projection = projectWau(rows, after.toISOString().slice(0, 10), 4);
expect(projection?.sampleWeeks).toBe(30);
expect(projection?.regression).toHaveLength(4);
expect(projection?.seasonal).toHaveLength(4);
expect(projection?.retention).toBeGreaterThan(0);
expect(projection?.retention).toBeLessThanOrEqual(1);
});
test('the corrected regression does not collapse toward zero', () => {
const start = new Date('2026-01-05T00:00:00Z'); // Monday
const rows: CompletionRow[] = [];
for (let w = 0; w < 30; w++) {
const weekStart = new Date(start);
weekStart.setUTCDate(weekStart.getUTCDate() + w * 7);
for (let p = 0; p < 4; p++) {
const d = new Date(weekStart);
d.setUTCDate(d.getUTCDate() + ((w + p) % 7));
rows.push(c(`core-${p}`, d.toISOString().slice(0, 10)));
}
}
const after = new Date(start);
after.setUTCDate(after.getUTCDate() + 30 * 7);
const projection = projectWau(rows, after.toISOString().slice(0, 10), 4);
const last = projection?.regression.at(-1)?.value ?? 0;
// A stable 4-player base should not decay away.
expect(last).toBeGreaterThanOrEqual(3);
});
test('needs at least three completed weeks', () => {
const few = [c('a', '2026-09-01'), c('b', '2026-09-02')];
expect(projectWau(few, asOf)).toBeNull();
});
});
+1 -1
View File
@@ -41,7 +41,7 @@ function makePayload(overrides: {
segmentMix: []
},
growth: { lifecycleFunnel: [], signups: [] },
totals: { completions: 0, players: 0, yesterdayCompletions: 0, yesterdayAvgGuesses: null },
totals: { completions: 0, players: 0, yesterdayCompletions: 0, yesterdayAvgGuesses: null, bestDay: null },
verseMix: {
coverage: [],
testamentMix: [],
+69
View File
@@ -11,9 +11,11 @@ import {
computePayload,
computePlayerState,
ensureSnapshot,
loadStatsData,
PAYLOAD_SCHEMA_VERSION,
type LoadedData
} from '$lib/server/stats/snapshot';
import { endOfVerseWindow, isDayClosed, latestClosedDate } from '$lib/server/stats/dates';
import { SETTINGS_DEFAULTS, type CompletionRow } from '$lib/types/stats';
import crypto from 'node:crypto';
@@ -72,6 +74,20 @@ describe('computePayload point-in-time cutoff', () => {
expect(payload.kpis.completionRate).not.toBeNull();
});
test('all-time best day picks the highest count and ignores future days', () => {
const payload = computePayload('2026-09-10', settings, {
...data,
completions: [
c('A', '2026-09-01'),
c('A', '2026-09-02'),
c('B', '2026-09-02'),
c('C', '2026-09-02'),
c('D', '2026-09-11') // future relative to asOfDate
]
});
expect(payload.totals.bestDay).toEqual({ date: '2026-09-02', count: 3 });
});
test('payload has all lab sections', () => {
const payload = computePayload('2026-09-10', settings, data);
expect(payload.retention.cohorts.length).toBeGreaterThan(0);
@@ -138,4 +154,57 @@ describe('snapshot persistence', () => {
const snap = await ensureSnapshot('2026-01-10', {}, testDb);
expect(snap.payload.meta?.schemaVersion).toBe(PAYLOAD_SCHEMA_VERSION);
});
test('groups completions by verse date, not a UTC 24h window', async () => {
// A player in a UTC-negative timezone finishing "Jan 10" late at night
// lands on Jan 11 04:00Z. The old UTC end-of-day cutoff dropped them.
await testDb.insert(dailyCompletions).values({
id: crypto.randomUUID(),
anonymousId: 'late',
date: '2026-01-10',
guessCount: 4,
completedAt: new Date('2026-01-11T04:00:00Z')
});
const { snapshot } = await buildSnapshot('2026-01-10', { kind: 'lazy' }, testDb);
expect(snapshot.payload.activity.dau).toBe(1);
});
test('isDayClosed gates persistence for in-progress days', async () => {
await seed();
const open = new Date().toISOString().slice(0, 10);
await buildSnapshot(open, { kind: 'lazy' }, testDb);
await buildSnapshot(open, { kind: 'lazy', replace: true }, testDb);
const live = await ensureSnapshot(open, {}, testDb);
expect(live.date).toBe(open);
// None of the open-day builds may leave a persisted row behind.
expect(await testDb.select().from(statsSnapshot)).toHaveLength(0);
});
});
describe('verse day window', () => {
test('closes at D+1 12:00Z (UTC-12 has finished the local day)', () => {
expect(endOfVerseWindow('2026-01-10').toISOString()).toBe('2026-01-11T12:00:00.000Z');
expect(isDayClosed('2026-01-10', new Date('2026-01-11T11:59:59Z'))).toBe(false);
expect(isDayClosed('2026-01-10', new Date('2026-01-11T12:00:00Z'))).toBe(true);
});
test('latestClosedDate reflects the window lag', () => {
expect(latestClosedDate(new Date('2026-01-11T11:00:00Z'))).toBe('2026-01-09');
expect(latestClosedDate(new Date('2026-01-11T13:00:00Z'))).toBe('2026-01-10');
});
test('completedBefore pins a point-in-time build', async () => {
await testDb.delete(analyticsEvents);
await testDb.delete(dailyCompletions);
await testDb.insert(dailyCompletions).values([
{ id: crypto.randomUUID(), anonymousId: 'early', date: '2026-01-10', guessCount: 1, completedAt: new Date('2026-01-10T05:00:00Z') },
{ id: crypto.randomUUID(), anonymousId: 'late', date: '2026-01-10', guessCount: 2, completedAt: new Date('2026-01-11T04:00:00Z') }
]);
const pinned = await loadStatsData('2026-01-10', testDb, {
completedBefore: new Date('2026-01-10T12:00:00Z')
});
expect(pinned.completions.map((c) => c.anonymousId)).toEqual(['early']);
const full = await loadStatsData('2026-01-10', testDb);
expect(full.completions).toHaveLength(2);
});
});
+15 -1
View File
@@ -1,5 +1,20 @@
# in progress
(admin/stats)
- [x] WAU/MAU graph
- [x] show full calendar weeks (ISO, Mon-Sun) / months, stopping at the last completed period
- [x] new players per day should show 3 months
- [x] full width graphs need to have mouse tracking correctly
- [x] re: "segment mix over time": we need a counter for recently returning players
- [x] (add clarification for past 30 days) maybe just remove "churned"
- [x] embed the community stats view at the top of the console (minus the verse)
- [x] WAU projection section below Pulse (Flow / Regression / Seasonal, all-history fit, p10–p90 bands)
- [x] hide the verse mix lab completely, for now
(community stats)
- [x] show yesterday's verse in the community stats
- [ ] show human readable numbers in the community stats
- [ ] fix the "last 30 days" github style view to be full-width
# todo
@@ -17,7 +32,6 @@
- How do you balance rewarding knowledge vs incentivising learning?
- instructions
- classic mode: identify what book the verse is from (e.g. Genesis, John, Revelations...) in as few guesses as possible.