Data Storage

Now Playing splits data across four backing stores, each chosen for the kind of access it handles best. Understanding what lives where can help you reason about performance, consistency, and what happens during upgrades.

The Four Stores

Store	Used For	Durability
PostgreSQL	Persistent records: users, tracks, settings	Durable (managed backups)
Redis	Real-time state, pub/sub, job queues, short-lived cache	Ephemeral
Cloudflare R2	Album artwork and exported files	Durable (object storage)
Backblaze B2	Cold database backups	Long-term durable

PostgreSQL

PostgreSQL 16 with the pgvector extension is the system of record for everything that must survive a restart. It holds:

• User accounts — managed by Better Auth, with sessions, OAuth links, and verification records.
• Tracks and play history — every enriched track, with timestamps and user attribution.
• User settings — DJ style presets, output configurations, detection weights.
• Overlay tokens — the friendly names and high-entropy tokens that define overlay URLs.
• Album art cache metadata — the content hashes, CDN URLs, and derived sizes that resolve which R2 object to serve.
• Embeddings — pgvector tables used for artwork similarity and text matching during enrichment.

PostgreSQL is hosted as a managed Cloud SQL instance on GCP. Both the primary and failover environments connect to it directly, which is why data is consistent regardless of which environment is serving traffic.

Redis

Redis is used for three distinct access patterns:

• Pub/Sub — services send messages to each other over named channels (for example, track:raw, controller:state, user:{userId}:track). Pub/Sub is fire-and-forget: if no subscriber is listening when a message is published, the message is dropped.
• Streams — durable, ordered append-only logs used for job queues. Messages persist until consumed. Router fan-out, track enrichment, and output delivery all use streams so that a briefly offline service resumes without losing work.
• Cache and state — short-lived key-value data such as in-progress artwork lookups, desktop controller state, and recent deduplication keys.

Redis is ephemeral by design — a Redis restart empties non-stream data. The stream data is managed on a per-service basis, with consumer groups tracking progress so restarts do not lose jobs.

Cloudflare R2

R2 is S3-compatible object storage hosted by Cloudflare. Now Playing uses it for two kinds of content:

• Album artwork — every unique artwork image, in multiple sizes (thumb, small, medium, large), stored as WebP with JPEG fallback.
• Exported files — on-demand CSV or text exports triggered from the dashboard.

Artwork is served through the Cloudflare CDN, so even the first request for a track already cached by another user is fast and globally distributed. Because R2 has no egress fees, serving artwork to thousands of overlays is effectively free.

Backblaze B2

B2 holds nightly database backups. PostgreSQL snapshots are compressed and uploaded for long-term retention. Backups are kept with a retention policy that balances cost against the ability to recover from a rare corruption event.

What Lives Where During a Track Update

A single track update touches all four stores:

1. Redis (stream) — the desktop-ingested track enters track:raw.
2. PostgreSQL — the track and its enrichment result are written to the tracks table and the user’s history.
3. R2 — the artwork bytes are uploaded (first time) or looked up (cache hit).
4. Redis (pub/sub) — the enriched track is published to user:{userId}:track and streamed out over SSE.

This split keeps the hot path fast (Redis, R2) while keeping the durable record (PostgreSQL) out of the critical-path timing.

What Survives Each Event

Event	PostgreSQL	Redis	R2
Container restart	Yes	Partial (streams survive)	Yes
Host reboot	Yes	Partial (streams survive)	Yes
Failover activation	Yes (same DB)	Partial	Yes
Database upgrade	Yes (with migration)	Yes	Yes
R2 object expiration	Yes	Yes	No (artwork re-uploaded on next play)

The design targets high durability for user data (PostgreSQL) and high availability for hot data (Redis streams, R2), with graceful recovery when anything ephemeral is lost.