ADR-008: Extract OHLC Data into Dedicated Microservice

Status: Accepted

Context

OHLC (candlestick) market data has unique operational characteristics that clash with mi-casa’s deployment model:

1. Live feed is sensitive to restarts. mi-casa deploys frequently as a general-purpose family app. Every deployment tears down the upstream market-data WebSocket connection, causing a gap in the live candle stream. There is no catch-up mechanism — missed ticks are lost.

2. TimescaleDB is not portable. TimescaleDB binary dumps are incompatible across major versions and extension states, making it impossible to seed a local development database from a production snapshot. Developers who want to test the markets feature locally must either connect to a shared remote database or backfill from scratch on every machine.

3. OHLC infrastructure changes at a different cadence. Schema migrations, market-data ingestion, and backfill logic are updated rarely and independently of general app features. Coupling them to mi-casa’s release cycle forces unnecessary risk onto unrelated changes.

Options considered:

Option	Pros	Cons
Keep OHLC in mi-casa	No network hop; single deployment	WebSocket drops on every deploy; TimescaleDB tied to app lifecycle
Dedicated microservice (chosen)	Deployed independently; live feed survives mi-casa redeploys; self-contained local dev	Extra service to operate; HTTP latency on candle reads; service-to-service auth required
Separate process, same host	Isolates deployment from feed	Still shares TimescaleDB ops; two processes to manage without container isolation
Third-party market data API	No infrastructure to own	Latency for live data; cost; no control over data model or retention

Decision

Extract all OHLC infrastructure into a standalone Bun microservice (microservices/ohlc-service/) that owns:

• TimescaleDB — the ohlc_candles hypertable and its migrations
• Live feed — cTrader Open API WebSocket → in-memory candle builder → persist closed candles → broadcast via WebSocket
• Backfill — historical data ingestion from cTrader
• REST API — GET /candles, GET /candles/latest, GET /candles/from, GET /export, GET /symbols, GET /status, POST /backfill (SSE stream)
• WebSocket server at /ws/ticks — broadcasts candle:update and candle:closed events to subscribers

mi-casa changes:

• ohlcCandleRepository replaced with an HTTP client that calls ohlc-service
• marketDataWebSocket connects to ohlc-service /ws/ticks instead of the upstream provider directly
• TimescaleDB connection and migrations removed from mi-casa startup
• ohlcPatterns, ohlcWatchlist, ohlcPatternDetectionEvents tables migrated from TimescaleDB → main Postgres (they hold application state, not time-series data)
• New env vars: OHLC_SERVICE_URL, OHLC_SERVICE_API_KEY
• Removed env vars: TIMESCALE_SERVER, TIMESCALE_DB, TIMESCALE_USER, TIMESCALE_PASSWORD
• New admin route: GET /api/ohlc/service-status proxies the ohlc-service /status response
• POST /api/ohlc/backfill proxies the SSE stream from ohlc-service

Authentication uses a static pre-shared API key (OHLC_SERVICE_API_KEY / OHLC_API_KEY). All HTTP requests carry it as an X-API-Key header; WebSocket connections pass it as a ?key= query parameter. The key is a server-side secret and is never forwarded to the browser.

Local development is simplified: run docker compose up ohlc-service timescale-db once, call POST /backfill, and OHLC data is available.

Historical note: ADR-010 later consolidated both live ticks and historical backfill onto cTrader, replacing the earlier Twelve Data / Dukascopy / Yahoo Finance mix.

See OHLC Service — Candle Data Flow for a detailed sequence diagram of the live feed and backfill paths.

Consequences

Positive

• mi-casa can be deployed and restarted at any time without interrupting the live candle feed
• TimescaleDB is isolated to ohlc-service — its lifecycle, migrations, and backups are managed independently of the main app
• Local development no longer requires third-party market-data credentials or a shared remote database — a single docker compose up + backfill is sufficient
• ohlc-service can be scaled, monitored, or replaced (e.g. swapped for a commercial data vendor) without touching mi-casa code
• ohlcPatterns, ohlcWatchlist, and ohlcPatternDetectionEvents now live in the same Postgres instance as the rest of the app — standard Drizzle migrations, seeds, and Postgres dumps work for them

Negative / Tradeoffs

• Every candle read from mi-casa now crosses a network hop to ohlc-service; latency is acceptable for the current use case but would be a bottleneck under high read frequency
• A second service must be running for the markets feature to function — if ohlc-service is down, candle endpoints return errors rather than degrading gracefully to cached data
• The static API key is a blunt auth mechanism: no per-consumer identity, no expiry, no rotation workflow. Acceptable for a single trusted internal consumer; would need revisiting if more services consume the API
• docker compose configuration grows: ohlc-service and timescale-db are new compose entries that developers must understand

Related

• Architecture Overview — updated directory structure and stack table
• Environment Variables — updated env var table (added OHLC_SERVICE_URL, OHLC_SERVICE_API_KEY; removed TimescaleDB vars)
• OHLC Service — Candle Data Flow — live feed sequence, backfill flow, and service-to-service auth pattern