Optimisation System — Dependency Map

This document maps every dependency in the parameter optimisation pipeline: what triggers it, how the grid is built, how the best config is selected, and how (and how far) it propagates back to live trading. Read alongside Trading System — Dependency Map and Backtest System — Dependency Map.

---

1. System Overview

Admin UI ──► POST /api/ohlc/optimise (mi-casa backend, admin-only)
                │
                ▼
        optimisationController.triggerOptimisation
                │ (proxies as SSE to ohlc-service)
                ▼
        POST /optimise (ohlc-service)
                │
        optimisationService.runOptimisation
                │
        ┌───────┴────────────────────┐
        │                           │
  saved_patterns mode         walk_forward mode
  backtestService             walkForwardScanner → walkForwardBacktestService
  (N × runBacktest)           (N × runBacktest on scanned patterns)
        │
        ▼
  score + select best run
        │
        ▼
  persist to optimisation_runs (ohlc-service DB)
        │
        ▼
  SSE result stream → Admin UI

  ─── Later ───────────────────────────────

Admin UI ──► POST /api/ohlc/optimise/apply (mi-casa backend)
                │
        optimisationController.applyBestConfig
                │ (fetches best run from ohlc-service)
                ▼
        patternTradingConfigService.upsertSystemConfig
                │
        pattern_trading_configs (mi-casa DB)

---

2. Prerequisites for Optimisation to Run

#	Condition	Source
1	OHLC_SERVICE_URL and OHLC_SERVICE_API_KEY set	optimisationController.ts
2	Caller authenticated with admin role	optimisationRoutes.ts
3	Valid symbol, timeframe in request	postSchema (Zod)
4	ohlc_patterns data exists (saved_patterns mode)	ohlcPatternRepository
5	ohlc_candles data exists for the full date range (walk_forward mode)	ohlcCandleRepository.getFrom
6	optimisation_runs table exists in ohlc-service DB	migration

---

3. The Parameter Grid

The UI submits a strategyGrid and ohlc-service expands it into combinations (N runs per request).

Key TP sweep rule:

• tpTargetMode is dynamic: always tp1; add tp1_tp2 only when TP2 multipliers are present; add tp1_tp2_tp3 only when TP3 multipliers are present alongside TP2.

Other strategy-specific params (for example ABCD/EMA/RSI fields) are carried in the grid and persisted per run in optimisation_runs.params.

ABCD optimisation runs use the same emerging ABC / first-PRZ-touch path as the backtest runner; they do not wait for a confirmed D swing.

---

4. Scoring and Best-Run Selection

After all combinations complete:

Step 1 — R:R filter

Runs that have profitFactor >= 1.0 and clear a sample-size floor are marked as passing. The sample-size floor is dynamic: max(5, ceil(maxCompletedTrades × 0.35)).

Step 2 — Composite score

compositeScore =
  expectancyScore × 0.25 +              // expectancy in R
  profitFactorScore × 0.18 +
  returnToDrawdownScore × 0.18 +
  drawdownScore × 0.12 +                // lower max drawdown ranks higher
  rrScore × 0.10 +                      // capped at 2:1
  winRate × 0.07 +
  tradeCountScore × 0.05 +
  consecutiveLossScore × 0.05           // lower max losing streak ranks higher

Step 3 — Best selection

• If any passing runs: pick highest compositeScore from passing.
• If no passing runs (best-effort): first prefer runs that still meet the sample-size floor, then pick highest compositeScore; ties prefer higher completedTrades, then higher totalPnlPips.
• isBestEffort: true in the result signals this fallback was used.

Step 4 — Persistence

All completed runs are inserted into optimisation_runs with isBest=true for the winner.

---

5. Result Shape (OptimisationResult)

{
  runGroupId: string       // UUID grouping all runs for one request
  bestConfig: TradingConfig
  bestRun: OptimisationRun
  allRuns: OptimisationRun[]
  isBestEffort: boolean    // true = no run passed qualification; best-effort fallback used
}

OptimisationRun mirrors BacktestResult statistics plus tradingConfig (JSONB), compositeScore, isBest, runGroupId.

---

6. Retrieving Results

GET /api/ohlc/optimise/results?symbol=&timeframe=&patternType=&limit=

Returns an array of run groups, each with:

• runGroupId, createdAt, totalRuns, bestRun

Query strategy in optimisationRunRepository.getLatestGroups:

1. Group by run_group_id, get newest groups first (by earliest created_at in group).
2. For each group: fetch isBest=true run first; fall back to highest compositeScore.

---

7. Applying the Best Config (POST /api/ohlc/optimise/apply)

applyBestConfig(runGroupId)
  → fetch bestRun from ohlc-service /optimise/results?runGroupId=...
  → ohlcConfigToInput(bestRun.tradingConfig)  → Partial<NewPatternTradingConfig>
  → patternTradingConfigService.upsertSystemConfig(bestRun.patternType, input)
  → stored in pattern_trading_configs (mi-casa DB)

What gets applied vs. what is lost

Config field	Applied to patternTradingConfigs?	Read by auto-trading engine?
slMultiplier	✅	✅ (via getMergedConfig)
tp1-3Multiplier	✅	✅
minSignalScore	✅	✅
entryLagCandles	✅	✅
entryStrategy	✅	✅
tpTargetMode	❌ (dropped — see Bug A below)	⚠️ read from account, not config
partialExitEnabled	❌ (dropped)	⚠️ read from account, not config
partialExitTp1Percent	❌ (dropped)	⚠️ read from account, not config

⚠️ Bug A: OhlcTradingConfig interface in optimisationController.ts only declares tpCount (legacy) and not tpTargetMode / partialExitEnabled. Since the optimisation JSONB stores tpTargetMode (not tpCount), the mapping produces tpCount: undefined → tpCount is never set in the apply call → the DB retains its previous value.

⚠️ Bug B (architectural): tpCount and partialClose in patternTradingConfigs are never read by the auto-trading engine. The engine reads tpTargetMode and partialExitEnabled from paperTradingAccounts. Storing these in patternTradingConfigs has no effect on live trades. The optimisation’s TP-strategy recommendation must be applied manually to the account settings to take effect.

Fallback when patternType is ‘all’

When optimisation runs across all pattern types, bestRun.patternType = 'all'. upsertSystemConfig('all', ...) creates/updates the patternType = 'all' system config. This config sits at step 8 of the getMergedConfig 8-step fallback chain — it’s the lowest-priority catch-all for all patterns that have no more specific config.

---

8. Config Propagation to Live Trading

After applyBestConfig:

pattern_trading_configs (patternType, userId=null) — system default
        │
        ▼
patternTradingConfigService.getMergedConfig(userId, patternType, symbol, timeframe)
        │  (8-step fallback: user-scoped → user-global → sys-scoped → sys-global)
        ▼
autoTradingEngine.evaluatePatternEvent
        │  uses from config: slMultiplier, tp1-3Multiplier, entryStrategy,
        │                    minSignalScore (via portfolioStrategy, separate),
        │                    entryLagCandles
        │
        │  uses from ACCOUNT (not config): tpTargetMode, partialExitEnabled,
        │                                  partialExitTp1Percent, maxOpenTrades,
        │                                  maxEntryDeviationPips, lotSize, currency
        ▼
paperTradeRepository.create(...)

See Trading System — Dependency Map §3–§6 for the full live trade path.

---

9. API Reference

Method	Path	Auth	Description
POST	/api/ohlc/optimise	JWT + admin	Trigger optimisation (SSE stream)
GET	/api/ohlc/optimise/results	JWT + admin	List past optimisation groups
POST	/api/ohlc/optimise/apply	JWT + admin	Apply best run’s config as system default

---

10. Database

optimisation_runs (ohlc-service DB)

Column	Type	Notes
id	uuid PK	pre-generated
run_group_id	uuid	groups all runs from one optimisation
symbol, timeframe, pattern_type	varchar	pattern_type = 'all' when no filter
trading_config	jsonb	full TradingConfig object (canonical fields)
completed_trades, wins, losses, max_consecutive_losses	integer
win_rate, avg_win_pips, avg_loss_pips	numeric
profit_factor, risk_reward_ratio	numeric nullable	null when no losses
total_pnl_pips, max_drawdown_pips, max_drawdown_pct	numeric
composite_score	numeric	0..1 weighted score
is_best	boolean	true for the winning run in the group
triggered_by_user_id	varchar nullable	who triggered it
created_at	timestamptz

pattern_trading_configs (mi-casa DB)

Receives updates from applyBestConfig. See Trading System — Dependency Map §5 for full column list and getMergedConfig fallback chain.

---

11. Known Bugs / Limitations

| ID | Summary | Severity | | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | -------------------- | ------ | | #458 | OhlcTradingConfig missing tpTargetMode → applied config always gets wrong tpCount | High | | #459 | tpCount / partialClose in pattern_trading_configs never read by engine — TP strategy from optimisation has no effect on live trading | High (architectural) | | #460 | minSignalScore backtest filter passes null-signalScore patterns — backtest over-optimistic vs. live (see Backtest System) | Medium | | #461 | Backtest entryPrice in output is lag=0 open even when entryLagCandles > 0 (see Backtest System) | Medium |