Paper Backtest Harness — Pattern
Before any strategy reaches a real venue, you need to know three things: does it produce trades, what does the P&L look like, and how fast does the dispatch path run. The pattern here is an in-process paper harness that wires together the same building blocks the production stack uses:
- The
Algotrait — exact same code path your venue edge runs. - A synthetic L2 book feed — deterministic, seedable, parameterised per venue.
- A paper matcher — turns the
Actionsbuffer intoFillevents against the synthetic book. - The fund-layer primitives —
Deployment,PreTradeGate,compute_pnl,LineageTag— used as a toolkit, not a framework. - A latency histogram — captures
on_bookdispatch time and order queue lifetime.
The point is to show the building blocks. You're expected to wire them together to match your own venue model.
The loop, in pseudocode
Setup:
feed = BookFeed::new(BookFeedConfig { … })
matcher = PaperMatcher::new(symbol, MatcherFees { … })
deployment = Deployment::new(name, strategy_id, version_id, caps…, clock)
gate = PreTradeGate::register(deployment.id, LimitSet { … })
algo = MyStrategy::default()
For each tick:
book = feed.next()
# (a) Match resting orders against the new book BEFORE the algo sees it
events = matcher.match_against_new_book(book)
drain_events(events) → on_fill/on_reject + gate.apply_fill + journal
# (b) Dispatch the algo — measure the latency
t0 = Instant::now()
algo.on_book(book, state, features, &mut actions)
on_book_hist.record(t0.elapsed())
# (c) Pre-trade gate every NEW action
for action in &mut actions:
if action.is_new() && gate.check(deployment.id, intent).is_err():
actions.clear_at(i) # rejected at admission
journal.record_rejected()
# (d) Submit surviving actions to the matcher
events = matcher.ingest_actions(actions, book)
drain_events(events) → on_fill/on_reject + journal
Report:
- compute_pnl(journal.fills, GroupBy::Total).enrich_unrealized(|sym| feed.mark(sym))
- histograms: on_book dispatch, order queue lifetime
- counters: submitted / accepted / rejected / fillsFive panels, one binary, deterministic. Same shape regardless of the strategy or venue.
Synthetic L2 feed — BookFeedConfig
The feed is a mean-reverting Gaussian walk on mid plus jittered geometric depth — enough microstructure to produce believable fills without recorded data. Parameterise it per venue:
let cfg = BookFeedConfig {
start_mid_usd: 50_000.0,
mean_mid_usd: 50_000.0,
half_spread_bps: 1, // top-of-book at mid ± 1 bp
tick_sigma_usd: 10.0, // per-tick stdev of mid
mean_revert_lambda: 0.01,
depth_levels: 10,
top_size_1e8: 100_000_000, // 1.0 unit at top
depth_geom: 0.7, // size decay per level
seed: 0xC0FFEE_DEADBEEF,
mid_floor_usd: 1.0,
mid_ceil_usd: f64::INFINITY,
};For a bounded-quote venue like Kalshi, set mid_floor_usd / mid_ceil_usd to enforce the legal range (Kalshi: [$0.05, $0.95] with margin). The feed clamps the random walk to those bounds; your strategy's quote-validation logic still has to respect tick size and [$0.01, $0.99].
Determinism matters for sims. The XorShift PRNG is seeded — same seed produces the same book sequence forever. Two runs with the same config produce the same fill ledger. That's the property that makes paper-harness debugging tractable.
Paper matcher — PaperMatcher
The matcher consumes the Actions buffer the strategy produced, classifies each entry, and emits events for the runner to fan out:
Action.is_cancel | Matcher behaviour |
|---|---|
ACTION_NEW, OrderType::IOC / FOK / MARKET | Match against the current book's top of the opposite side. Fill at that price as taker, fee = taker_fee_bps. No-fill → synthesise a STALE_VENUE reject so the strategy's working-flag logic clears. |
ACTION_NEW, OrderType::POST_ONLY | If it would cross the BBO → emit reject (INVALID_PARAMS), no rest. Otherwise rest and stamp an Instant for queue-lifetime measurement. |
ACTION_NEW, OrderType::LIMIT | Same as POST_ONLY, but a crossing limit fills immediately as taker. |
ACTION_CANCEL | Remove from resting list (no-op if already gone). |
ACTION_AMEND | In-place qty/px replace on the resting record. |
On the next tick, match_against_new_book walks the resting list and fills any order whose price has been crossed by the new top-of-book on the opposite side — fee = maker_fee_bps_signed (negative for rebates).
This is a worst-of-spread fill model for IOCs and an adverse-selection fill model for resting posts. Pessimistic on both axes, which is the right place to start.
Sign-convention rules (don't double-sign)
Two sign conventions live in adjacent code:
FillRow.qty_1e8is a magnitude; thesidestring"BUY"/"SELL"carries the direction.compute_pnlreads both and signs internally.- The fund pre-trade gate's
apply_fill(signed_delta_cents)takes a signed cents value: positive for buys (position up), negative for sells (position down).
Mixing the two — passing a signed quantity to record_fill while also passing signed cents to apply_fill — silently turns every sell into a buy, because compute_pnl then flips the sign a second time. Symptom: 100% long positions even when buy/sell fills are balanced. Encode the magnitude once, sign once.
Two-phase position tracking
The production CC's fill_router applies position deltas on actual fills, not on submit intents. Your paper harness should do the same:
| When | What |
|---|---|
gate.check(intent, now_ns) at submit | Reads current position, projects with intent.signed_delta(), rejects if projected.abs() > max_position_cents. Does not mutate position. |
gate.record_submitted(now_ns) after admit | Bumps the per-second rate-limit counter. |
gate.apply_fill(signed_cents) on a MatcherEvent::Fill | The single line that moves the position counter. |
If you apply the delta optimistically at submit time, balanced buy/sell intent flow nets to zero in the gate's view even when only one side is actually filling. Your cap silently never fires.
Honest latency, not synthetic latency
The matcher stamps Instant::now() when an order goes into the resting list, and measures elapsed() when it fills. That delta is the queue lifetime — useful for understanding adverse-selection cost. Don't compute latency off a synthetic recv_ns clock the feed assigned; you'll get nanosecond constants that mean nothing.
Two histograms worth recording:
on_book dispatch # Instant::now() before/after algo.on_book(…)
tick → fill # Instant submitted_at → Instant fill_emitted_atThe first measures NativeAlgo dispatch cost (single-digit ns in release mode). The second measures end-to-end paper-OMS round-trip and is dominated by the loop's wall-clock per tick.
Per-tick checklist
Every harness iteration should do these steps in this order:
feed.next()— produce the new book.matcher.match_against_new_book(book)— resting orders that the new book crosses fill before the algo sees the update. This is the adverse-selection tick.drain_events(...)— apply fills to the gate (apply_fill), record into the journal, firealgo.on_fill/algo.on_reject.algo.on_book(book, state, features, &mut actions)— measure dispatch time around this call.apply_pre_trade_gate(...)— walk NEW actions, project, reject the ones that breach. Zero rejected slots in-place.matcher.ingest_actions(actions, book)— submit surviving actions, immediate fills + new resting orders.drain_events(...)— fan results back into algo + journal.
Steps 1 → 7 are pure compute on a single thread; the whole loop runs inside one process, no I/O.
Report — what good output looks like
order metrics
submitted new 10000
submitted cancel 8473
submitted amend 0
accepted 9672
rejected 328
fills recorded 1525
accept ratio 96.72%
P&L (USD)
realized $25,913.00
unrealized -$25,026.82
fees $0.00
rebates $3,805.64
net $886.18
gross volume $38,131,675.90
latency (nanoseconds)
on_book dispatch count=5000 p50=41 p99=42 max=84 mean=28
tick → fill count=1525 p50=125 p99=208 max=250 mean=119If accept ratio is dropping below ~95 %, the position cap or the rate limit is firing — inspect the breach dimension. If fills recorded == 0, the strategy's quotes are sitting outside the feed's typical price walk — narrow the synthetic half-spread or widen tick_sigma. If on_book dispatch p50 jumps above 100 ns in release mode, you've introduced allocation on the hot path.
See also
- NativeAlgo dev guide — the strategy side of the harness.
- Fund-layer primitives —
Deployment,PreTradeGate,LineageTag,compute_pnl. - Sim engine — the production backtester, used when the synthetic feed isn't representative enough.
- PnL & Timing — the in-strategy timing helpers (
algo_sdk::time::Timer) you can use insideon_book.