Sequence/docs

Paper Backtest Harness — Pattern

Before any strategy reaches a real venue, you need to know three things: does it produce trades, what does the P&L look like, and how fast does the dispatch path run. The pattern here is an in-process paper harness that wires together the same building blocks the production stack uses:

  • The Algo trait — exact same code path your venue edge runs.
  • A synthetic L2 book feed — deterministic, seedable, parameterised per venue.
  • A paper matcher — turns the Actions buffer into Fill events against the synthetic book.
  • The fund-layer primitivesDeployment, PreTradeGate, compute_pnl, LineageTag — used as a toolkit, not a framework.
  • A latency histogram — captures on_book dispatch time and order queue lifetime.

The point is to show the building blocks. You're expected to wire them together to match your own venue model.


The loop, in pseudocode

text
Setup:
  feed       = BookFeed::new(BookFeedConfig { … })
  matcher    = PaperMatcher::new(symbol, MatcherFees { … })
  deployment = Deployment::new(name, strategy_id, version_id, caps…, clock)
  gate       = PreTradeGate::register(deployment.id, LimitSet { … })
  algo       = MyStrategy::default()
 
For each tick:
  book = feed.next()
 
  # (a) Match resting orders against the new book BEFORE the algo sees it
  events = matcher.match_against_new_book(book)
  drain_events(events) → on_fill/on_reject + gate.apply_fill + journal
 
  # (b) Dispatch the algo — measure the latency
  t0 = Instant::now()
  algo.on_book(book, state, features, &mut actions)
  on_book_hist.record(t0.elapsed())
 
  # (c) Pre-trade gate every NEW action
  for action in &mut actions:
    if action.is_new() && gate.check(deployment.id, intent).is_err():
      actions.clear_at(i)   # rejected at admission
      journal.record_rejected()
 
  # (d) Submit surviving actions to the matcher
  events = matcher.ingest_actions(actions, book)
  drain_events(events) → on_fill/on_reject + journal
 
Report:
  - compute_pnl(journal.fills, GroupBy::Total).enrich_unrealized(|sym| feed.mark(sym))
  - histograms: on_book dispatch, order queue lifetime
  - counters: submitted / accepted / rejected / fills

Five panels, one binary, deterministic. Same shape regardless of the strategy or venue.


Synthetic L2 feed — BookFeedConfig

The feed is a mean-reverting Gaussian walk on mid plus jittered geometric depth — enough microstructure to produce believable fills without recorded data. Parameterise it per venue:

rust
let cfg = BookFeedConfig {
    start_mid_usd:       50_000.0,
    mean_mid_usd:        50_000.0,
    half_spread_bps:     1,             // top-of-book at mid ± 1 bp
    tick_sigma_usd:      10.0,          // per-tick stdev of mid
    mean_revert_lambda:  0.01,
    depth_levels:        10,
    top_size_1e8:        100_000_000,   // 1.0 unit at top
    depth_geom:          0.7,           // size decay per level
    seed:                0xC0FFEE_DEADBEEF,
    mid_floor_usd:       1.0,
    mid_ceil_usd:        f64::INFINITY,
};

For a bounded-quote venue like Kalshi, set mid_floor_usd / mid_ceil_usd to enforce the legal range (Kalshi: [$0.05, $0.95] with margin). The feed clamps the random walk to those bounds; your strategy's quote-validation logic still has to respect tick size and [$0.01, $0.99].

Note

Determinism matters for sims. The XorShift PRNG is seeded — same seed produces the same book sequence forever. Two runs with the same config produce the same fill ledger. That's the property that makes paper-harness debugging tractable.


Paper matcher — PaperMatcher

The matcher consumes the Actions buffer the strategy produced, classifies each entry, and emits events for the runner to fan out:

Action.is_cancelMatcher behaviour
ACTION_NEW, OrderType::IOC / FOK / MARKETMatch against the current book's top of the opposite side. Fill at that price as taker, fee = taker_fee_bps. No-fill → synthesise a STALE_VENUE reject so the strategy's working-flag logic clears.
ACTION_NEW, OrderType::POST_ONLYIf it would cross the BBO → emit reject (INVALID_PARAMS), no rest. Otherwise rest and stamp an Instant for queue-lifetime measurement.
ACTION_NEW, OrderType::LIMITSame as POST_ONLY, but a crossing limit fills immediately as taker.
ACTION_CANCELRemove from resting list (no-op if already gone).
ACTION_AMENDIn-place qty/px replace on the resting record.

On the next tick, match_against_new_book walks the resting list and fills any order whose price has been crossed by the new top-of-book on the opposite side — fee = maker_fee_bps_signed (negative for rebates).

This is a worst-of-spread fill model for IOCs and an adverse-selection fill model for resting posts. Pessimistic on both axes, which is the right place to start.


Sign-convention rules (don't double-sign)

Two sign conventions live in adjacent code:

  • FillRow.qty_1e8 is a magnitude; the side string "BUY"/"SELL" carries the direction. compute_pnl reads both and signs internally.
  • The fund pre-trade gate's apply_fill(signed_delta_cents) takes a signed cents value: positive for buys (position up), negative for sells (position down).

Mixing the two — passing a signed quantity to record_fill while also passing signed cents to apply_fill — silently turns every sell into a buy, because compute_pnl then flips the sign a second time. Symptom: 100% long positions even when buy/sell fills are balanced. Encode the magnitude once, sign once.


Two-phase position tracking

The production CC's fill_router applies position deltas on actual fills, not on submit intents. Your paper harness should do the same:

WhenWhat
gate.check(intent, now_ns) at submitReads current position, projects with intent.signed_delta(), rejects if projected.abs() > max_position_cents. Does not mutate position.
gate.record_submitted(now_ns) after admitBumps the per-second rate-limit counter.
gate.apply_fill(signed_cents) on a MatcherEvent::FillThe single line that moves the position counter.

If you apply the delta optimistically at submit time, balanced buy/sell intent flow nets to zero in the gate's view even when only one side is actually filling. Your cap silently never fires.


Honest latency, not synthetic latency

The matcher stamps Instant::now() when an order goes into the resting list, and measures elapsed() when it fills. That delta is the queue lifetime — useful for understanding adverse-selection cost. Don't compute latency off a synthetic recv_ns clock the feed assigned; you'll get nanosecond constants that mean nothing.

Two histograms worth recording:

text
on_book dispatch     # Instant::now() before/after algo.on_book(…)
tick → fill          # Instant submitted_at → Instant fill_emitted_at

The first measures NativeAlgo dispatch cost (single-digit ns in release mode). The second measures end-to-end paper-OMS round-trip and is dominated by the loop's wall-clock per tick.


Per-tick checklist

Every harness iteration should do these steps in this order:

  1. feed.next() — produce the new book.
  2. matcher.match_against_new_book(book) — resting orders that the new book crosses fill before the algo sees the update. This is the adverse-selection tick.
  3. drain_events(...) — apply fills to the gate (apply_fill), record into the journal, fire algo.on_fill / algo.on_reject.
  4. algo.on_book(book, state, features, &mut actions) — measure dispatch time around this call.
  5. apply_pre_trade_gate(...) — walk NEW actions, project, reject the ones that breach. Zero rejected slots in-place.
  6. matcher.ingest_actions(actions, book) — submit surviving actions, immediate fills + new resting orders.
  7. drain_events(...) — fan results back into algo + journal.

Steps 1 → 7 are pure compute on a single thread; the whole loop runs inside one process, no I/O.


Report — what good output looks like

text
order metrics
  submitted new     10000
  submitted cancel  8473
  submitted amend   0
  accepted          9672
  rejected          328
  fills recorded    1525
  accept ratio      96.72%
 
P&L (USD)
  realized          $25,913.00
  unrealized       -$25,026.82
  fees                    $0.00
  rebates             $3,805.64
  net                   $886.18
  gross volume   $38,131,675.90
 
latency (nanoseconds)
  on_book dispatch    count=5000   p50=41   p99=42    max=84    mean=28
  tick → fill         count=1525   p50=125  p99=208   max=250   mean=119

If accept ratio is dropping below ~95 %, the position cap or the rate limit is firing — inspect the breach dimension. If fills recorded == 0, the strategy's quotes are sitting outside the feed's typical price walk — narrow the synthetic half-spread or widen tick_sigma. If on_book dispatch p50 jumps above 100 ns in release mode, you've introduced allocation on the hot path.


See also

  • NativeAlgo dev guide — the strategy side of the harness.
  • Fund-layer primitivesDeployment, PreTradeGate, LineageTag, compute_pnl.
  • Sim engine — the production backtester, used when the synthetic feed isn't representative enough.
  • PnL & Timing — the in-strategy timing helpers (algo_sdk::time::Timer) you can use inside on_book.