---
title: "Simulation–Production Parity"
book: "Research, Data and Risk Platforms"
subject: quant
language: en
chapter: 12
exercises: 8
source: https://one-course.com/books/quant/15/en/chapter/12-simulationproduction-parity
---

# Chapter 12 — Simulation–Production Parity

A strategy that had passed its replay parity test quoted a tick wider than intended on its first morning in production, for the whole morning. Its opening period ended on a timer, and the timer had been set from the machine’s clock: in production the machine’s clock and the venue’s agreed, in the backtest they did not, and the parity test had compared the backtest with itself. The code was the same object in both places; the environments around it were not. This chapter builds the host that lets one strategy object run in a backtest, in the simulator and on a production-shaped path, and the harness that finds, event by event, where two of those runs part.

## 12.1 One strategy, several environments

**Definition 12.1 (Simulation–production parity).**

*Simulation–production parity* is the property that a strategy, given the same inputs in the same order, produces the same outputs in simulation and in production: the same orders and cancels, at the same prices and quantities, in response to the same events.

Parity is a property of the strategy and of everything around it. Book 7 (chapter 19) tested it by replaying a live day’s inputs into the backtest and comparing orders; Book 12 (chapter 24) met its cousin in machine learning, the training–serving skew. A strategy can fail it by itself (it reads something that is not an input: the machine’s clock, a random number, the order of a dictionary) or through its environments (they deliver the same market in different types, orders or semantics). The platform removes the second kind by construction and exposes the first kind by test.

**Definition 12.2 (Strategy host, environment adapter).**

A *strategy host* is the component that holds a strategy object, gives it its only view of the world — a clock, market data and order entry — and delivers its inputs in the platform’s [event model](https://one-course.com/books/quant/15/en/chapter/11-backtest-engine-architecture#def-pl-backtest-engine-architecture-event). An *environment adapter* connects the host to one environment (a backtest, a simulator, a production connection), translating that environment’s events into the host’s inputs and the strategy’s requests into the environment’s messages.

![One strategy object in three environments. The host gives it a clock, market data and order entry, and delivers every input in the event model; each environment adapter translates its environment. The in-process and production-shaped adapters trade on the same simulated session; the replay reads the production run’s input journal.](https://one-course.com/images/onecourse/chapters/quant-15/pl-simulation-production-parity/fig-1908dc199908.svg)

***Figure 12.1.** One strategy object in three environments. The host gives it a clock, market data and order entry, and delivers every input in the [event model](https://one-course.com/books/quant/15/en/chapter/11-backtest-engine-architecture#def-pl-backtest-engine-architecture-event); each [environment adapter](#def-pl-simulation-production-parity-host) translates its environment. The in-process and production-shaped adapters trade on the same simulated session; the replay reads the production run’s input journal.*

The chapter’s three environments trade on Book 10’s simulator with one `firm.tape` hour as background flow (chapter 11). The first calls the simulator’s agent context directly. The second is shaped like production: the strategy’s orders pass through Book 13’s order gateway (`firm.ordergw`: its order state machine, its throttle and its worst-case exposure check), are encoded as the simulator’s order-entry messages in SoupBinTCP frames on an in-memory byte stream, and the venue’s reports come back as bytes the same way; every input the strategy receives is written to an input journal in Book 13’s format (chapter 23: a receive time and a 48-byte record). The third replays that journal into a fresh strategy object: the backtest of recorded inputs.

## 12.2 Abstracting time

The host’s clock is the only clock a strategy may read: `ctx.now` is the time of the input being delivered, and `ctx.set_timer(delay, tag)` asks for a timer input `delay` later. In the simulator both are simulated time; in production both would be the venue-synchronised clock of the machine; in the replay, `now` is the journal’s receive time and timers are not set but replayed, since the timers that fired in production are inputs in the journal. A strategy that reads the machine’s clock directly reads something the host does not control, and its behaviour then depends on where and how fast it runs. The chapter’s harness makes this visible by giving each environment the machine clock it would really have: the venue’s time in production, and, in the simulator and the replay, the clock of a machine replaying the session at 20:00, a hundred times faster than real time.

## 12.3 Abstracting market data

**Definition 12.3 (Market-data abstraction).**

A *market-data abstraction* is the one representation of market events that a hosted strategy receives in every environment: here the top of the book as integers — bid, bid size, ask, ask size — with prices in 1/10 000 of the currency unit, whatever the source’s own encoding.

The rule that prices are integers is Book 10’s and the series’: a price is a count of the smallest unit, and every conversion to a binary floating-point number is a chance to be off by one. Python’s documentation says it plainly: most decimal fractions cannot be represented exactly as binary fractions. `99.99 - 0.01` is `99.97999999999999`, and `int()` of ten thousand times it is 999 799, one unit below the tick grid of 999 800.

## 12.4 Abstracting order entry

**Definition 12.4 (Order-entry abstraction).**

An *order-entry abstraction* is the one interface through which a hosted strategy sends and cancels orders and learns their fate, with the same semantics in every environment: which orders count as working (here, sent and not yet known to be finished, whether or not acknowledged), which callbacks arrive (acknowledgement, fill, cancel, reject), and what happens to a request that can no longer succeed.

Semantics are where environments differ silently. Book 13’s gateway (chapter 21) refuses to cancel an order it already knows to be filled; a simulator’s agent context passes the cancel to the venue, which rejects it as too late. The same strategy then sees a reject in one environment and nothing in the other, and the venue sees an extra message that takes 500 nanoseconds of its matching engine’s time. The harness below found exactly this difference on its first clean run, before any defect had been planted: the in-process adapter now refuses, as the gateway does, to cancel an order a report has already finished.

```python
    def working(self) -> list[int]:
        states = ("live",) if self.env.acked_only else ("pending", "live")
        return [o for o, v in self.orders.items() if v["state"] in states]

    def set_timer(self, delay_ns: int, tag: int) -> None:
        self.env.set_timer(delay_ns, tag)

    # -- inputs, buffered per time and delivered in the event model's order
    def input(self, t: int, cls: str, *args) -> None:
        self.buf.append((t, cls, args))

    def flush(self) -> None:
        batch, self.buf = self.buf, []
        if not self.d.arrival_order:          # the event model; stable within a class
            batch.sort(key=lambda x: (x[0], RANK[x[1]]))
        else:                                 # as the adapter read them: order entry first
            batch.sort(key=lambda x: (x[0], x[1] == "book"))
        for t, cls, args in batch:
            self.now = t
            self.run.inputs.append((t, cls) + tuple(args))
            self._dispatch(cls, args)
```

***Listing 12.1.** The host’s side of the abstractions: which orders are working, timers, and the delivery of simultaneous inputs in the event model (market data, then order entry, then timers). code/firm/strathost/firm_strathost.py*

```python
    def send(self, oid, side, price, qty) -> int:
        px = self.price_out(price)
        side_ = "B" if side == 1 else "S"
        verdict, msg = self.gw.new(self.ctx.now_ns, oid + 1, side_, int(qty), px)
        if verdict != "send":
            self.host.input(self.ctx.now_ns, "reject", oid)
            return px
        o = C.NT["in"]["O"](msg[1], 1, msg[2], msg[3], msg[4], "D", "Y", "N", 0, 0, 0, "N", 0)
        self.to_venue += C.soup_frame("U", C.encode("in", o))
        self._pump_venue()
        return px

    def cancel(self, oid) -> None:
        verdict, msg = self.gw.cancel(self.ctx.now_ns, oid + 1)
        if verdict == "send":
            self.to_venue += C.soup_frame("U", C.encode("in", C.NT["in"]["X"](msg[1], 0)))
            self._pump_venue()

    def _pump_venue(self) -> None:
        """The venue side of the byte stream: whole frames become the simulator's orders."""
        frames, rest = C.soup_parse(bytes(self.to_venue))
        self.to_venue = bytearray(rest)
        for _typ, payload in frames:
            m = C.decode("in", payload)
            if type(m).__name__ == "In_O":
                self.ctx.send(X.Order(1, m.side, m.qty, m.price, cl_ord_id=m.cl_ord_id))
            else:
                self.ctx.cancel(m.cl_ord_id)
```

***Listing 12.2.** The production-shaped adapter: an order passes Book 13’s gateway, is encoded and framed onto a byte stream, and becomes an order at the venue side of the stream. code/firm/strathost/firm_strathost.py*

## 12.5 The parity harness

**Definition 12.5 (Parity harness).**

A *parity harness* runs one strategy in two environments on the same inputs and compares their outputs event by event, reporting the first output that differs, the inputs delivered before it, and how many outputs differ in all.

The harness compares the production-shaped run with the in-process simulator run (the same session, two adapters) and with the replay of its own journal (the same inputs, delivered again). Outputs are compared as the venue would see them: time, order or cancel, side, price on the wire, quantity. The first difference and the number of inputs before it are what a developer needs to start reading; the count of differing outputs says how much a defect costs in an hour.

```python
def compare(a: HostRun, b: HostRun) -> Parity:
    n = min(len(a.outputs), len(b.outputs))
    first = next((i for i in range(n) if a.outputs[i] != b.outputs[i]), -1)
    if first < 0 and len(a.outputs) != len(b.outputs):
        first = n
    if first < 0:
        return Parity(True, -1, len(a.inputs), 0)
    pool: dict = {}
    for x in b.outputs:
        pool[x] = pool.get(x, 0) + 1
    affected = 0
    for x in a.outputs:
        if pool.get(x, 0):
            pool[x] -= 1
        else:
            affected += 1
    src = a if first < len(a.outputs) else b
    return Parity(False, first, src.marks[first], affected, src.outputs[first][0])
```

***Listing 12.3.** The comparison: the first differing output, the inputs delivered before it, and the outputs of one run with no identical output in the other. code/firm/strathost/firm_strathost.py*

The chapter’s strategy ([Listing 12.4](#lst-pl-simulation-production-parity-quoter)) quotes a lot at the touch, one tick wider during an opening period of thirty seconds from the start of the session, which a timer ends; it stops adding at three lots of inventory. Four defects are planted, one at a time, in the production-shaped run:

- *wall clock* : the strategy takes the end of its opening period and its timer’s delay from the machine’s clock;
- *float prices* : the production adapters hand prices to the strategy as floating-point currency and convert them back with `int()` ;
- *callback order* : the production host delivers simultaneous inputs as its adapter read them, order entry first;
- *ack semantics* : production’s `working()` lists only acknowledged orders.

```python
class OpeningQuoter(S.HostedStrategy):
    """A lot at the touch, a tick wider until the opening ends; stop adding at three lots."""

    def __init__(self, wall_clock: bool = False, limit: int = 300):
        self.wall, self.limit, self.mine = wall_clock, limit, {}

    def _now(self, ctx) -> int:
        return S.machine_ns() if self.wall else ctx.now   # the defect reads the machine

    def on_start(self, ctx):
        self.open_end = self._now(ctx) + OPENING
        ctx.set_timer(self.open_end - ctx.now, 1)        # mixes clocks under the defect

    def on_timer(self, ctx, tag):
        for oid in list(self.mine):                       # opening over: requote at the touch
            ctx.cancel(oid)
        self.mine = {}

    def on_book(self, ctx, top):
        bid, _, ask, _ = top
        tick = TICK / S.SCALE if isinstance(bid, float) else TICK
        wide = self._now(ctx) < self.open_end
        want = {1: bid - tick if wide else bid, -1: ask + tick if wide else ask}
        live = set(ctx.working())
        self.mine = {o: sp for o, sp in self.mine.items() if o in live}
        have = set()
        for oid, (side, px) in list(self.mine.items()):
            if px != want[side]:
                ctx.cancel(oid)
                del self.mine[oid]
            else:
                have.add(side)
        for side in (1, -1):
            if side not in have and side * ctx.position < self.limit:
                self.mine[ctx.send(side, want[side], 100)] = (side, want[side])
```

***Listing 12.4.** The hosted strategy. With the wall-clock defect on, `_now` reads the machine and the timer’s delay mixes the machine’s clock with the host’s. code/platforms/12-simulation-production-parity/python/pl_parity.py*

## 12.6 Four defects

![Each planted defect in the production-shaped run against the in-process simulator, on the same one-hour session: the inputs delivered before the first output that differs, and the outputs (orders and cancels) of the production run with no identical output in the simulator run. Data: fig_parity.py.](https://one-course.com/images/onecourse/chapters/quant-15/pl-simulation-production-parity/fig-7a524f4e6dc8.svg)

***Figure 12.2.** Each planted defect in the production-shaped run against the in-process simulator, on the same one-hour session: the inputs delivered before the first output that differs, and the outputs (orders and cancels) of the production run with no identical output in the simulator run. Data: `fig_parity.py`.*

[Figure 12.2](#fig-pl-simulation-production-parity-defects) and [Table 12.1](#tab-pl-simulation-production-parity-defects) give the harness’s report for each defect on seed 1.

| defect | first diverging output | inputs before | outputs affected |
| --- | --- | --- | --- |
|  | index | seconds |  | vs simulator | vs replay |
| wall clock | 14 | 30.0 | 175 | 539 of 553 | 305 |
| float prices | 3 | 24.4 | 126 | 1 686 of 1 690 | 1 161 |
| callback order | 20 | 33.0 | 203 | 387 of 463 | 416 |
| ack semantics | 2 | 1.0 | 2 | 8 449 of 8 452 | 8 450 |

***Table 12.1.** The harness’s report for each planted defect: the index and time (seconds from the start of the session, one second before the open) of the first output of the production-shaped run that differs from the in-process simulator’s, the inputs delivered before it, and the outputs of the production run with no identical counterpart in the simulator run and in the replay of its own journal.*

**Example 12.6 (The morning the timer fired).**

With the wall-clock defect, production is unaffected: its machine clock is the venue’s, the timer fires thirty seconds after the start and the strategy moves to the touch. In the simulator the timer’s delay is the machine’s 20:00 plus thirty seconds minus the session’s 09:29:59, more than ten hours, and it never fires in the hour; the machine clock, a hundred times slower than the session’s, keeps the opening period open for fifty minutes. The first diverging output is production’s cancel at 30.0 seconds, after 175 inputs, and 539 of production’s 553 outputs have no twin in the simulator. The replay of production’s own journal receives the timer from the journal but still reads its own machine clock, requotes a tick wide, and parts at 30.1 seconds.

![The wall-clock defect in the three environments over the hour: shaded dark, the strategy quotes a tick wide; light, at the touch. In production the machine’s clock is the venue’s and the opening lasts thirty seconds. In the simulator and the replay the machine runs a hundred times slower than the session, from 20:00, and the opening lasts fifty minutes.](https://one-course.com/images/onecourse/chapters/quant-15/pl-simulation-production-parity/fig-7416644b49a6.svg)

***Figure 12.3.** The wall-clock defect in the three environments over the hour: shaded dark, the strategy quotes a tick wide; light, at the touch. In production the machine’s clock is the venue’s and the opening lasts thirty seconds. In the simulator and the replay the machine runs a hundred times slower than the session, from 20:00, and the opening lasts fifty minutes.*

The other three defects read the same way. With float prices the first difference is the third output, at 24.4 seconds: the strategy’s wide bid $99.99 - 0.01$ leaves as 999 799; 1 161 of production’s 1 501 orders are off the tick grid, rejected by the venue and sent again at the next quote. With inputs delivered in arrival order the strategy first differs at 33.0 seconds, after 203 inputs, on an instant where a fill and a quote change arrived together: delivered fill first, it sent its next sell order at once; delivered as the [event model](https://one-course.com/books/quant/15/en/chapter/11-backtest-engine-architecture#def-pl-backtest-engine-architecture-event) delivers them, quote first, it sent the same order 0.19 seconds later. Neither order is wrong; two orders are. With working meaning acknowledged, the next quote change after the first two orders, half a microsecond later and long before their acknowledgements, makes the strategy send both again, and every quote change inside the forty-microsecond round trip does the same: 8 452 outputs in the hour, 8 406 of them refused by the gateway’s worst-case exposure limit, which is all that stops the storm.

**Remark 12.7 (Why the replay alone was not enough).**

A replay parity test (Book 7, chapter 19) catches a strategy that reads what is not an input, but only if the replay environment differs from production in that input: it caught the wall clock here because the harness gave the replay a machine clock of its own. It catches adapter defects only when the replay host has the correct semantics and production’s does not. Comparing two live-shaped environments on the same session catches what the replay misses; comparing the replay with production catches what two simulators share.

## 12.7 Parity after the fixes

With every defect fixed — the strategy reads only the host’s clock, prices are integers end to end, the host delivers simultaneous inputs in the [event model](https://one-course.com/books/quant/15/en/chapter/11-backtest-engine-architecture#def-pl-backtest-engine-architecture-event) everywhere, and working means sent and not finished everywhere — the three environments agree on every output for ten one-hour sessions ([Table 12.2](#tab-pl-simulation-production-parity-clean)): 8 887 outputs from 188 079 inputs, no difference between the production-shaped run, the simulator and the replay.

| seed | inputs | outputs | simulator | replay |
| --- | --- | --- | --- | --- |
| 1 | 9 492 | 553 | identical | identical |
| 2 | 16 085 | 832 | identical | identical |
| 3 | 36 396 | 1 463 | identical | identical |
| 4 | 37 621 | 1 359 | identical | identical |
| 5 | 13 911 | 784 | identical | identical |
| 6 | 29 785 | 1 426 | identical | identical |
| 7 | 6 947 | 409 | identical | identical |
| 8 | 13 410 | 726 | identical | identical |
| 9 | 20 027 | 979 | identical | identical |
| 10 | 4 405 | 356 | identical | identical |

***Table 12.2.** Parity after the fixes: for ten one-hour sessions, the inputs and outputs of the production-shaped run and whether the in-process simulator run and the replay of the production journal produced identical outputs.*

## 12.8 Tutorial: one strategy, three environments, four defects

**Goal.** Host one strategy in three environments, plant four parity defects and let the harness find each; then show parity. **End state:** Tables [12.1](#tab-pl-simulation-production-parity-defects) and [12.2](#tab-pl-simulation-production-parity-clean).

1. **Environments** : `run_env` with `prod` and with `sim` on the same session; `replay` of the production run’s journal.
2. **The harness** : `compare(prod, sim)` and `compare(prod, replay)` .
3. **Defects** : `pl_parity.defect_table()` , one defect at a time.
4. **Parity** : `pl_parity.clean()` on seeds 1 to 10.

**What to change next.** Plant a fifth defect — a strategy that iterates over a set of order identifiers when it cancels — and see whether the harness finds it; give the in-process adapter a different latency from production’s and see what parity then means.

## 12.9 Build: the strategy host

**Purpose.** One strategy object that runs unchanged in the backtest, the simulator and production, and a harness that proves it.

**Interface.** `HostedStrategy` (`on_start`, `on_book`, `on_ack`, `on_fill`, `on_cancel`, `on_reject`, `on_timer`); the context (`now`, `set_timer`, `send`, `cancel`, `working`, `position`); `run_env(env, strategy, seconds, seed, defects)`; `replay(journal, strategy)`; `compare(a, b) -> Parity`; `Defects`.

**Rules.** The strategy reads nothing the host does not give it; prices are integers; simultaneous inputs in the [event model](https://one-course.com/books/quant/15/en/chapter/11-backtest-engine-architecture#def-pl-backtest-engine-architecture-event); the same order-entry semantics in every adapter; production’s inputs journalled in Book 13’s format; Book 13’s gateway and Book 10’s simulator used, not edited.

**Acceptance tests.** `code/firm/strathost/tests/`: the delivery order of simultaneous inputs, with and without the defect; clean parity of production, simulator and replay on a short session, and the journal’s records; divergence under each adapter defect; the machine clock; the float adapter’s truncation; no cancel of a finished order.

**Stretch.** A fourth environment on a real socket; parity across implementation languages (the C++20 and Rust gateways of Book 13); a bisecting harness on Book 13’s `first_difference`.

Sources and further reading

- One Quant Book 7, chapter 19 (replay parity); One Quant Book 13, chapters 21 and 23 (the order gateway, input journals); One Quant Book 10, chapter 26 (the simulator and its protocols).
- Python documentation, “Floating-Point Arithmetic: Issues and Limitations”.

## 12.10 Exercises

**Exercise 12.1 ★.**

Why is `int(ten thousand times (99.99 - 0.01))` 999 799, and what is the correct conversion?

**Solution of Exercise 12.1.**

Neither 99.99 nor 0.01 has an exact binary representation, and their difference is stored as `99.97999999999999`; ten thousand times it is just below 999 800, and `int()` truncates. Keep prices as integer counts of the smallest unit (999 900 $-$ 100); if a float must be converted, round to the nearest unit, never truncate.

**Exercise 12.2 ★.**

In the replay environment, why are timers not set but replayed?

**Solution of Exercise 12.2.**

The timers that fired in production are inputs recorded in the journal, at the times they fired. Setting them again in the replay would deliver each timer twice, or at a different time if the strategy computed its delay differently; the replay must reproduce what the strategy received.

**Exercise 12.3 ★.**

With the wall-clock defect, why is production itself unaffected?

**Solution of Exercise 12.3.**

In production the machine’s clock is synchronised with the venue’s, so the machine’s time and the host’s time agree and the defect has no visible effect there; it shows wherever the two differ — every simulation and every replay.

**Exercise 12.4 ★★.**

Why does the ack-semantics defect produce a storm rather than a few duplicate orders, and what stops it?

**Solution of Exercise 12.4.**

Every quote change that arrives before the acknowledgements makes the strategy believe it has no working order and send another, and each new order is itself a quote change for the next round; the storm grows with the quote rate inside the forty-microsecond round trip. The gateway’s worst-case exposure limit stops it: it counts every order that might still fill and refused 8 406 of the sends.

**Exercise 12.5 ★★.**

The harness found an unplanned difference on its first clean run. Describe it, and say which adapter was right.

**Solution of Exercise 12.5.**

When the strategy cancelled an order whose fill had arrived in the same instant but not yet been delivered, the gateway refused the cancel locally, while the in-process adapter sent it; the venue rejected it as too late, the strategy received a reject that production never sent, and the venue’s extra message delayed the next order by 500 nanoseconds. The gateway was right: a cancel that cannot succeed should not reach the venue; the in-process adapter now refuses it too.

**Exercise 12.6 ★★.**

Why can the count of outputs affected differ between the comparison with the simulator and with the replay?

**Solution of Exercise 12.6.**

The two comparisons pair production’s outputs with different second runs: the simulator trades its own history after the divergence (different fills, different quotes), while the replay receives production’s inputs and differs only where the strategy’s decisions differ. A defect whose effects compound through the market affects more outputs against the simulator.

**Exercise 12.7 ★★★.**

*Coding.* Run `run_three` for seed 2 with each defect. Are the first diverging outputs at the same times as for seed 1?

**Solution of Exercise 12.7.**

No: the first differences depend on the session — when the first price needs a float subtraction that lands off the grid, when a fill and a quote change first coincide — except for the wall clock, whose first difference is the timer at thirty seconds in any session with quotes before it, and the ack semantics, which diverges at the first quote change after the first orders.

**Exercise 12.8 ★★★.**

*Find the flaw.* “Our parity test replays yesterday’s backtest inputs into the backtest and gets identical orders, so the strategy is ready for production.”

**Solution of Exercise 12.8.**

The test compares the backtest with itself: any input the backtest environment supplies wrongly (a clock, a price type, an event order, an acknowledgement rule) is supplied wrongly both times, and the orders agree. Parity must compare the backtest with production’s inputs (replay a production journal) and a production-shaped environment with the simulator on the same session.

## 12.11 Problem: The Morning the Timer Fired

**Problem 12.1.**

Weekend problem — four defects and a clean bill

The strategy of [Listing 12.4](#lst-pl-simulation-production-parity-quoter), three environments, one-hour sessions.

**Part I — The host.**

1. What may a hosted strategy read, and through what?
2. Why does the host buffer inputs by instant?
3. What does the production-shaped adapter add to the in-process one?
4. What does the input journal hold, and in which format?
5. What does the replay environment do with the strategy’s orders?

**Part II — The defects.**

6. How does the wall-clock defect show in the simulator and in the replay?
7. What does the float-price defect do to the wide bid?
8. When does the callback-order defect matter?
9. What does ack semantics change, and why the storm?
10. Which of the four would a replay of production’s journal into a correct host miss, and why?

**Part III — The report.**

11. Give the inputs before the first diverging output for each defect.
12. Give the outputs affected in the hour for each.
13. Which defect is the fastest to show, and which the slowest?
14. What unplanned difference did the harness find?
15. How many outputs and inputs do the ten clean sessions have, and how many differences?

**Part IV — The verdict.**

16. State the *named result* : the inputs until each defect’s first diverging order, the orders affected in an hour, and the parity after the fixes.
17. What would you require of every strategy before production?
18. What would you require of every [environment adapter](#def-pl-simulation-production-parity-host) ?
19. Why compare two environments on the same session as well as replaying a journal?
20. In one sentence: what makes parity a property of the platform rather than of each strategy?

**Solution of Problem 12.1.**

1. The host’s clock, market data and order entry, through its context and callbacks; nothing else.
2. So that inputs of one instant are delivered in the [event model](https://one-course.com/books/quant/15/en/chapter/11-backtest-engine-architecture#def-pl-backtest-engine-architecture-event) ’s order whatever order the environment produced them in.
3. Book 13’s gateway (state machine, throttle, exposure), encoding and framing on a byte stream, and the input journal.
4. Every input as delivered, with its receive time, in Book 13’s input-journal format: 8 bytes of time and a 48-byte record.
5. Records them without sending them anywhere: the replay’s outputs are compared, not executed.
6. In the simulator the timer never fires in the hour and the opening lasts fifty minutes; in the replay the journalled timer fires but the strategy reads its own machine clock and requotes wide, parting at 30.1 seconds.
7. $99.99 - 0.01$ becomes 999 799, off the tick grid; the venue rejects it.
8. When a fill and a quote change arrive in the same instant.
9. Whether a sent but unacknowledged order counts as working; the strategy re-sends at every quote change inside the round trip.
10. None of the four here: each also changed what the replay host delivered or read (its clock, its integer prices, its event order, its semantics); a defect shared by the backtest and production environments would be missed.
11. 175 (wall clock), 126 (float prices), 203 (callback order), 2 (ack semantics).
12. 539 of 553, 1 686 of 1 690, 387 of 463 and 8 449 of 8 452 outputs against the simulator.
13. Ack semantics at the second input; callback order the slowest, after 203.
14. Cancels of orders already finished: the gateway refused them, the in-process adapter sent them.
15. 8 887 outputs from 188 079 inputs over ten sessions, with no difference in either comparison.
16. **Named result.** The wall clock diverges after 175 inputs, float prices after 126, callback order after 203 and ack semantics after 2, affecting 539, 1 686, 387 and 8 449 outputs in the hour; with the fixes, production-shaped, simulator and replay runs agree on every output of ten sessions.
17. That it reads only the host’s clock and inputs, and passes the harness against production’s journal and the simulator on the same session.
18. Integer prices, delivery in the [event model](https://one-course.com/books/quant/15/en/chapter/11-backtest-engine-architecture#def-pl-backtest-engine-architecture-event) , and one set of order-entry semantics, tested by the harness.
19. Two environments on the same session catch what a replay shares with production; a replay of production’s journal catches what two simulators share.
20. Because the host fixes time, prices, order and semantics for every strategy, and the harness tests them once for all.

## 12.12 Interview questions

**Interview question 12.1 ★ developer.**

Your strategy behaves differently in production from the backtest on the same day. Where do you look first?

**Solution of Interview question 12.1.**

At the first order that differs, and at the inputs just before it, from a journal of production’s inputs replayed into the backtest: then at what the strategy reads that is not an input (clocks, randomness, iteration order), and at how the environments differ in types, event order and order-entry semantics.

*What the interviewer is looking for: Journal, first difference, then the usual suspects.*

**Interview question 12.2 ★★ developer.**

Why should a trading system represent prices as integers?

**Solution of Interview question 12.2.**

Because most decimal prices have no exact binary representation, so arithmetic and conversions produce values off the tick grid, and equality tests fail; integers in the smallest unit are exact, compare exactly and hash identically in every language.

*What the interviewer is looking for: Representation error, truncation, off-grid orders.*

**Interview question 12.3 ★★ developer, researcher.**

How would you make a strategy’s use of time testable?

**Solution of Interview question 12.3.**

Give it one injected clock — time of the current input and timers as inputs — and forbid every other time source; then journal the timer firings in production and replay them, and test with a harness whose machine clock differs from the host’s.

*What the interviewer is looking for: Injected clock, timers as inputs.*

**Interview question 12.4 ★★ developer.**

A quote update and a fill have the same timestamp. Which should the strategy see first, and why does it matter?

**Solution of Interview question 12.4.**

By a fixed rule the platform applies everywhere — the chapter’s delivers market data first, then order entry, then timers — because otherwise the same instant produces different decisions in different environments; which rule matters less than that there is one.

*What the interviewer is looking for: A total order, the same in every environment.*

**Interview question 12.5 ★★★ developer.**

Design a harness that proves a strategy behaves identically in simulation and production.

**Solution of Interview question 12.5.**

One host and one strategy interface; adapters for the backtest, the simulator and production; production’s inputs journalled; a harness that compares outputs event by event against the replay of the journal and against the simulator on the same session, reports the first difference and its inputs, and runs in continuous integration with planted defects that it must find.

*What the interviewer is looking for: Journal, two comparisons, first difference, planted defects as tests of the harness.*