16Signal Serving and Parameter Management

A risk parameter entered as 50 meant fifty basis points to the person who typed it and fifty percent to the strategy that read it. Nothing checked the unit; the change went to every strategy at once; the first sign was the fill rate. A parameter is code that nobody compiles: it changes behaviour as much as a release does, it is edited by more people, and it usually has no types, no tests and no history. This chapter gives parameters and research signals the machinery code already has — types, review, history, gradual rollout — and measures, on ten simulated strategies, what each piece of that machinery saves when fifty means the wrong thing.

16.1 From research signal to production input

A research signal becomes a production input when a strategy reads it on a schedule and trades on it. Between the two sit the questions a research notebook never asks: which version is live, how old the value may be before it is not to be used, what the strategy does when it is missing, and who changed it. Book 12 (chapters 24 and 25) answered them for machine-learning features and models — the feature store, feature freshness, the model registry and its audit trail. This chapter applies the same answers to the plainer inputs every strategy has: signals computed by research jobs and parameters typed by people.

16.2 The signal service

The fallback is part of the signal’s contract, chosen when the signal is designed, not by the strategy’s author at three in the morning: for a directional signal, usually zero (no view); for a risk input, usually the most conservative value.

class SignalService:
    """Versioned values; a reader gets the latest if fresh, else the declared fallback."""

    def __init__(self, max_age: float, fallback: float):
        self.max_age, self.fallback = max_age, fallback
        self.values: dict = {}                 # name -> [(ts, version, value)]

    def publish(self, name: str, value: float, version: str, ts: float) -> None:
        self.values.setdefault(name, []).append((ts, version, value))

    def get(self, name: str, now: float):
        vals = [v for v in self.values.get(name, []) if v[0] <= now]
        if not vals:
            return self.fallback, None, "missing-fallback"
        ts, version, value = max(vals)
        if now - ts > self.max_age:
            return self.fallback, version, "stale-fallback"
        return value, version, "fresh"

With a freshness bound of five minutes and a fallback of zero, a momentum signal published at time 0 and again at 60 seconds is served as 0.7 (version 3, fresh) at 200 seconds and as 0 with the status “stale-fallback” at 600 seconds; a signal never published is served as the fallback with the status “missing-fallback”. The strategy knows which it got.

16.3 Configuration as data

A value’s two times are the valid time and knowledge time of Book 7 (chapter 3): the time from which it applies and the time the store learned it. They let the store answer the question every post-mortem asks — what was live at 10:31 last Tuesday — in both of its senses: as the firm knew it then, and as it is known now that a correction has been recorded.

16.4 Schemas, approvals and history

The chapter’s parameter is an inventory limit, as a share of the strategy’s capital: canonical unit a fraction, entered in basis points, percent or as a fraction, bounded by 2%, changed by at most 50 basis points at a time, and needing two approvers for any change above 20 basis points — the four-eyes principle of One Quant Book 16 (chapter 12) applied to a number.

    def propose(self, strategy, name, value, unit, effective, recorded, author) -> Change:
        s = self.schemas[name]
        c = Change(len(self.changes), strategy, name, math.nan, f"{value} {unit}",
                   effective, recorded, author)
        self.changes.append(c)
        if unit not in s.units:
            c.reasons.append(f"unit {unit!r} not accepted for {name} "
                             f"(accepted: {', '.join(s.units)})")
        else:
            c.value = float(value) * s.units[unit]
            if not s.lo <= c.value <= s.hi:
                c.reasons.append(f"{c.value:g} {s.unit} outside [{s.lo:g}, {s.hi:g}]")
            old = self.as_of(strategy, name, recorded, recorded)
            step = abs(c.value - old) if old is not None else 0.0
            if step > s.max_step:
                c.reasons.append(f"step {step:g} {s.unit} above the allowed {s.max_step:g}")
            c.needed = 2 if step > s.two_approvals_above else 1
        c.status = "refused" if c.reasons else "pending"
        body = {"cid": c.cid, "strategy": strategy, "name": name, "entered": c.entered,
                "status": c.status, "reasons": c.reasons}
        self.audit.append("propose", body, recorded)
        return c

The fifty of the hook, entered without a unit, is refused: a value with no unit is not accepted. Entered as “50 percent”, it is refused twice over: 0.5 is outside $[0, 0.02]$, and a step of 0.495 exceeds the allowed 0.005. Entered as “50 bp” it is accepted as 0.005. A raise to 75 basis points (a step of 25) waits after one approval, cannot be approved by its author, and applies after the second. Every proposal, approval and application is a line of Book 12’s hash-chained audit trail (firm.exptrack): editing any line breaks the chain, and verify says where.

    def approve(self, cid: int, approver: str, ts: float) -> Change:
        c = self.changes[cid]
        if c.status != "pending" or approver == c.author or approver in c.approvals:
            self.audit.append("approve-refused", {"cid": cid, "approver": approver}, ts)
            return c
        c.approvals.append(approver)
        self.audit.append("approve", {"cid": cid, "approver": approver}, ts)
        if len(c.approvals) >= c.needed:
            c.status = "applied"
            rec = max(ts, c.recorded)
            row = (c.effective, rec, c.value, cid)
            self.rows.setdefault((c.strategy, c.name), []).append(row)
            body = {"cid": cid, "value": c.value, "effective": c.effective}
            self.audit.append("apply", body, rec)
        return c

    def as_of(self, strategy, name, valid, known=None):
        """The value in force at `valid`, as the store knew it at `known` (default: now)."""
        rows = self.rows.get((strategy, name), [])
        rows = [r for r in rows if r[0] <= valid and (known is None or r[1] <= known)]
        if not rows:
            return None
        return max(rows, key=lambda r: (r[0], r[1]))[2]

16.5 Rolling out a change

Flags are how the canary deployment and staged rollout of Book 7 (chapter 21) apply to parameters: the new value is behind a flag switched on for one strategy, watched, and extended. Drift is what a flag, left in place, becomes: the chapter’s drift compares, strategy by strategy, the declared value with the one each strategy reports it is running, and on the demonstration finds the one strategy running 0.5 where the store says 0.005. The best-known case of both is Knight Capital’s: on 1 August 2012 a flag once used for an old function was repurposed for new code, the new code had not reached one of eight servers, and orders carrying the flag triggered the old code there; the loss exceeded $460 million (Book 7, chapter 21, and Book 13, chapter 22, tell it in full).

16.6 Fifty of what?

The chapter’s incident runs ten quoting strategies (chapter 11’s quoter, one lot a side) for an hour each on Book 10’s simulator, each on its own firm.tape session, with $4 million of capital each. The intended limit, 50 basis points of capital, is two lots at $100; read as 50 percent it is two hundred. The change arrives ten minutes into the session. A monitor compares each strategy’s fills in every five-minute window with its usual count for that window — here, the same session without the change, a better baseline than any real monitor has — and fires above 1.3 times; a firing rolls the parameter back. Exposure is the inventory beyond what the intended limit allows (three lots, with one order pending), in dollars.

def detection(base: dict, wrong: dict, start: float) -> float | None:
    """End of the first window after `start` whose fills exceed RATIO times the usual count."""
    b, w = windows(base), windows(wrong)
    for k in range(len(w)):
        if k * WINDOW >= start and w[k] > RATIO * max(b[k], 1):
            return (k + 1) * WINDOW
    return None

Table 16.1 is the result. All at once is detected soonest — ten monitors watch instead of one, and the first fires at five minutes — and costs most: $410 000 of excess inventory at the peak and 1.35 million dollar-minutes. The canary’s own monitor takes fifteen minutes to fire (its session is quiet), but only the canary holds the wrong value: $30 000 at the peak and 53 000 dollar-minutes, 25 times less. The schema check costs nothing, because the error never becomes a value. The strategies differ widely in how soon their fill rate shows the error: five of the ten after five minutes, three after fifteen, one after thirty and one after fifty.

16.7 Tutorial: a units error, three ways

16.8 Build: the parameter store

16.9 Exercises

16.10 Problem: Fifty of What?

16.11 Interview questions