Quantitative Finance · Book 16 · The firm

The Desk and the Firm

The Desk and the Firm · The firm

16Compliance and Market Abuse

In August 2022 the UK’s Financial Conduct Authority fined Citigroup Global Markets £12.6 million because, when the Market Abuse Regulation extended the duty to monitor orders as well as trades in 2016, the firm did not properly implement it: it took eighteen months to identify and assess the market abuse risks its business was exposed to, and in the meantime it could not effectively monitor its trading for certain types of insider dealing and market manipulation. Nobody was accused of abusing the market. The failure was significant gaps in the firm’s surveillance: for some kinds of abuse no alert would have been generated, so nobody could have read one. Compliance is a function that must see the trading before it can judge it.

16.1 The rulebook a trader lives under

A trader’s conduct is bounded by two prohibitions and the duties that support them. The prohibitions are on using information others do not have and on distorting prices; Book 9 described the manipulative patterns (spoofing, layering, marking the close, momentum ignition, benchmark manipulation) and Book 3 wash trading. This chapter is about the function that keeps a firm on the right side of them.

Definition 16.1 (Market abuse, inside information, insider dealing)

Market abuse is insider dealing, the unlawful disclosure of inside information, and market manipulation, including attempts. Inside information is information of a precise nature, not made public, relating directly or indirectly to an issuer or a financial instrument, which would be likely to have a significant effect on prices if it were made public. Insider dealing is using inside information to buy or sell, for oneself or another, the instruments to which it relates, including by cancelling or amending an order placed before the information was received.

As of September 2026 — The EU Market Abuse Regulation

Regulation (EU) No 596/2014 defines inside information (article 7) and insider dealing (article 8); prohibits insider dealing, recommending or inducing it and unlawfully disclosing inside information (article 14), and market manipulation (article 15); governs market soundings, whose records are kept for at least five years (article 11); requires any person professionally arranging or executing transactions to maintain effective arrangements, systems and procedures to detect and report suspicious orders and transactions, and to notify the competent authority without delay on reasonable suspicion (article 16(2)); and requires issuers and those acting for them to keep insider lists (article 18).

As of September 2026 — US rules on insider trading and personal dealing

Rule 10b-5 (17 CFR 240.10b-5) makes it unlawful to employ any device, scheme or artifice to defraud, or to engage in any act that operates as a fraud, in connection with the purchase or sale of any security; insider trading cases are brought under it. Section 15(g) of the Exchange Act (15 U.S.C. 78o(g)) requires every registered broker-dealer to establish, maintain and enforce written policies and procedures reasonably designed to prevent the misuse of material non-public information. Rule 204A-1 (17 CFR 275.204A-1) requires a registered investment adviser’s code of ethics to make its access persons report their holdings and, within 30 days after each quarter, their personal transactions, and to obtain approval before buying in an initial public offering or a limited offering.

16.2 Surveillance as a function

Book 15 built the trade surveillance system: detectors that score orders and trades, surveillance alerts, triage, and the communications archive. The compliance function owns what happens next: which detectors run on which businesses, where their thresholds sit, who reviews the alerts, and what is reported.

Definition 16.2 (Suspicious transaction and order report)

A suspicious transaction and order report (STOR) is the notification a firm that arranges or executes transactions must send its regulator, without delay, when it has a reasonable suspicion that an order or transaction could be insider dealing or market manipulation, attempted or completed.

The hook’s failure was coverage: detectors that were not run cannot alert. The subtler failure is capacity. Every detector has a threshold; below it nothing is reviewed, above it every alert waits for an analyst. Set it low and the queue floods, and analysts reviewing in the order alerts arrive see mostly noise; set it high and true cases never alert.

Proposition 16.3 (Filling the queue)

Let a detector score NN candidates, and let an alert rate qq send the qNqN highest-scored for review, of which TP(q)\mathrm{TP}(q) are true cases, non-decreasing in qq, with a precision TP(q)/(qN)\mathrm{TP}(q)/(qN) non-increasing in qq. If analysts review at most CC alerts, in arrival order, the expected number of true cases found is TP(q)\mathrm{TP}(q) for q≤C/Nq\le C/N and C TP(q)/(qN)C\,\mathrm{TP}(q)/(qN) above it, and is largest at q∗=C/Nq^*=C/N: the threshold whose alerts just fill the analysts’ capacity.

Proof. Below C/NC/N every alert is reviewed and the true cases found are TP(q)\mathrm{TP}(q), which does not fall as qq rises. Above it a share C/(qN)C/(qN) of alerts is reviewed, in an order unrelated to the score, so the expected true cases found are CC times the precision, which does not rise with qq. Both pieces equal TP(C/N)\mathrm{TP}(C/N) at q=C/Nq=C/N. ∎

def triage(score, label, alert_rate, capacity, order="arrival"):
    score, label = np.asarray(score, float), np.asarray(label)
    thr = np.quantile(score, 1 - alert_rate)
    alerted = np.nonzero(score > thr)[0]
    a, tp = len(alerted), int(label[alerted].sum())
    reviewed = min(a, capacity)
    if order == "score":
        top = alerted[np.argsort(-score[alerted], kind="stable")[:reviewed]]
        found = float(label[top].sum())
    else:
        found = tp * reviewed / a if a else 0.0
    return {"threshold": float(thr), "alerts": a, "true_alerts": tp, "reviewed": reviewed, "found": found,
            "missed": int(label.sum()) - found}


def best_rate(score, label, capacity, rates, order="arrival"):
    res = {r: triage(score, label, r, capacity, order)["found"] for r in rates}
    return max(res, key=res.get), res
Listing 16.1. The triage queue: alerts above the threshold for a chosen alert rate, reviewed up to capacity in arrival or score order, and the alert rate that finds the most true cases. code/firm/compliance/firm_compliance.py

16.3 Tutorial: the alert queue and the pre-clearance desk

Goal. Run a year of personal-trade pre-clearance, then send a surveillance alert stream through a triage queue with fixed analyst hours and find the threshold that catches the most true cases. End state: the pre-clearance decisions by reason and the queue chart (Figure 16.1).

  1. The alerts. Book 9’s synthetic spoofing data, firm.surveil.spoofing_days: 11 600 account-days, 100 of them planted abuse, scored by its best detector (large-order fill gap times cancellations after the other side fills).
  2. The analysts. Each alert takes half an hour; the team has 50 hours for this detector (100 alerts), or 150 (300 alerts).
  3. The queue. fm_compliance.queue(hours) sweeps the alert rate from 0.1% to 20% with firm.compliance.triage.
  4. The requests. fm_compliance.year() sends 1 215 personal-trade requests from 150 employees through firm.compliance.preclear against lists that change through the year.
True cases found by an alert queue reviewed in arrival order, against the alert rate, for two analyst budgets. Each curve peaks where the alerts just fill the capacity (); at the default rate of 5% the queue floods and most true alerts are never read. Data: fm_compliance.queue.
Figure 16.1. True cases found by an alert queue reviewed in arrival order, against the alert rate, for two analyst budgets. Each curve peaks where the alerts just fill the capacity (Proposition 16.3); at the default rate of 5% the queue floods and most true alerts are never read. Data: fm_compliance.queue.

With 50 analyst hours the best alert rate is 0.8%, just under the capacity rate of 100/11 600=0.86%100/11\,600=0.86\%, and the queue finds 65 of the 100 planted cases. At the default rate of 5% the detector alerts on 580 account-days, 89 of them true, but analysts reading the first 100 in arrival order find 15 on average: 85% of the abuse is missed, not by the detector, which flagged most of it, but by the queue (Figure 16.2). Tripling the hours moves the best rate to 2.5% and the true cases found to 81; at the default rate it finds 46. Reviewing in score order instead of arrival order recovers what the threshold loses: 66 found at the default rate with 50 hours, 81 with 150. A naive detector, the order-to-trade ratio, finds none of the 100 at any threshold, because the market makers of Book 9 cancel more than any spoofer.

The surveillance funnel at the default threshold: the detector finds most of the abuse and the queue loses it. Data: fm_compliance.queue.
Figure 16.2. The surveillance funnel at the default threshold: the detector finds most of the abuse and the queue loses it. Data: fm_compliance.queue.

16.4 Personal dealing

Employees who know what the firm is about to trade, or what a client is about to announce, can profit from it in their own accounts. The rules are older than algorithmic trading and apply to everyone with access.

Definition 16.4 (Personal account dealing, pre-clearance)

Personal account dealing is trading by employees for their own accounts or accounts they control or benefit from. Pre-clearance is the requirement that an employee obtain the compliance function’s approval before a personal trade, which is refused if the instrument is restricted, if the firm is trading or about to trade it, if a minimum holding period has not passed, or if the employee holds inside information about it.

Definition 16.5 (Restricted list, watch list)

A restricted list names the instruments in which the firm and its employees may not trade, or may trade only with approval, usually because the firm holds inside information about the issuer and says so publicly or to its staff. A watch list names, confidentially and for compliance only, the instruments about which some part of the firm holds inside information, so that trading in them anywhere in the firm can be monitored without revealing the information.

def preclear(req, lists, firm_trades, holdings, insiders=frozenset(), rules=None):
    """req: (employee, symbol, side, day), side +1 buy / -1 sell; firm_trades: {symbol: sorted days};
    holdings: {(employee, symbol): day bought}; insiders: set of (employee, symbol) on the insider list that day."""
    rules = rules or Rules()
    emp, sym, side, day = req
    if lists.on("restricted", sym, day):
        return False, "restricted list"
    if lists.on("watch", sym, day):
        return False, "watch list"
    if (emp, sym) in insiders:
        return False, "insider"
    if any(day - rules.blackout_days <= t <= day for t in firm_trades.get(sym, ())):
        return False, "firm traded recently"
    bought = holdings.get((emp, sym))
    if side < 0 and bought is not None and day - bought < rules.holding_days:
        return False, "holding period"
    return True, "approved"
Listing 16.2. Pre-clearance of a personal trade: the first rule that applies gives the reason for a refusal. code/firm/compliance/firm_compliance.py
A year of personal-trade requests by decision: 1 215 requests, 964 approved; four in five refusals are for names the firm traded in the two days before. Data: fm_compliance.reasons.
Figure 16.3. A year of personal-trade requests by decision: 1 215 requests, 964 approved; four in five refusals are for names the firm traded in the two days before. Data: fm_compliance.reasons.

Of the year’s 1 215 requests, 964 (79.3%) were approved. Of the 251 refusals, 202 (80.5%) were for names the firm had traded in the previous two days, the front-running rule; 23 were for restricted names, 5 for names on the watch list, 6 for sales within the 30-day holding period, and 15 were requests by wall-crossed employees in the names of their projects, every one of the planted requests refused. The watch-list refusals show the list’s cost: an employee refused for a reason compliance cannot give learns that something is happening in the name. A firm can reduce that signal by giving no reasons for any refusal, or by monitoring watch-list names instead of refusing trades in them.

16.5 Information barriers, insider lists and wall-crossing

A firm that advises issuers, lends to them or underwrites their securities holds inside information in one part and trades in another. It keeps them apart with barriers, and it records who crosses them.

Definition 16.6 (Information barrier, wall-crossing, insider list)

An information barrier is the set of organisational, physical and electronic controls that keep inside information held in one part of a firm from people in another who trade or advise on the instruments concerned. A wall-crossing is the controlled passing of inside information to a person on the other side of a barrier, or to an outside investor sounded about a transaction, with their prior consent, recorded, and binding them not to trade until the information is public or no longer inside. An insider list records every person with access to inside information about an issuer, when they obtained it, and why.

Information barriers: the private side holds inside information, the public side trades; compliance sits across both, keeps the lists, records wall-crossings and pre-clears and surveils the public side. Schematic.
Figure 16.4. Information barriers: the private side holds inside information, the public side trades; compliance sits across both, keeps the lists, records wall-crossings and pre-clears and surveils the public side. Schematic.

A crossed person is on the insider list for the project’s issuer from the moment they are crossed until the information is made public or loses its significance (the project is cleansed); pre-clearance refuses their trades in the name throughout, and surveillance watches the name across the firm. The records serve two purposes: they make the controls work, and they are what the firm shows its regulator when a trade in the name is questioned. A market sounding under the regulation’s rules is the same act performed on an outside investor, with the same consent and records.

Method 16.7 (Running the compliance function)

  1. Map the firm’s businesses to the abuse each could commit or facilitate, and each to a detector; check the coverage every time a business, product or venue is added.
  2. Set each detector’s threshold so that its alerts fill the analysts’ hours, and review in score order; measure the true-alert rate and adjust.
  3. Keep the lists point in time; pre-clear every personal trade; record every wall-crossing and cleansing.
  4. Report suspicious orders and transactions without delay, and record the reasons for every decision not to report.
  5. Test the whole chain once a year with planted cases, as the tutorial does.

16.6 Build: the compliance engine

Purpose. Restricted and watch lists through time, pre-clearance of personal trades, an insider-list register with wall-crossings, and the capacity of an alert queue on firm.surveil’s detectors.

Interface. firm.compliance: Lists, Rules, preclear, Register (cross, cleanse, insiders), triage, best_rate.

Rules. The first applicable rule decides a refusal and is recorded; lists and registers are dated; a cleansed project’s insiders leave the list on the cleansing day; triage never reviews more alerts than capacity.

Acceptance tests. code/firm/compliance/tests/: each refusal reason; the register through a project; the queue on a detector with known precision.

Stretch. Alerts from several detectors sharing one team; review times that depend on the alert; the regulator’s reporting deadline as a constraint on the queue.

Sources and further reading

  • Financial Conduct Authority, press release of 19 August 2022 on Citigroup Global Markets Limited.
  • Regulation (EU) No 596/2014, articles 7, 8, 11, 14–16 and 18.
  • 17 CFR 240.10b-5; 15 U.S.C. 78o(g); 17 CFR 275.204A-1.

16.7 Exercises

Exercise 16.1 ★

Which of the chapter’s refusal rules would stop a trader who wants to buy a stock the firm bought yesterday, and why does the rule exist?

Solution

Solution of Exercise 16.1.

The blackout rule: the firm traded the name within the two days before the request. It stops employees trading on knowledge of the firm’s own orders, ahead of them or alongside them.

Exercise 16.2 ★

A detector scores 11 600 account-days and analysts can review 100 alerts. What alert rate fills the queue?

Solution

Solution of Exercise 16.2.

100/11 600=0.86%100/11\,600=0.86\%.

Exercise 16.4 ★★

At the default 5% rate the detector flags 89 of the 100 cases. Why does the queue find only 15?

Solution

Solution of Exercise 16.4.

The 580 alerts exceed the capacity of 100, and analysts read them in arrival order: on average 17% of the 89 true alerts are read, 89×100/580=15.389\times100/580=15.3.

Exercise 16.5 ★★

Why does the order-to-trade ratio find none of the planted spoofing cases?

Solution

Solution of Exercise 16.5.

Legitimate market makers cancel almost all their orders, and more than spoofers do, so a high ratio ranks market makers first; the spoofers’ signature is the gap between their large and small orders’ fill rates and the timing of their cancellations.

Exercise 16.6 ★★

A research analyst is wall-crossed on a takeover on day 40 and the project is cleansed on day 75. On which days is a personal trade in the target refused, and why is surveillance of the whole firm’s trading in the name still needed?

Solution

Solution of Exercise 16.6.

Days 40 to 74: from the crossing until the cleansing on day 75. Pre-clearance only covers the crossed person’s own account; the information could reach the firm’s trading or other people, which only surveillance of all trading in the name would detect.

Exercise 16.7 ★★★

Coding. Run fm_compliance.queue with 150 hours, in arrival order and in score order. What do the best rates and the default rate find?

Solution

Solution of Exercise 16.7.

In arrival order the best rate is 2.5% and finds 81; the default 5% finds 46. In score order the default rate also finds 81: reviewing the highest scores first makes a low threshold harmless.

Exercise 16.8 ★★★

Find the flaw. “We lowered every threshold to catch more abuse; our surveillance is stronger than last year.”

Solution

Solution of Exercise 16.8.

More alerts with the same analysts means a smaller share read: past the capacity rate, lowering the threshold reduces the true cases found in arrival order (from 65 at 0.8% to 15 at 5% in the chapter). Strength is true cases found, not alerts generated.

16.8 Problem: The Alert Queue

Problem 16.1

Weekend problem — the alert queue

A new head of compliance inherits a surveillance team that clears its queue every quarter by closing old alerts unread, and a pre-clearance process run by email.

Part I — The rules.

  1. Define market abuse, inside information and insider dealing.
  2. What does the EU regulation require of firms that arrange or execute transactions?
  3. State the US rules on insider trading, broker-dealer policies and advisers’ codes of ethics.
  4. What failure did the FCA find at Citigroup Global Markets in 2022?

Part II — The queue.

  1. Define a STOR.
  2. State and prove Proposition 16.3.
  3. Give the best rates and true cases found for 50 and 150 analyst hours.
  4. Give what the default rate finds and misses with 50 hours.
  5. What does reviewing in score order change?
  6. Why is a detector’s recall not the function’s recall?

Part III — Personal dealing and barriers.

  1. Define personal account dealing, pre-clearance, the restricted list and the watch list.
  2. Give the year’s pre-clearance decisions by reason.
  3. Define an information barrier, a wall-crossing and an insider list.
  4. How long does a crossed person stay on the insider list, and what follows for their trades?

Part IV — The plan.

  1. What is wrong with closing old alerts unread?
  2. How would you set each detector’s threshold?
  3. How would you check coverage when the firm starts trading a new product?
  4. How would you test the whole chain?
  5. State the named result: the alert threshold that maximises true cases found for a fixed number of analyst hours, and the share missed at the default threshold.
  6. In two sentences, write the plan.
Solution

Solution of Problem 16.1.

  1. See Definition 16.1.
  2. Effective arrangements, systems and procedures to detect and report suspicious orders and transactions, and notification without delay on reasonable suspicion.
  3. See Box 16.2.
  4. It failed to implement the 2016 requirement properly, took eighteen months to assess its risks, and could not effectively monitor certain types of abuse: a £12.6 million fine.
  5. See Definition 16.2.
  6. See Proposition 16.3.
  7. 0.8% and 65 found with 50 hours; 2.5% and 81 with 150.
  8. 580 alerts, 89 true, 15 found and 85 missed.
  9. At the default rate it finds 66 with 50 hours and 81 with 150: the order, not the threshold, decides what is read.
  10. Alerts not read are not found: the function’s recall is the detector’s times the share of true alerts reviewed.
  11. See Definitions 16.4 and 16.5.
  12. 964 approved; refused: 202 firm traded recently, 23 restricted, 15 insiders, 6 holding period, 5 watch list.
  13. See Definition 16.6.
  14. From the crossing until the cleansing; every personal trade in the issuer’s instruments is refused meanwhile.
  15. An alert closed unread is a case not found, and a record that the firm chose not to look.
  16. So that each detector’s alerts fill its share of the analysts’ hours, reviewed in score order, and adjusted to the measured true-alert rate.
  17. Map the product to the abuse it allows, check that a detector covers it before the first trade, and test with planted cases.
  18. Plant known cases in the data each year and measure how many the whole chain, detectors and queue, finds.
  19. The alert rate whose alerts just fill the capacity, q∗=C/Nq^*=C/N (0.86%, best on the grid 0.8%, 65 found); at the default 5% the queue misses 85% of the cases.
  20. Size thresholds to the team and review alerts in score order, with coverage checked for every new business and tested with planted cases; pre-clear every personal trade against dated lists and record every wall-crossing.

16.9 Interview questions

Interview question 16.1 ★ trader

What is inside information, and what must you do if you receive some by mistake?

Solution

Solution of Interview question 16.1.

Precise, non-public, price-sensitive information about an issuer or instrument; do not trade or pass it on, and tell compliance at once, who will restrict or watch the name and record you as an insider.

What the interviewer is looking for: the definition and escalation to compliance.

Interview question 16.2 ★ trader, researcher

Why does your firm make you pre-clear personal trades?

Solution

Solution of Interview question 16.2.

To stop trading on the firm’s orders or on inside information held elsewhere in the firm, and to leave a record that the firm checked.

What the interviewer is looking for: front-running, inside information and the record.

Interview question 16.3 ★★ developer

Design the data model for a restricted list that can answer “was this name restricted at 10:32 on 3 March?”

Solution

Solution of Interview question 16.3.

Entries with the instrument, list, start and end timestamps and the reason, append-only, with corrections as new entries; queries as of a time.

What the interviewer is looking for: bitemporal, append-only records.

Interview question 16.4 ★★ researcher

A surveillance detector has 90% recall at a 5% alert rate. Is it good?

Solution

Solution of Interview question 16.4.

It depends on capacity: at 5% of a large population the alerts may far exceed what analysts can read, and the cases found are recall times the share read. Ask for the precision and the queue.

What the interviewer is looking for: recall is not the function’s result.

Interview question 16.5 ★★ risk

What is a wall-crossing, and what records should it leave?

Solution

Solution of Interview question 16.5.

The controlled passing of inside information across a barrier with consent; the record of who, when, what, the consent, the insider-list entry, and the cleansing date.

What the interviewer is looking for: consent, records and cleansing.

Interview question 16.6 ★★★ researcher, developer

Several detectors share one team of analysts. How would you allocate their hours?

Solution

Solution of Interview question 16.6.

Give hours where the next hour finds the most true cases: equalise the precision of the marginal alert across detectors, reviewing each in score order.

What the interviewer is looking for: marginal precision equalised.

Terms defined in this chapter

See all 2333 terms in the glossary