Quantitative Finance · Book 10 · Execution

Microstructure and Execution

Microstructure and Execution · Execution

13Metaorders, Latent Liquidity and Cross-Impact

Why should impact grow like the square root of size, whatever the stock, the decade or the market? The answer most models now give is that the book a trader sees is a small part of the liquidity that exists: the rest is latent, waiting to be revealed at prices a large order has not yet reached. This chapter builds that latent book, recovers the square root and the linear regime below it, checks when a metaorder is priced fairly, and turns to the impact that trading one stock has on another.

13.1 Why impact is concave

Chapter 11 measured a concave impact and chapter 12 a transient one. Three explanations of the concavity are in use. Splitting: large metaorders are cut into children whose impacts decay between them. Information: Farmer, Gerig, Lillo and Waelbroeck (2013) derived the shape of impact from competition among execution services and a condition of fair pricing. Liquidity: Toth and co-authors (2011, chapter 11) argued that the average supply and demand vanish at the price and grow linearly away from it. The third has become a model.

13.2 Latent liquidity and the locally linear book

Definition 13.1 (Latent liquidity, latent order book)

Latent liquidity is the intention to trade at a price that has not been placed in the book, because its owner waits for the price to come near. The latent order book is the density of these intentions by price, of which the visible book is the small part closest to the price.

Definition 13.2 (Locally linear order book)

A locally linear order book has a latent density that vanishes at the price and grows linearly away from it, φ(x)=L ∣x−p∣\varphi(x)=L\,|x-p|, with LL the liquidity per unit of price squared.

Donier, Bonart, Mastromatteo and Bouchaud (2015) derived it from the reaction–diffusion of latent orders: buyers’ and sellers’ intentions diffuse as their valuations change, and annihilate (trade) where they meet. The signed density φ\varphi (asks positive) then obeys a diffusion equation whose stationary solution is linear. A buy metaorder at rate mm removes asks at the price, and the price, the zero of φ\varphi, moves up. If the metaorder is fast compared with diffusion, it simply eats the linear book: moving the price by II consumes ∫0IL x dx=LI2/2\int_0^IL\,x\,dx=LI^2/2, so I=2Q/LI=\sqrt{2Q/L}. If it is slow, the book refills by diffusion as it trades, and impact is linear in size. Bucci, Benzaquen, Lillo and Bouchaud (2019) documented the crossover.

firm.crossimpact.llob integrates the diffusion on a grid and removes the metaorder’s shares from the asks nearest the price at each step.

def llob(q: float, horizon: float, L: float = 1.0, D: float = 1.0, dt: float = 0.002,
         width: float = 40.0, dx: float = 0.1, after: float = 0.0):
    x = np.arange(-width, width + dx / 2, dx)
    phi = L * x
    rate = q / horizon
    times, price = [0.0], [0.0]
    steps = int(round((horizon + after) / dt))
    for k in range(steps):
        t = (k + 1) * dt
        lap = np.zeros_like(phi)
        lap[1:-1] = (phi[2:] - 2 * phi[1:-1] + phi[:-2]) / dx**2
        phi = phi + D * dt * lap
        phi[0], phi[-1] = L * x[0], L * x[-1]          # far ends stay on the linear profile
        if t <= horizon + 1e-12:
            need = rate * dt                             # shares taken from the nearest asks
            i = int(np.flatnonzero(phi > 0)[0])
            while need > 0 and i < len(phi):
                take = min(need, phi[i] * dx)
                phi[i] -= take / dx
                need -= take
                i += 1
        times.append(t)
        price.append(_zero(x, phi))
    return np.array(times), np.array(price)
Listing 13.1. The latent order book: the signed density diffuses, the metaorder removes the nearest asks, and the price is the zero of the density. code/firm/crossimpact/firm_crossimpact.py

With L=1L=1, D=1D=1 and a ten-second execution, the peak impact is 0.05 ticks for Q=0.3Q=0.3, 0.18 for 1, 0.52 for 3, 1.74 for 10, 4.9 for 30, 12.2 for 100, 23.3 for 300 and 44.0 for 1 000 (Figure 13.1). The local exponent is 1.0 up to Q=10Q=10, then 0.94, 0.76, 0.59 and 0.53: linear below LDT=10LDT=10, a square root above, where the impact approaches 2Q/L\sqrt{2Q/L} (24.5 for 300, 44.7 for 1 000).

Peak impact of a ten-second metaorder on a locally linear latent book (L=1, D=1): linear in size when the book refills faster than the order eats it, a square root when it does not. Data: mx_cross.llob_scan.
Figure 13.1. Peak impact of a ten-second metaorder on a locally linear latent book (L=1L=1, D=1D=1): linear in size when the book refills faster than the order eats it, a square root when it does not. Data: mx_cross.llob_scan.

13.3 Fair pricing and the end of a metaorder

Definition 13.3 (Fair pricing condition)

The fair pricing condition (Farmer, Gerig, Lillo and Waelbroeck, 2013) states that the average price paid for a metaorder equals the price after it is completed: whoever filled it was not paid for being picked off, and the metaorder’s owner paid for what its trading revealed.

If impact grows like the square root of the time traded, the average impact paid over the execution is two thirds of the peak, so fair pricing predicts that the price settles at two thirds of its peak once the metaorder ends. On the latent book, a metaorder of 10 reaches 1.74 ticks and pays 1.16 on average, 0.66 of the peak (Figure 13.2); after the end the price falls through the average paid 1.9 seconds later, and keeps falling, to 0.16 of the peak after 100 seconds. For a metaorder of 300 the average is again 0.66 of the peak, crossed 2.3 seconds after the end, with 0.27 left after 100 seconds. The model without memory loss makes impact transient, so the price is fair only for an instant; in the full model, finite memory of latent orders leaves a permanent part.

A metaorder of 10 on the latent book: the impact during the execution (shaded) and after it, against the average price paid. The price crosses the average two seconds after the end and keeps falling. Data: mx_cross.fair.
Figure 13.2. A metaorder of 10 on the latent book: the impact during the execution (shaded) and after it, against the average price paid. The price crosses the average two seconds after the end and keeps falling. Data: mx_cross.fair.

13.4 Cross-impact

Definition 13.4 (Cross-impact, cross-impact matrix)

Cross-impact is the effect of trading one asset on the price of another. The cross-impact matrix Λ\Lambda gives each asset’s price change per unit of each asset’s order flow, rt=Λqt+noiser_t=\Lambda q_t+\text{noise}.

Pasquariello and Vega (2015) found in US stocks that one industry’s or stock’s daily order imbalance moves other industries’ and stocks’ returns, persistently and often negatively. Benzaquen, Mastromatteo, Eisler and Bouchaud (2017) built a multivariate propagator in which liquidity shares the sectorial structure of correlations, and found that trades mediate a significant part of the covariance of returns.

Two simulated stocks with order flows correlated 0.6 (thousands of shares per interval) and a true matrix Λ=(1.00.40.40.8)\Lambda=\begin{pmatrix}1.0&0.4\\0.4&0.8\end{pmatrix} ticks per thousand shares, with unit noise, give over 2 000 intervals the estimate (0.960.420.420.81)\begin{pmatrix}0.96&0.42\\0.42&0.81\end{pmatrix}. Beyond its own flow, the other stock’s flow explains 4.5% of the first stock’s return variance and 5.3% of the second’s: correlated flows already carry most of the common move, and cross-impact adds the rest.

13.5 Arbitrage-free cross-impact

Schneider and Lillo (2019) extended the no-dynamic-arbitrage argument to several assets: with bounded decay kernels, cross-impact must be linear and odd in trading intensity and symmetric, the impact of ii on jj equal to that of jj on ii; otherwise a trader could cycle through the assets at a profit. An estimated matrix is not exactly symmetric; symmetric_psd projects it on the nearest symmetric positive semi-definite matrix before any optimiser uses it.

What does cross-impact change for execution? Selling 10 (thousand shares) of each stock over one unit of time with transient impact decaying at rate 3, the cheapest joint schedule costs 52.1 (Figure 13.3). An own-impact model, which ignores the cross terms, estimates the same schedule at 36.0, 31% too low. For a pair trade, selling 10 of the first and buying 10 of the second, the joint optimum costs 20.0 and the own-impact estimate is 36.0, 80% too high: each leg’s cross-impact helps the other. When cross-impact decays like own impact, the optimal schedules are the same with or without it; what the model gets wrong is the cost. Schedules differ when the legs are traded one after the other, as a desk that thinks the legs independent may do: then the joint schedule saves 19% of the sequential cost for the sale and 48% for the pair trade. When cross-impact decays more slowly than own impact (0.3 against 3), the joint optimum also changes shape, and saves a further 3.7% on the pair trade.

def liquidation(lam, rho: float, times, x_total, joint: bool = True,
                rho_cross: float | None = None):
    """Own impact decays at rho, cross-impact at rho_cross (default rho)."""
    lam = np.asarray(lam, float)
    t = np.asarray(times, float)
    dist = np.abs(t[:, None] - t[None, :])
    e = np.exp(-rho * dist)
    ec = np.exp(-(rho if rho_cross is None else rho_cross) * dist)
    n, k = len(t), len(lam)
    diag = np.diag(np.diag(lam))
    gamma = np.kron(diag, e) + np.kron(lam - diag, ec)       # asset-major blocks
    if joint:
        a = np.kron(np.eye(k), np.ones((1, n)))              # sum of each asset's trades
        m = np.block([[gamma, a.T], [a, np.zeros((k, k))]])
        sol = np.linalg.solve(m, np.concatenate([np.zeros(n * k), x_total]))
        q = sol[: n * k]
    else:
        w = np.linalg.solve(e, np.ones(n))
        q = np.concatenate([w * xi / w.sum() for xi in x_total])
    return q.reshape(k, n), float(0.5 * q @ gamma @ q)
Listing 13.2. Joint liquidation under transient own- and cross-impact, or asset by asset ignoring the cross terms. code/firm/crossimpact/firm_crossimpact.py
Selling two stocks, or trading them as a pair, under cross-impact: the cost of the joint optimum, what an own-impact model estimates for it, and the cost of trading the legs in turn. Data: mx_cross.cross_study.
Figure 13.3. Selling two stocks, or trading them as a pair, under cross-impact: the cost of the joint optimum, what an own-impact model estimates for it, and the cost of trading the legs in turn. Data: mx_cross.cross_study.

13.6 Tutorial: selling the stock, moving its neighbour

Goal. Recover the square root from a latent book, check fair pricing, and estimate and use cross-impact. End state: Figures 13.1, 13.2 and 13.3.

  1. Latent book. firm_crossimpact.llob(q, horizon, L, D, dt, width, dx, after); mx_cross.llob_scan() over eight sizes.
  2. Fair pricing. fair_pricing(times, price, horizon) and mx_cross.fair(q).
  3. Cross-impact. two_assets(), estimate(r, q), symmetric_psd, explained_by_other.
  4. Liquidation. liquidation(Lambda, rho, times, X, joint, rho_cross) for a sale and a pair trade; draw with fig_cross.py.

What to change next. Add cancellation and deposition of latent orders (finite memory) and look for a permanent impact; estimate cross-impact on three assets and decompose it on the eigenvectors of the flows’ covariance.

13.7 Build: latent liquidity and cross-impact

Purpose. The square-root law’s mechanism and the multi-asset impact that portfolio executions (chapters 15 and 22) and Book 11’s multi-asset market making need.

Interface. llob(q, horizon, L, D, dt, width, dx, after), fair_pricing(times, price, horizon), estimate(r, q), symmetric_psd(Lambda), explained_by_other(r, q), liquidation(Lambda, rho, times, X, joint, rho_cross).

Rules. Explicit diffusion (stable for D dt/dx2≤12D\,dt/dx^2\le\tfrac12) with fixed linear far ends; the price is the interpolated zero of the density; cross-impact by least squares of returns on flows; symmetric projection by eigenvalue clipping.

Acceptance tests. code/firm/crossimpact/tests/: a fast metaorder moves the price by about 2Q/L\sqrt{2Q/L}; a planted matrix recovered; the projection symmetric and positive; the joint liquidation no dearer than asset by asset.

Stretch. Latent orders with deposition and cancellation; the eigenliquidity decomposition; nonlinear cross-impact.

Sources and further reading

  • J. D. Farmer, A. Gerig, F. Lillo and H. Waelbroeck, “How efficiency shapes market impact”, Quantitative Finance 13(11), 2013.
  • J. Donier, J. Bonart, I. Mastromatteo and J.-P. Bouchaud, “A fully consistent, minimal model for non-linear market impact”, Quantitative Finance 15(7), 2015.
  • P. Pasquariello and C. Vega, “Strategic cross-trading in the U.S. stock market”, Review of Finance 19(1), 2015.
  • M. Benzaquen, I. Mastromatteo, Z. Eisler and J.-P. Bouchaud, “Dissecting cross-impact on stock markets: an empirical analysis”, Journal of Statistical Mechanics, 2017.
  • M. Schneider and F. Lillo, “Cross-impact and no-dynamic-arbitrage”, Quantitative Finance 19(1), 2019.
  • F. Bucci, M. Benzaquen, F. Lillo and J.-P. Bouchaud, “Crossover from linear to square-root market impact”, Physical Review Letters 122, 2019.

13.8 Exercises

Exercise 13.1 ★

A locally linear latent book has L=2L=2 (thousand shares per tick squared). How far does a fast metaorder of 50 thousand shares move the price?

Solution

Solution of Exercise 13.1.

2×50/2=50=7.07\sqrt{2\times50/2}=\sqrt{50}=7.07 ticks.

Exercise 13.2 ★

Why is impact linear in size when the metaorder is slow compared with the latent book’s diffusion?

Solution

Solution of Exercise 13.2.

The latent orders diffuse back toward the price faster than the metaorder removes them, so it always meets a nearly full book: the price moves in proportion to what it has taken, like a constant-depth book.

Exercise 13.3 ★

Show that if impact grows like t\sqrt t during an execution at a constant rate, the average impact paid is two thirds of the peak.

Solution

Solution of Exercise 13.3.

With I(t)=ITt/TI(t)=I_T\sqrt{t/T} and constant rate, the average is 1T∫0TITt/T dt=23IT\frac1T\int_0^TI_T\sqrt{t/T}\,dt=\tfrac23I_T.

Exercise 13.4 ★★

The estimated cross-impact matrix is (1.00.5−0.1−0.2)\begin{pmatrix}1.0&0.5\\-0.1&-0.2\end{pmatrix}. What is wrong with it, and what does the projection do?

Solution

Solution of Exercise 13.4.

It is not symmetric (0.5 against −0.1-0.1) and has a negative diagonal entry, so a trader could cycle through the two assets at a profit. The projection symmetrises it and clips its negative eigenvalues to zero, giving the nearest symmetric positive semi-definite matrix.

Exercise 13.5 ★★

Why does an own-impact model underestimate the cost of selling two correlated stocks and overestimate that of a pair trade?

Solution

Solution of Exercise 13.5.

Selling both, each sale’s cross-impact lowers the other’s price further, a cost the own-impact model leaves out (36.0 against 52.1). In the pair trade, selling the first lowers the second’s price while it is being bought, a gain the model leaves out (36.0 against 20.0).

Exercise 13.6 ★★

Why is the joint optimal schedule the same with and without cross-impact when both decay at the same rate?

Solution

Solution of Exercise 13.6.

The cost matrix is then a Kronecker product Λ⊗E\Lambda\otimes E, whose inverse is Λ−1⊗E−1\Lambda^{-1}\otimes E^{-1}: the optimum is (Λ−1μ)⊗E−11(\Lambda^{-1}\mu)\otimes E^{-1}\mathbf 1, the same time shape E−11E^{-1}\mathbf 1 for every asset, scaled to each asset’s total. Only the cost depends on the cross terms.

Exercise 13.7 ★★★

Coding. Rerun the latent-book scan with a horizon of 100 seconds instead of 10. Where does the crossover move, and why?

Solution

Solution of Exercise 13.7.

With a 100-second execution the exponent stays at 1.0 up to Q=100Q=100 and falls to 0.94 and 0.75 only for 300 and 1 000: the crossover moves up tenfold, to about LDT=100LDT=100, because the book has ten times longer to refill.

Exercise 13.8 ★★★

Find the flaw. “The price fell back below our average fill two seconds after we finished, so the market makers who filled us lost money.”

Solution

Solution of Exercise 13.8.

The price falling below the average fill after the end is what fair pricing with a transient impact predicts, not proof of anyone’s loss; the price after the end keeps moving (in the latent book it keeps falling). Who lost depends on how long the fillers held and where they closed; measure their mark-outs at their own horizons.

13.9 Problem: Selling the Stock, Moving Its Neighbour

Problem 13.1

Weekend problem — selling the stock, moving its neighbour

A portfolio manager must sell two correlated stocks and later run a pair trade in them. Explain the impact and plan the executions.

Part I — Latent liquidity.

  1. Give the three explanations of concave impact.
  2. Define the latent and the locally linear book.
  3. Derive 2Q/L\sqrt{2Q/L} for a fast metaorder.
  4. Give the simulated peak impacts and local exponents, and where the crossover is.

Part II — Fair pricing.

  1. State the fair pricing condition.
  2. Solve exercise 3.
  3. When does the simulated price cross the average paid, and what happens next?
  4. Why is impact transient in this model, and what would make part of it permanent?

Part III — Cross-impact.

  1. Define the cross-impact matrix.
  2. What did Pasquariello and Vega, and Benzaquen and co-authors, find?
  3. Give the estimated matrix and the explained shares.
  4. State Schneider and Lillo’s conditions and why they are needed.

Part IV — Execution and the verdict.

  1. Give the joint cost and the own-impact estimate for the sale.
  2. And for the pair trade.
  3. What does trading the legs one after the other cost?
  4. When does cross-impact change the optimal schedule itself?
  5. State the named result: the share of each asset’s price move explained by the other asset’s order flow, and the cost saved by a cross-impact-aware liquidation.
  6. How should the manager execute the sale?
  7. And the pair trade?
  8. In one sentence: why does the neighbour matter?
Solution

Solution of Problem 13.1.

1. Splitting with decaying impact, information and fair pricing, and a latent book that vanishes at the price. 2. Intentions to trade not placed in the book; a latent density L∣x−p∣L|x-p|. 3. ∫0ILx dx=LI2/2=Q\int_0^ILx\,dx=LI^2/2=Q. 4. 0.05, 0.18, 0.52, 1.74, 4.9, 12.2, 23.3 and 44.0 ticks; exponent 1.0 up to Q=10Q=10, then 0.94, 0.76, 0.59, 0.53: the crossover near LDT=10LDT=10. 5. The average price paid equals the price after completion. 6. Two thirds of the peak. 7. 1.9 seconds after the end for Q=10Q=10 (2.3 for 300); it keeps falling, to 0.16 (0.27) of the peak after 100 seconds. 8. The latent orders never lose memory in this version: the book refills around the old price; deposition and cancellation of latent orders would anchor part of the move. 9. rt=Λqt+noiser_t=\Lambda q_t+\text{noise}. 10. Imbalance in one stock or industry moves others’ returns, persistently, often negatively; trades mediate much of the covariance of returns, with liquidity sharing the correlations’ sectorial structure. 11. (0.960.420.420.81)\begin{pmatrix}0.96&0.42\\0.42&0.81\end{pmatrix}; 4.5% and 5.3% of return variance beyond each stock’s own flow. 12. Linear, odd and symmetric cross-impact, for bounded kernels, or cycles through the assets would pay. 13. 52.1 against 36.0. 14. 20.0 against 36.0. 15. 64.6 for the sale and 38.4 for the pair: the joint schedule saves 19% and 48%. 16. When cross-impact decays at a different rate from own impact; with a slower cross decay the pair trade saves a further 3.7%. 17. Named result: the other stock’s flow explains 4.5% and 5.3% of each stock’s return variance beyond its own; trading the legs together rather than in turn saves 19% of the cost of the sale and 48% of the pair trade, and an own-impact model misstates the cost by −31%-31\% and +80%+80\%. 18. Both stocks together over the whole horizon, with a cost estimate that includes cross-impact. 19. Both legs simultaneously, so that each leg’s cross-impact helps the other. 20. Its price moves with our trades in the stock we are selling, and so do our costs and our marks on it.

13.10 Interview questions

Interview question 13.1 ★ researcher

Explain the square-root law with the latent order book in three sentences.

Solution

Solution of Interview question 13.1.

Most liquidity is latent, and its density grows linearly away from the price. Moving the price by II consumes LI2/2LI^2/2 of it. So a metaorder of QQ moves the price by 2Q/L\sqrt{2Q/L} when the book cannot refill in time.

What the interviewer is looking for: Latent book; the integral; the fast regime.

Interview question 13.2 ★★ researcher

What is the fair pricing condition, and what does it predict for the price after a metaorder?

Solution

Solution of Interview question 13.2.

The average price paid equals the price after completion; with square-root impact the price should settle near two thirds of the peak.

What the interviewer is looking for: The condition; the two-thirds consequence.

Interview question 13.3 ★★ trader

You sell a large position in a stock. Which other positions of yours does it hurt, and how would you know?

Solution

Solution of Interview question 13.3.

Positions in correlated names, through cross-impact on their prices: marks and later sales suffer. Estimate cross-impact on my own trades or on market order flow, by regressing returns of each name on the flows of the others.

What the interviewer is looking for: Cross-impact; estimation on flows.

Interview question 13.4 ★★ researcher

How would you estimate a cross-impact matrix, and what constraints would you impose?

Solution

Solution of Interview question 13.4.

Regress interval returns on the vector of order-flow imbalances, with enough intervals and names grouped by sector; impose symmetry and positive semi-definiteness (project the estimate), and consider an eigen-decomposition of the flows’ covariance to reduce noise.

What the interviewer is looking for: Regression on flows; symmetry and PSD; dimension reduction.

Interview question 13.5 ★★ developer

Simulate a diffusing density on a grid with a moving boundary. What limits the time step?

Solution

Solution of Interview question 13.5.

An explicit scheme needs D dt/dx2≤12D\,dt/dx^2\le\tfrac12 for stability; the boundary (the price) is found at each step as the zero of the density, interpolated between cells; an implicit scheme removes the step limit at the cost of a linear solve.

What the interviewer is looking for: The stability condition; boundary tracking.

Interview question 13.6 ★★★ trader, researcher

Why might a pair trade cost less than the sum of its legs, and how would you exploit that in execution?

Solution

Solution of Interview question 13.6.

With positive cross-impact, selling one leg lowers the other’s price while buying it: the legs’ cross terms offset. Trade them simultaneously and balanced, and estimate the cost with the cross terms included.

What the interviewer is looking for: Offsetting cross-impact; simultaneous execution.

Terms defined in this chapter

See all 2333 terms in the glossary