পরিমাণগত অর্থায়ন · বই 19 · ক্যারিয়ার

Interviews II: Markets, Trading Games and the Final Round

এই বইটি শুধু ইংরেজিতে পাওয়া যায়।

Interviews II: Markets, Trading Games and the Final Round · ক্যারিয়ার

18Research and Strategy Cases

Signal design, backtest critique, costs and capacity, sizing and ramp-up, what live trading does to a simulation, take-home data cases, and how to size a desk or a venture. Every number is invented, and each one is meant to be reused: replace it with your own and the method still works.

Question 18.1 ★ QR, QT, Analyst • any • junior

A strategy makes 8 basis points a day on average, with a daily standard deviation of 1.2%. What is its annualised Sharpe ratio, on 252 days?

Answer

Answer to Question 18.1.

It is about 1.06.

0.0008/0.012=0.06670.0008/0.012 = 0.0667 per day, and ×252=1.06\times\sqrt{252} = 1.06.

Classic wrong answer. Annualising the standard deviation by 252. Mean scales with time, and volatility with its square root.

Question 18.2 ★ QR, QT, Analyst • any • junior

A backtest has a Sharpe ratio of 1 over 4 years. What is its tt-statistic? How long would a Sharpe of 0.5 need for the same tt-statistic?

Answer

Answer to Question 18.2.

The tt-statistic is 2, and a Sharpe of 0.5 needs 16 years.

The tt is Sharpe times the square root of years: 1×4=21 \times \sqrt{4} = 2. Then (2/0.5)2=16(2/0.5)^2 = 16.

Classic wrong answer. Thinking daily data buys precision for the mean. Daily or monthly, the length of the sample sets the tt.

Question 18.3 ★ QR, QT • hedge fund • junior

A book makes 10 basis points a day gross on capital, and trades 40% of its capital per day at 5 basis points per dollar traded. Daily volatility is 60 basis points. What are the gross and net Sharpes?

Answer

Answer to Question 18.3.

The gross Sharpe is about 2.65, and the net is about 2.12.

The cost is 0.4×5=20.4 \times 5 = 2 basis points a day. So 10/60×252=2.6510/60 \times \sqrt{252} = 2.65, and 8/60×252=2.128/60 \times \sqrt{252} = 2.12.

Classic wrong answer. Comparing cost with a Sharpe directly. Convert cost to the same daily units first.

Question 18.4 ★ QR, QT, Trader • hedge fund • junior

You may trade 5% of daily volume, and it takes 5 days to build a position. A stock trades $50 million a day. What is the largest position? With 200 such stocks, what is the largest gross book?

Answer

Answer to Question 18.4.

It’s $12.5 million per stock, and $2.5 billion across 200.

5% of $50 million is $2.5 million a day, for 5 days.

Classic wrong answer. Assuming that exit is free. You have the same limit on the way out, and in a stress the volume you get is not the volume you assumed.

Question 18.5 ★ QT, QR, Analyst • any • junior

A strategy wins 1 unit 55% of the time, and loses 1 unit otherwise. What is the expected value per trade? What is the Sharpe per trade, and for 100 independent trades a year?

Answer

Answer to Question 18.5.

The expected value is 0.1 a trade, the Sharpe per trade is about 0.1, and over 100 trades it’s about 1.0.

The mean is 0.55−0.45=0.10.55 - 0.45 = 0.1. The standard deviation is 1−0.01=0.995\sqrt{1 - 0.01} = 0.995. So 0.1/0.995×10=1.0050.1/0.995 \times 10 = 1.005.

Classic wrong answer. Treating a 55% hit rate as large. A rate is meaningless without the sizes: 55% with a small win and a large loss loses money.

Question 18.6 ★ QR, QT, Dev • any • junior

A daily return is standard normal. A researcher goes long if today’s return is positive and short if it is negative, and books today’s return. What Sharpe does that show? What does it become with the correct one-day shift?

Answer

Answer to Question 18.6.

It shows an annualised Sharpe of about 21. With a one-day shift, it’s zero.

The position is the sign of today’s return, so the P&L is ∣r∣|r|. Its mean is 2/π=0.798\sqrt{2/\pi} = 0.798, and its standard deviation is 1−2/π=0.603\sqrt{1 - 2/\pi} = 0.603. So the daily Sharpe is 1.32, and ×15.87=21\times 15.87 = 21.

With a shift, you hold today’s sign against tomorrow’s independent return, and the mean is 0.

Classic wrong answer. Trusting a Sharpe that high. A result of 21 is a bug. Look for a date join that used same-day data.

Question 18.7 ★ QR, QT, Analyst • any • junior

You test 20 strategies with no skill at a 5% significance level, one-sided and independent. What is the chance that at least one looks significant? How many are expected to?

Answer

Answer to Question 18.7.

It’s about 64%, and 1 is expected.

1−0.9520=0.6421 - 0.95^{20} = 0.642, and 20×0.05=120 \times 0.05 = 1.

Classic wrong answer. Reporting the best test at its own 5%. The right test level for the best of 20 is about 0.25%.

Question 18.8 ★ QR, QT, Risk • any • junior

For a strategy with normal annual returns, what is the chance of a losing year at a Sharpe of 1 and of 2?

Answer

Answer to Question 18.8.

It’s about 16% at a Sharpe of 1, and about 2.3% at a Sharpe of 2.

The mean over the standard deviation is the Sharpe, so a loss has probability Φ(−SR)\Phi(-\mathrm{SR}).

Classic wrong answer. Expecting a Sharpe of 1 to make money every year. One year in six should lose, and a manager has to survive that.

Question 18.9 ★ QR, QT, Analyst • any • junior

Thirty of 1,000 stocks a year go to zero, and the others return 10% on average. A backtest uses only stocks that are alive today. What does it show, and what is the true figure?

Answer

Answer to Question 18.9.

It shows 10%, and the truth is about 6.7%.

0.97×10%+0.03×(−100%)=6.7%0.97 \times 10\% + 0.03 \times (-100\%) = 6.7\%.

So the gap is 3.3 points a year from this one error, which is enough to create a strategy from nothing. Long-small-cap and distressed strategies are the ones most flattered.

Classic wrong answer. Using today’s index members or today’s listed names as the backtest universe. Use the membership as of each past date.

Question 18.10 ★ QR, Risk, QT • any • junior

A strategy has 25% annual volatility. To run at 10%, what leverage do you use? How does that change its Sharpe ratio and its return?

Answer

Answer to Question 18.10.

Use 0.4 of leverage. The Sharpe doesn’t change, and return falls in the same ratio as the volatility.

10/25=0.410/25 = 0.4, ignoring financing on the cash.

Classic wrong answer. Thinking leverage changes the Sharpe. It doesn’t, before financing and costs.

Question 18.11 ★★ QR, QT • hedge fund • junior, experienced

You try 100 independent strategies with no skill over 5 years, and pick the best one. What Sharpe ratio do you expect to see? Why is the quick rule 2ln⁡N\sqrt{2\ln N} a little high? What is a backtest’s headline Sharpe of 1 now worth?

Answer

Answer to Question 18.11.

You expect a Sharpe of about 1.12. The quick rule gives 1.36. So a headline Sharpe of 1 is below what pure noise gives after 100 tries.

The expected maximum of 100 standard normals is about 2.51. A Sharpe measured over TT years has standard error 1/T1/\sqrt{T}. So 2.51/5=1.122.51/\sqrt{5} = 1.12.

The rule is an upper limit that is reached slowly as NN grows: 2ln⁡100=3.03\sqrt{2\ln 100} = 3.03, and 3.03/5=1.363.03/\sqrt{5} = 1.36.

So you must report the number of trials. That’s why a good research log counts every variant, not just the final one.

Classic wrong answer. Treating out-of-sample data as immune. If you’ve looked at it several times, it’s in-sample too.

Question 18.12 ★★ QR, QT • hedge fund • junior, experienced

A signal has an information coefficient of 0.03, and cross-sectional stock volatility is 2% a day. What is the expected top-minus-bottom decile spread per day? Assume the signal and returns are jointly normal.

Answer

Answer to Question 18.12.

It’s about 21 basis points a day.

With correlation ρ\rho and joint normals, the mean of a decile on the signal is ρσ\rho\sigma times the signal’s mean in that decile. The top decile’s mean is φ(1.2816)/0.1=1.755\varphi(1.2816)/0.1 = 1.755. So the spread is 2×1.755×0.03×2%=0.21%2 \times 1.755 \times 0.03 \times 2\% = 0.21\%.

That’s before costs and with no constraint. An IC of 0.03 is decent. It looks small, and the spread shows why it’s worth something.

Classic wrong answer. Multiplying IC by volatility and stopping. That gives 6 basis points, per standard deviation of signal. The decile spread has to go about 3.5 standard deviations.

Question 18.13 ★★ QR, QT • hedge fund • junior, experienced

A signal follows an AR(1) with a daily coefficient of 0.9. What is its half-life? If it predicts 4 basis points a day today, what total return does it predict? What share of that do you capture by holding for 6.6 days, and for 20? How do you use this?

Answer

Answer to Question 18.13.

The half-life is about 6.6 days. The total is 40 basis points. You capture about 50% in 6.6 days, and about 88% in 20 days.

ln⁡0.5/ln⁡0.9=6.58\ln 0.5/\ln 0.9 = 6.58. The total is 4/(1−0.9)=404/(1 - 0.9) = 40. The captured share is 1−0.9h1 - 0.9^h.

So the half-life tells you a holding horizon, and the total tells you how much cost you can pay. If a round trip costs 10 basis points, trading this is well worth it, but not every small change in signal is.

Classic wrong answer. Reading the daily coefficient as the signal’s value. The daily number looks small, but it’s the sum that pays for the trade.

Question 18.14 ★★ QR, QT • hedge fund • junior, experienced

You regress 20-day forward returns on a slow signal, using daily observations, 1,000 days. What is wrong with the usual tt-statistic, and by how much? What if the signal changes completely every day? What do you use instead?

Answer

Answer to Question 18.14.

With a slow signal, the usual tt-statistic is too large by about 4.4 times, close to 20=4.47\sqrt{20} = 4.47, so a true null is rejected about 65% of the time at the 5% level. If the signal changes completely every day, the tt-statistic is about right. Use Newey-West errors with at least 20 lags, or non-overlapping data.

Forward returns share 19 of 20 days with the next observation. When the signal is also slow, the product of signal and error stays correlated over those days, and it’s that product that sets the standard error. In a simulation with AR coefficient 0.99, the tt has a standard deviation of 4.4.

With an independent signal each day, the product is uncorrelated despite the overlap, and the classic error holds. So the problem needs both overlap and persistence.

Classic wrong answer. Saying overlap alone inflates the tt. The residual is autocorrelated either way, but it’s its product with the regressor that matters.

Question 18.15 ★★ QR, QT, Trader • hedge fund • junior, experienced

Impact in basis points is σq\sigma\sqrt{q}, with σ=200\sigma = 200 basis points a day and qq the order as a share of daily volume. What is the impact at 5% of volume? At 2.5%? What does halving your order size do to total impact cost, and what does splitting it across two days do?

Answer

Answer to Question 18.15.

It’s about 45 basis points at 5%, and about 32 at 2.5%. Halving the order cuts total dollars of cost by about 65%. Splitting it over two days lowers it by about 29% in total.

2000.05=44.7200\sqrt{0.05} = 44.7, and 2000.025=31.6200\sqrt{0.025} = 31.6. So per dollar, the cost falls by a factor of 2\sqrt{2}.

Total cost is size times impact, and so goes with q1.5q^{1.5}. Half the size costs 0.51.5=35%0.5^{1.5} = 35\% of the total. Trading half on each of two days costs 2×35%=71%2 \times 35\% = 71\%.

That’s the case for splitting, as long as the signal lasts. It has its own price: delay costs, and a market that’s seen part of your order.

Classic wrong answer. Using a linear model of cost, which says size doesn’t matter. Every size question needs a concave impact per dollar and a convex total.

Question 18.16 ★★ QR, QT, Trader • hedge fund • junior, experienced

A strategy has a gross edge of 30 basis points per unit traded. Cost grows as 10Q10\sqrt{Q} basis points, with QQ in units of $100 million. What size maximises profit? What is the profit and the net edge there? At what size is the profit zero?

Answer

Answer to Question 18.16.

It’s maximised at $400 million. The net edge there is 10 basis points, and the profit is $400,000 per unit of turnover. Profit is zero at $900 million.

Profit per turn is Q(30−10Q)Q(30 - 10\sqrt{Q}). Setting the derivative to zero: 30−15Q=030 - 15\sqrt{Q} = 0, so Q=4Q = 4. The cost is 20, and the net is 10 basis points on $400 million.

In general, at the best size, cost takes two-thirds of the gross edge. And at Q=9Q = 9, cost equals edge.

So capacity isn’t a single number. There’s the size that maximises dollars, a smaller size that maximises Sharpe, and a larger one at which you make nothing.

Classic wrong answer. Quoting the zero-profit size as capacity. That’s where you’ve given all the edge to the market, and it’s more than twice the sensible size.

Question 18.17 ★★ QR, QT, Risk • hedge fund • junior, experienced

A strategy goes live. How long until a true Sharpe of 2, 1 and 0.5 gives a tt-statistic of 2? What does that imply for ramp-up? Propose a schedule, with the rules that move you up and down.

Answer

Answer to Question 18.17.

It takes 1, 4 and 16 years. So you won’t prove a modest Sharpe live, and ramp-up has to run on risk limits and on whether the live numbers fit the model.

t=SRTt = \mathrm{SR}\sqrt{T}, so T=(2/SR)2T = (2/\mathrm{SR})^2.

A schedule.

  1. Paper or small size, a few weeks: check that fills, costs and data match the backtest.
  2. 10% of target: at least 2 to 3 months. Compare cost per dollar, signal decay and fill rates to the model. These are tested much faster than P&L: they have many more observations.
  3. 25% to 50%: move up when the costs and exposures match, not because P&L is good.
  4. Target, with a drawdown limit calibrated to the Sharpe you truly believe, and a rule to cut size in half.

Classic wrong answer. Waiting for live P&L to prove it, or scaling up after a good month. Both read P&L too much. What you can test quickly is whether the machine is working as modelled.

Question 18.18 ★★ QR, QT, Trader • hedge fund • junior, experienced

A backtest made 20 basis points per dollar traded, and live trading made 11. You attribute 3 to delay, 2 to spread, 2 to impact and 1 to missed fills. What remains? How do you find it? Which of these get worse with size?

Answer

Answer to Question 18.18.

One basis point remains: 20−11−3−2−2−1=120 - 11 - 3 - 2 - 2 - 1 = 1. Delay and impact get worse with size, and spread stays, per dollar, as does the missed-fill rate, unless you reach a large share of volume.

To find it, check fees, borrow and funding, the mark prices, and the signal in live against the same day in the backtest. Run the backtest on live dates with the live orders, and compare per order.

The missed fills usually matter more than they look: you get filled when the price is going against you, and you miss when it’s going your way.

Classic wrong answer. Explaining the gap as “alpha decay” without a breakdown. Decay is the last explanation, after every mechanical cost is counted.

Question 18.19 ★★ QR, Risk, QT • hedge fund • junior, experienced

Your estimate of a strategy is a 15% return and a 10% volatility. What is the Kelly leverage, and the expected log growth? If the true return is half that, what do you earn at that leverage? And at half-Kelly?

Answer

Answer to Question 18.19.

The Kelly leverage is 15 times, and the expected log growth is about 113%. If the true return is half, you earn nothing in log terms at that leverage. At half-Kelly, you earn about 28%.

f=μ/σ2=0.15/0.01=15f = \mu/\sigma^2 = 0.15/0.01 = 15. Growth is fμ−12f2σ2=2.25−1.125=1.125f\mu - \tfrac12 f^2\sigma^2 = 2.25 - 1.125 = 1.125. With μ\mu halved, 15×0.075−1.125=015 \times 0.075 - 1.125 = 0. At half-Kelly, 7.5×0.075−0.5×56.25×0.01=0.2817.5 \times 0.075 - 0.5 \times 56.25 \times 0.01 = 0.281.

So full Kelly on an estimate is exactly what makes a real strategy earn zero. Since the mean is the hardest thing to estimate, using a fraction of Kelly isn’t timid: it’s what you have to do.

Classic wrong answer. Treating Kelly as a target. It’s a ceiling on an estimated edge, and it assumes no jumps and free continuous rebalancing.

Question 18.20 ★★ QR, QT, Dev • any • junior, experienced

You get a take-home with five years of daily prices for 500 stocks. Before fitting anything, what do you check? Give the order, and how you’d show it in the write-up.

Answer

Answer to Question 18.20.

First, check that the data are what they say they are: prices, then corporate actions, then survivorship, then time stamps. Do this before any signal, and show it on the first page.

The order.

  1. Shape. Count stocks by date, and missing days. Look for market holidays and gaps.
  2. Bad prints. Largest daily moves, with a date and a ticker. Do they reverse the next day? Check whether they fit a real event.
  3. Corporate actions. A split shows as a −50%-50\% return. Are prices adjusted? For dividends as well?
  4. Survivorship. Do names leave the file? If all 500 are there for all five years, the set was chosen with hindsight, so say what that biases.
  5. Time stamps. When does a value become known? Is a close usable on that day?
  6. Sanity. Cross-sectional average against a known index, if you have one. Return distribution and correlations.

The write-up. A table of what you found, what you changed and what you didn’t. Give a before-and-after for one clear error. Reviewers remember this more than a clever model.

How to build your own. Keep a checklist as code you can rerun. A second data set needs the same checks.

Classic wrong answer. Starting with a model. A high Sharpe found on an error is the classic failure of a take-home.

Follow-up. You find 12 prints with moves over 50%. Do you delete them? Don’t just delete them. Check each one. Real events stay, as they’re the return that a holder got. Fix or flag what’s wrong, and show the result both ways.

Question 18.21 ★★ QR, QT • hedge fund • junior, experienced

Two signals have information coefficients of 0.03 and 0.02, and their correlation is 0.5. What is the best combination, and its information coefficient? What if the correlation is 0.667? What does a low-correlation weak signal do?

Answer

Answer to Question 18.21.

The best mix is 80% and 20%, with an IC of about 0.0306. At a correlation of 0.667, the second gets no weight. A weak signal with low correlation can be worth more than a strong one that’s redundant.

Solve Cw=ICCw = \mathrm{IC}: w∝(0.0267,0.0067)w \propto (0.0267, 0.0067). The combined IC is IC⊤C−1IC=0.0306\sqrt{\mathrm{IC}^\top C^{-1}\mathrm{IC}} = 0.0306.

The weight on the second is zero when ρ=0.02/0.03=0.667\rho = 0.02/0.03 = 0.667.

At a correlation of 0, the combined value would be 0.032+0.022=0.036\sqrt{0.03^2 + 0.02^2} = 0.036. That’s much more than the 0.0306. So check a new signal’s residual IC after the existing ones, and not its standalone IC.

Classic wrong answer. Adding signals with equal weight, or adding the ICs. The ICs don’t add, and a correlated signal adds very little.

Question 18.22 ★★ Analyst, QR, Trader • hedge fund, bank • junior, experienced

You are asked to size a desk: $200 million of capital, 10% volatility, a Sharpe of 1, and fixed costs of $10 million. What are the P&L, the net return on capital and the break-even Sharpe? What numbers would change your mind?

Answer

Answer to Question 18.22.

P&L is $20 million, and net of costs, $10 million, or 5% on capital. The break-even Sharpe is 0.5.

10% on $200 million is $20 million of risk, and a Sharpe of 1 gets $20 million.

What changes my mind. Is that Sharpe net of trading costs? Is capital firm in a drawdown? Can costs fall, or do they rise with size? Is a 5% return above what the firm’s capital costs? At 8% hurdle, you’d need $16 million more than breakeven, so a Sharpe of 1.3.

Classic wrong answer. Comparing $10 million with zero. Capital has a cost, and a desk that makes less than that is shrinking the firm.

Question 18.23 ★★ Analyst, QT, Trader • crypto, market maker • junior, experienced

A market-making venture targets a market with $1 billion of daily volume, a 2% share and a net capture of 1 basis point. What is the annual revenue? Fixed costs are $2 million. What share breaks even? What do you do with this?

Answer

Answer to Question 18.23.

Revenue is about $730,000 a year, which doesn’t cover $2 million. Break-even is about a 5.5% share. So the plan as stated doesn’t work unless share or capture rises a lot.

109×0.02×10−4×365=$730,00010^9 \times 0.02 \times 10^{-4} \times 365 = \$730{,}000. So 2,000,000/(109×10−4×365)=5.48%2{,}000{,}000/(10^9 \times 10^{-4} \times 365) = 5.48\%.

This tells you which number to challenge. Is 1 basis point net of adverse selection? Is 5% realistic against whoever’s there? Is there a way to add markets, reusing the same infrastructure?

Classic wrong answer. Multiplying by a big total volume and being impressed. Share and net capture are the two hard numbers, and most ventures assume both.

Question 18.24 ★★ QR, QT, Trader • hedge fund • junior

A trend strategy wins 35% of the time, with an average win of 3 and an average loss of 1. What is the expectancy per trade and the standard deviation? What is the Sharpe over 100 independent trades a year? How often will you see 10 losses in a row in a given 10-trade run?

Answer

Answer to Question 18.24.

The expectancy is 0.4 a trade, the standard deviation about 1.91, and the Sharpe over 100 trades about 2.1. A run of 10 losses comes in about 1.3% of cases.

0.35×3−0.65=0.40.35 \times 3 - 0.65 = 0.4. The variance is 0.35×9+0.65−0.16=3.640.35 \times 9 + 0.65 - 0.16 = 3.64. So 0.4/1.91=0.210.4/1.91 = 0.21 per trade, and ×10=2.1\times 10 = 2.1.

0.6510=1.35%0.65^{10} = 1.35\%. Over a career of many trades, you will see it often.

Classic wrong answer. Assuming that independent trades are realistic. Trend positions held at the same time are correlated, so the real Sharpe is far below 2.1.

Question 18.25 ★★ QR, QT, Analyst • hedge fund • junior, experienced

A strategy returns 15% at 10% volatility, with a beta of 0.5 to a market with an 8% return and 16% volatility. Take the risk-free rate as zero. What are its alpha, residual volatility and information ratio? Why would you report those and not the Sharpe?

Answer

Answer to Question 18.25.

Alpha is 11%, residual volatility 6%, and the information ratio about 1.83.

15−0.5×8=1115 - 0.5 \times 8 = 11. Residual variance is 0.01−0.25×0.0256=0.00360.01 - 0.25 \times 0.0256 = 0.0036.

The Sharpe of 1.5 mixes market exposure with skill. A desk can buy beta for almost nothing, and is paying for the residual. Here the IR is above the Sharpe, because the beta part has a Sharpe of only 0.5.

Classic wrong answer. Comparing strategies by Sharpe when some carry market exposure. In a rising market, beta looks like skill.

Question 18.26 ★★ QR, QT, Analyst • hedge fund • junior, experienced

A candidate shows a backtest with a Sharpe of 3.5 over two years on large caps, 5 basis points of cost, rebalanced at the close. What do you ask? Give the order and what each answer would change.

Answer

Answer to Question 18.26.

Ask first how many variants were tried. Then ask about the period, the cost model, the execution assumption and the universe. A Sharpe of 3.5 in two years should make you look for what’s wrong before anything else.

In order.

  1. Trials. How many specifications? With 100, an in-sample Sharpe of 1.8 over 2 years comes from noise.
  2. Period. Which two years? Does it include a stress? What’s the Sharpe in each half, and by month?
  3. Execution at the close. Is the signal computed on that same close? If so, that’s look-ahead. Is the closing auction big enough?
  4. Costs. Is 5 basis points realistic for that turnover and for the size? What’s the break-even cost?
  5. Universe. Large caps today, or as of each date?
  6. Exposures. Beta, sector, factor. What’s left after removing them?

Each answer narrows it. If it survives all of that, I’d want it live at small size, as the next two years are the only clean test.

How to build your own. Keep this list in a fixed order, so that you ask the same questions of your own work.

Classic wrong answer. Attacking the model first. The usual problems are in the data and the process, well before the formula.

Question 18.27 ★★★ QR, QT • hedge fund • experienced

You have 100 strategies with no skill. Split the 5 years in half. Pick the best in the first half. What is its expected first-half Sharpe? What is its expected second-half Sharpe, and its rank? How do you turn that into a test of overfitting?

Answer

Answer to Question 18.27.

Its first-half Sharpe is about 1.59, on 2.5 years, and its second-half Sharpe is about 0, with a rank near 50 of 100. So in-sample rank tells you nothing.

The maximum is 2.51 in standard errors, and the standard error at T=2.5T = 2.5 is 1/2.5=0.6321/\sqrt{2.5} = 0.632. So 1.59.

With no skill, the two halves are independent, so the selected strategy is just one random draw on the other half. Its expected Sharpe is 0, and its rank is uniform: below median half the time.

The test. Split the data many ways. Each time, take the best in-sample and record where it ranks out of sample. The probability of backtest overfitting is the share of splits in which it’s below the median. About 50% means you’ve selected noise. A low number means that in-sample rank carries information.

Classic wrong answer. Using one split, as one split gives only one draw. And don’t reuse the out-of-sample half to choose, or it becomes in-sample.

Question 18.28 ★★★ QR, QT, Analyst • hedge fund • junior, experienced

You regress noise on 20 random features with 500 points. What R2R^2 do you expect in sample and out of sample? Why does out-of-sample go negative? What does this say about a reported in-sample R2R^2 of 4%?

Answer

Answer to Question 18.28.

It’s about 4% in sample, and about −4%-4\% out of sample. So an in-sample R2R^2 of 4% with 20 features and 500 points is what pure noise gives.

In sample, least squares fits about k/T=20/500k/T = 20/500 of the noise. Out of sample, the fitted coefficients are themselves noise, and they add variance to the forecast: another k/Tk/T. So the mean square error exceeds that of predicting zero, and R2R^2 is negative.

An honest reading uses adjusted R2R^2: about 0 here.

Classic wrong answer. Thinking 4% is a small and therefore modest claim. In markets, 4% would be huge. So the size says it’s probably from fitting.

Question 18.29 ★★★ QR, QT, Risk • hedge fund • junior, experienced

A strategy has a Sharpe of 2 at 10% volatility in calm periods (80% of the time), and a Sharpe of −1-1 at 20% volatility in stressed periods (the other 20%). What are its mean, volatility and blended Sharpe? Why is that lower than either the average or what a calm-only sample shows? What would you do?

Answer

Answer to Question 18.29.

The mean is 12%, the volatility about 20.4%, and the Sharpe about 0.59.

Calm mean: 2×10%=20%2 \times 10\% = 20\%. Stress: −1×20%=−20%-1 \times 20\% = -20\%. So 0.8×20−0.2×20=120.8 \times 20 - 0.2 \times 20 = 12.

The variance within regimes is 0.8×0.01+0.2×0.04=0.0160.8 \times 0.01 + 0.2 \times 0.04 = 0.016. The variance between them is 0.8×0.082+0.2×0.322=0.02560.8 \times 0.08^2 + 0.2 \times 0.32^2 = 0.0256. The total is 0.0416.

The Sharpe is lower than the time-weighted average of 1.4, because the shift of the mean between regimes adds variance. A calm-only backtest shows 2, at half the risk.

So I’d measure it by regime, reduce it when a stress measure rises, and set the limit using the stress regime’s number.

Classic wrong answer. Averaging Sharpes across regimes. Sharpes don’t average. Combine the means and variances first.

Question 18.30 ★★★ QR, QT, Trader • hedge fund • junior, experienced

A spread follows an Ornstein-Uhlenbeck process with a half-life of 10 days and a stationary standard deviation of 2%. You enter at 2 standard deviations, and exit at 0, with a cost of 0.2%. What is the profit per trade? About how long does it take? What is the return if you are always in a trade? What does the model leave out?

Answer

Answer to Question 18.30.

The profit is 3.8% a trade, it takes about 21 days, and if you were always in a trade, it would be about 47% a year. It leaves out the breakdown of the relation.

The gross profit is 2×2%=4%2 \times 2\% = 4\%, less 0.2%.

Mean reversion gives κ=ln⁡2/10=0.0693\kappa = \ln 2/10 = 0.0693. An expected path takes forever to reach exactly zero, but the noise gets you there. The first-passage result has a closed form: the chance of not yet having reached zero from x0x_0 after time tt is erf(x0e−κt/(σ2(1−e−2κt)))\mathrm{erf}\bigl(x_0 e^{-\kappa t}/(\sigma\sqrt{2(1 - e^{-2\kappa t})})\bigr), with σ\sigma the stationary standard deviation. From 2 standard deviations, that gives a mean of about 20.6 days and a median of about 16.5. So about 12.3 trades a year. That gives 3.8%×12.3=47%3.8\% \times 12.3 = 47\%.

Left out. Waiting for 2 standard deviations takes time, so you aren’t always in. The half-life and the mean are estimated, and may shift. The biggest risk is when a spread is wide because the relation is broken, as with a merger, bankruptcy or a new competitor. Then a mean-reversion model tells you to add at the worst time. So set a stop, or a time limit, using the half-life.

Classic wrong answer. Using the deterministic time, ln⁡(2σ/ϵ)/κ\ln(2\sigma/\epsilon)/\kappa, which diverges at zero. Noise matters. Simulate, or use the first-passage result.

Question 18.31 ★★★ QR, QT, Risk • hedge fund • junior, experienced

A strategy returns 12% at 10% volatility, and has a 2% chance per year of a 40% loss that its backtest never saw. What are the real mean, volatility and Sharpe? How long must a backtest be to see one such event with 90% probability?

Answer

Answer to Question 18.31.

The mean is about 11%, the volatility about 12.3%, and the Sharpe about 0.89, rather than 1.2. You need about 114 years to see it with 90% probability.

Mean: 0.98×12%+0.02×(−40%)=10.96%0.98 \times 12\% + 0.02 \times (-40\%) = 10.96\%. Variance: 0.98(0.01+0.0144)+0.02×0.16−0.10962=0.01510.98(0.01 + 0.0144) + 0.02 \times 0.16 - 0.1096^2 = 0.0151.

1−0.98n≥0.91 - 0.98^n \ge 0.9 means n≥ln⁡0.1/ln⁡0.98=114n \ge \ln 0.1/\ln 0.98 = 114.

So no backtest can rule it out. You have to reason from the structure: what would make this trade lose 40%? Is it selling insurance? Is it levered? And hold some capital against that scenario, with a stress test rather than an estimate.

Classic wrong answer. Saying that the backtest shows no such risk. A backtest can’t show what hasn’t happened. The longest data can only say that the rare event’s rate is low, and not that it’s zero.

Question 18.32 ★★★ QR, QT, LLE • market maker, hedge fund • junior, experienced

A signal predicts 2 basis points, with a half-life of 200 milliseconds, and it costs 1 basis point to trade. What is the net edge at 50, 200 and 400 milliseconds of latency? What is the break-even latency? What does cutting latency from 200 to 50 buy you, and what would you check first?

Answer

Answer to Question 18.32.

The net edge is about 0.68 basis points at 50 milliseconds, 0 at 200, and −0.5-0.5 at 400. The break-even latency is 200 milliseconds. So cutting latency from 200 to 50 takes the edge from zero to 0.68.

2×2−L/2002 \times 2^{-L/200}: 1.68, 1.00 and 0.50, less 1 of cost.

What I would check first. Where’s the 200 milliseconds going? Are the competitors at 10 milliseconds, in which case the measured signal is already gone by the time you see it? Latency to the fill matters, and not just to the send. The signal’s decay is an average: in the moments when it’s largest, it might also decay fastest.

Cutting latency is an investment with a clear return only if you keep the fills. If faster rivals pick off the best trades, what you get are the leftovers.

Classic wrong answer. Assuming that a decay curve measured from your own past fills is the true signal. Those fills are already selected. Measure from the quote stream.

Question 18.33 ★★★ QR, QT, Risk • any • junior, experienced

You add a stop-loss at −5%-5\% to a 20-day trade, on a driftless random walk with a daily volatility of 1%. What happens to the mean return, to the volatility and to the chance of stopping? When does a stop truly help, and when does it hurt?

Answer

Answer to Question 18.33.

The mean is unchanged at about 0, the volatility falls a little, from about 4.47% to about 4.28%, and the stop triggers about 21% of the time. A stop helps with momentum, and it hurts with mean reversion.

On a martingale, stopping at any rule keeps the expected value: that’s the optional stopping theorem. So on a random walk, the stop can only reshape the distribution: it cuts some tail losses in exchange for missing some recoveries.

So it isn’t a source of return. It’s a tool for risk when you have a limit. If returns trend, an early loss predicts more loss, and the stop helps. If they mean-revert, the stop sells at the worst time.

Also, price gaps can jump a stop, so its protection is limited. And costs go up.

Classic wrong answer. Testing stops on a backtest and being pleased with the lower drawdown. Look at the Sharpe, and ask whether the rule is reading a real trend or just cutting risk.

Question 18.34 ★★★ QR, QT, Dev • hedge fund • junior, experienced

You have ten persistent features and a 20-day forward return, with no true relation, over 1,000 days. What does five-fold random cross-validation show? What does cross-validation with blocks show, and with blocks with a 20-day purge? Why?

Answer

Answer to Question 18.34.

Five-fold random shows an R2R^2 of about +15%+15\% on pure noise. Blocks show about −46%-46\%, and purged blocks about −55%-55\%. So only blocked versions see that nothing is there.

Why. Each random fold’s test row sits next to training rows with almost the same features, and labels that share 19 of 20 days. The model memorises nearby noise, and the test rows reward it.

Blocks. These separate the test data in time, so the rows aren’t near neighbours. They’re still near at the block edges, and a 20-day purge removes those rows. The strongly negative R2R^2 shows what ten persistent features can fit in a short span, and how badly that extends to a new period.

So in finance, use blocked and purged folds, with an embargo after each test block.

Classic wrong answer. Trusting a high cross-validated R2R^2 on overlapping labels. Random folds are for independent rows. Time series with overlap need to keep the test separate from training.

Question 18.35 ★★★ QR, QT • any • junior, experienced

A strategy with even payoffs wins 52% of the time. How many trades until a tt-statistic of 1.96 is expected? How many for 80% power? How do your answers change for a 51% win rate?

Answer

Answer to Question 18.35.

It takes about 2,400 trades for an expected tt of 1.96, and about 4,900 for 80% power. At 51%, it’s four times as many: about 9,600 and 19,600.

The edge per trade is 2p−1=0.042p - 1 = 0.04, with standard deviation near 1. So t=0.04Nt = 0.04\sqrt{N}, and N=(1.96/0.04)2=2,401N = (1.96/0.04)^2 = 2{,}401.

For 80% power, you need the mean to be 1.96+0.841.96 + 0.84 standard errors from 0: (2.8/0.04)2=4,900(2.8/0.04)^2 = 4{,}900.

Halving the edge quadruples the sample. A 50% power setting is what the first number means: half of the time you’d still miss it.

Classic wrong answer. Using many correlated trades as if they were independent. Trades on the same day in the same market count for much less.

Question 18.36 ★★★ QR, QT, Trader • hedge fund • junior, experienced

Your strategy made 20% on $100 million, and 12% on $300 million. Fit a net return of a−bQa - b\sqrt{Q}. What does it predict at $500 million? What size is optimal if capital costs 5%? Why should you look at marginal returns, not average ones?

Answer

Answer to Question 18.36.

It fits a=30.9%a = 30.9\% and b=1.09b = 1.09 (with QQ in millions), and predicts about 6.5% at $500 million. The best size is about $250 million if capital costs 5%.

The two equations are 20=a−10b20 = a - 10b, and 12=a−17.32b12 = a - 17.32b. So b=8/7.32=1.093b = 8/7.32 = 1.093, and a=30.93a = 30.93.

Net dollars: Q(a−bQ)Q(a - b\sqrt{Q}). Its marginal return is a−1.5bQa - 1.5b\sqrt{Q}. Setting this to 5: Q=25.93/1.64=15.8\sqrt{Q} = 25.93/1.64 = 15.8, so Q=250Q = 250.

The marginal is what the next dollar earns. At $300 million, the average is 12%, but the last dollar earns only about 30.93−28.4=2.5%30.93 - 28.4 = 2.5\%, which is below the cost. An average that still looks good can hide this.

Classic wrong answer. Scaling while the average return stays above the hurdle. That adds capital past the point where the extra dollars earn less than they cost.

Question 18.37 ★★★ Analyst, QR • hedge fund • junior, experienced

Your book runs $300 million at 10% volatility and a Sharpe of 1. A vendor sells a dataset for $1 million a year. Its seller shows an uncorrelated Sharpe of 0.5. What is it worth to you? What if the true Sharpe is half of that? How would you test it before paying?

Answer

Answer to Question 18.37.

It’s worth about $3.5 million a year if the seller’s right, and about $0.9 million if the true Sharpe is half: less than the price. So test it well before paying.

Combined Sharpe for uncorrelated strategies is 1+0.25=1.118\sqrt{1 + 0.25} = 1.118. At $30 million of risk, the extra is 0.118×30=$3.540.118 \times 30 = \$3.54 million. At 0.25: 1.0625=1.031\sqrt{1.0625} = 1.031, and $0.92\$0.92 million.

Before paying. Ask for a trial on data you hold back. Check that the data’s time stamps are as of the date, and not revised. Find how long it’s been sold and to how many clients. Measure its IC after your existing signals. Check that the seller’s sample period doesn’t include the period that made them launch it.

Classic wrong answer. Comparing the seller’s standalone Sharpe with the price. What you buy is the increase, which goes as the square of the new Sharpe and depends on correlation with your book.

Question 18.38 ★★★★ QR, QT, Risk • hedge fund • experienced

A strategy backtested at a Sharpe of 2.4 over 3 years, and was the best of 200 independent trials. Live for 6 months, it shows 0.5. Your prior for the true Sharpe of any idea is normal with mean 0 and standard deviation 0.5. Estimate the true Sharpe after the backtest, and after the live period. Is the strategy broken? How do you size it, what would make you kill it, and what do you tell the portfolio manager?

Answer

Answer to Question 18.38.

My estimate is a true Sharpe of about 0.35 after the backtest, and about 0.36 after 6 live months, with a standard deviation of about 0.37. The strategy isn’t broken: the live number fits that. The backtest was overstated. I would run it at a size set by 0.36, not 2.4, and kill it on a defined drawdown or a change in its mechanics.

Step 1: the haircut. The best of 200 null trials over 3 years has an expected Sharpe of 2.746/3=1.5852.746/\sqrt{3} = 1.585. So only 2.4−1.585=0.8152.4 - 1.585 = 0.815 is left to explain.

Step 2: the prior. Treat 0.815 as a noisy observation, with variance 1/31/3, under a prior of variance 0.25. The posterior variance is 1/(4+3)=0.1431/(4 + 3) = 0.143, and its root is 0.378. The mean is 0.143×0.815×3=0.3490.143 \times 0.815 \times 3 = 0.349.

Step 3: the live period. Six months is a variance of 2, with an observation of 0.5. The new precision is 7+0.5=7.57 + 0.5 = 7.5, so the variance is 0.133, and the root is 0.365. The mean is (7×0.349+0.5×0.5)/7.5=0.359(7 \times 0.349 + 0.5 \times 0.5)/7.5 = 0.359.

So live data barely moves it: six months can’t say much. The chance that the true Sharpe is below zero is about 16%.

Is it broken? A live 0.5 is far below 2.4 in size, but six months of data has a standard error near 1.4. So it is only about 0.1 standard errors from 0.36, and only about 1.3 from 2.4: the live number cannot separate them. The right question isn’t what P&L did. It’s whether the costs and fills match, and whether the signal predicts at the same rate as in the backtest.

Size. A size made for 2.4 would be nearly seven times too big. For a Sharpe of 0.36 at a given risk budget, half-Kelly leverage is small, and in a book it’s what it adds to the portfolio that matters. I’d start with a low risk share, say 15% of what the backtest suggests.

Kill rule. Set in advance: a drawdown of about 2 to 2.5 years of the expected return, a clear change in the mechanics, a failure of the cost model, or a fall in the signal’s measured IC to near zero. Not because of a bad month.

To the manager. “Backtest 2.4 is not the number. After the search and with a skeptical prior, I think about 0.35, plus or minus 0.4, so it has a 16% chance of having no edge. I’d keep it small, spend the next six months on the fills and costs, and revisit. Here’s the rule that would make me cut it.”

Classic wrong answer. Comparing live with backtest and deciding it’s broken, or that it’s fine. Neither can be concluded from six months. What moved your view was the number of trials, and that’s what most people leave out.

Follow-up. What if the 200 trials were highly correlated variants? Then the effective number is smaller, the haircut falls, and the estimate rises. Estimate the effective number from the correlations, and show both.

Question 18.39 ★★★★ Analyst, QT, QR • crypto, market maker • junior, experienced

You are asked whether to back a crypto market-making venture in a set of markets with $20 billion of daily volume on each of 365 days a year. Net capture is 0.5 basis points of your traded volume, fixed costs are $6 million a year, and inventory is fixed at $20 million whatever your share. You think venue failure costs 0.9% of inventory a year, and the capital in that inventory costs 15% a year. Work out revenue at shares of 1%, 2% and 5%, and break-even shares with and without the capital charge, at this capture and at half of it. What would you ask? What would make you say no?

Answer

Answer to Question 18.39.

Revenue is about $3.65 million, $7.3 million and $18.25 million at shares of 1%, 2% and 5%. The break-even share is about 1.7% without the capital charge and about 2.5% with it, and at half the capture about 3.4% without the charge and about 5.0% with it. I would back it only with evidence of the capture rate and of a path to 3%, and a clear plan for stopping.

Revenue. $20 billion×s×0.5×10−4×365=$365 million×s\$20 \text{ billion} \times s \times 0.5 \times 10^{-4} \times 365 = \$365 \text{ million} \times s.

Costs. Fixed costs of $6 million, and venue risk of 0.9%×$20 million=$180,0000.9\% \times \$20 \text{ million} = \$180{,}000. Capital at 15% is $3 million.

Break-even. 6.18/365=1.69%6.18/365 = 1.69\%. With capital, 9.18/365=2.52%9.18/365 = 2.52\%. At half the capture, 6.18/182.5=3.39%6.18/182.5 = 3.39\%, and with capital, 9.18/182.5=5.03%9.18/182.5 = 5.03\%.

What I’d ask.

  • Capture: is 0.5 basis points net of adverse selection, fees, inventory loss and funding? In which markets? How was it measured?
  • Share: who’s in these markets now? A 2% share means taking it from someone who won’t give it up.
  • Rebates and fee tiers: they often make the difference, and a venue can change them.
  • Inventory: $20 million across many venues is thin. Is more capital needed, and does it stay available in a stress?
  • People: what’s the plan if two key people leave?

What would make me say no.

  • If the capture rate is from a backtest, and no live data exists.
  • If the plan needs both share above 3% and capture at the high end.
  • If a venue failure would wipe out the equity: limit exposure, or don’t start.
  • If a large part of revenue relies on a single venue’s rebate.

What to do next. Fund a small pilot with a budget and a date: run in two markets for six months, and measure capture, share and adverse selection. Those three numbers decide everything. Stop if capture is under half of the plan.

Classic wrong answer. Building a large spreadsheet, with detailed five-year revenue forecasts. An interviewer wants the two or three numbers the result hangs on, and what would change your decision.

Question 18.40 ★★★★ QR, QT • hedge fund • experienced

A take-home gives you a signal ss for 500 stocks: a unit-variance AR(1), half-life 5 days. Daily expected return per unit of ss is 2 basis points, and stock noise is 2% a day. You hold positions proportional to ss, with gross exposure of 1, refreshed every hh days. It costs 5 basis points per dollar traded. Derive gross alpha, turnover, net return and Sharpe in hh. Which refresh is best, and what would you write up?

Answer

Answer to Question 18.40.

Net return is best at about 5 to 6 days, with about 0.94 basis points a day and a Sharpe of about 1.33. Daily refresh loses money after costs, and a 20-day refresh makes about 0.57 basis points a day, with a Sharpe of about 0.8.

Derivation. The AR coefficient is ρ=2−1/5=0.8706\rho = 2^{-1/5} = 0.8706. With positions st0/E∣s∣s_{t_0}/E|s| normalised to a gross of 1, per-stock normalisation brings in E∣s∣=2/πE|s| = \sqrt{2/\pi}.

Gross alpha. The holding at age kk earns 2ρk/E∣s∣2\rho^k/E|s| basis points. Averaged over k=0,…,h−1k = 0, \dots, h - 1,

α(h)=2π/2  1−ρhh(1−ρ).\alpha(h) = 2\sqrt{\pi/2}\;\frac{1 - \rho^h}{h(1 - \rho)}.

Turnover. The refresh moves st−st−hs_{t} - s_{t-h}, which is normal with variance 2(1−ρh)2(1 - \rho^h). So its mean absolute size relative to E∣s∣E|s| is 2(1−ρh)\sqrt{2(1 - \rho^h)}, per hh days.

Cost. 52(1−ρh)/h5\sqrt{2(1 - \rho^h)}/h basis points a day.

Volatility. With 500 independent stocks, π/2×200/500=11.2\sqrt{\pi/2} \times 200/\sqrt{500} = 11.2 basis points a day.

hhAlpha (bp/day)Cost (bp/day)NetSharpe
12.512.54−0.04-0.04−0.05-0.05
32.201.370.821.16
51.941.000.941.33
61.820.890.941.33
101.450.610.841.19
200.910.340.570.80

Each column is rounded on its own. A simulation of the full book agrees with the table.

What I’d write up.

  • The headline: best refresh about a week, and net Sharpe about 1.3, with a gross Sharpe of about 2.7. So costs take about half.
  • The curve is flat between 3 and 10 days. So I wouldn’t fine-tune hh; I’d pick one in the middle.
  • What’s missing: costs here are linear, with no impact; the noise is independent across stocks, so the Sharpe is high. With a market factor, much of it would go.
  • A better rule than a fixed refresh is to trade partway towards the target each day, which does better than any fixed hh when cost is linear. I would try that next, and say so.

Classic wrong answer. Reporting the gross Sharpe at h=1h = 1, about 3.5, as the signal’s quality. Costs are part of the signal’s value: at daily refresh this signal makes nothing.