26Cloud Against On-Premises

On 4 November 2021 CME Group and Google Cloud announced a ten-year partnership: CME would move its technology to Google Cloud, starting the next year with data and clearing services and eventually all of its markets. The same year most trading firms kept every order path in their own racks, as Book 14 describes. Both decisions can be right, because they are about different workloads. This chapter prices the one where the question is most often asked and least often computed — the research cluster — with every cloud price taken from a dated, published list, every assumption about owned hardware stated, and chapter 13’s simulator run on interruptible capacity.

26.1 What moves to the cloud and what cannot

A public cloud (Book 14, chapter 16) rents machines by the hour in regions and availability zones the firm does not control. Four questions sort a workload. How sensitive is it to latency and jitter — the order path, Book 14 shows, is still measured in the firm’s own racks next to the matching engine. How variable is its demand — a sweep that needs thousands of cores for a weekend and none on Monday is the cloud’s case. Where is its data — the next section’s question. And who must be able to inspect it — the last section’s.

Infrastructure as code is what makes elastic capacity usable: a cluster that exists only while a sweep runs must be created and destroyed by a program, identically every time, and an owned cluster described the same way can be rebuilt after a failure. It applies to both sides of this chapter’s comparison.

26.2 The economics of compute: owned, reserved, on demand, interruptible

The chapter prices one cloud instance of 96 hardware threads in one US region from the provider’s public price list of 25 September 2026: 4.284 dollars an hour on demand, 2.834 reserved for one year and 1.886 for three years without upfront payment, and 1.737 on the spot market on 28 September. Per thread-hour that is 4.46, 2.95, 1.96 and 1.81 cents. Owned hardware has no such list. The model builds a two-socket server of 512 threads from the list price of its CPU (10 931 dollars each), assumes the CPUs are half of the server’s price, spreads it over four years, powers it at the US average commercial price of electricity (14.53 cents a kilowatt-hour in July 2026) with a 1.3 kW draw and a building overhead of 1.4, and adds $300 a month of rack space and a two-hundredth of an engineer’s $250 000 a year. Those last figures are assumptions, not facts, and the results below are shown for server prices up to three times the assumption.

@dataclass(frozen=True)
class OwnedServer:
    price: float              # purchase, USD
    life_months: int          # straight-line depreciation
    kw: float                 # average draw at load, kW
    pue: float                # facility overhead (power usage effectiveness)
    power_price: float        # USD per kWh
    space: float              # USD per server-month
    staff: float              # USD per server-month

    @property
    def monthly(self) -> float:
        power = self.kw * self.pue * HOURS_PER_MONTH * self.power_price
        return self.price / self.life_months + power + self.space + self.staff

    def per_thread_hour(self, threads: int) -> float:
        return self.monthly / (threads * HOURS_PER_MONTH)

The owned server then costs $1 508 a month, 0.40 cents per thread-hour when fully used. The comparison turns on utilisation: capacity that is paid for every hour — owned or reserved — costs its hourly price divided by the share of hours it is used, while on-demand and spot capacity cost their price only when used (Figure 26.1).

A thread is not a unit of work: the owned CPU and the cloud instance’s CPU run a given backtest at different speeds, and a firm should compare them with chapter 14’s measurements of its own workload before trusting any break-even. A factor of two in speed divides or multiplies the owned server’s whole cost per unit of work by two, and the break-even with it: 4.5% or 18% against on demand.

26.3 Buying for a year of demand

A research cluster’s demand is not a utilisation but a profile. The model’s year has an interactive base of 1 500 threads, 1 000 more in weekday working hours, a nightly batch of 4 000 more on weekdays, a weekend sweep of 12 000 more for eight hours every Saturday, and twenty unplanned bursts of 8 000 for four hours: a mean of 3 364 threads against a peak of 13 500. Sorting the hours by demand gives the load-duration curve, and each level of capacity is used in the share of hours the demand reaches it.

def plan_capacity(demand, owned: float, reserved: float, on_demand: float,
                  spot: float | None = None, spot_share: float = 1.0) -> Plan:
    """Buy each unit of capacity the cheapest way: the k-th unit is needed in the share
    f(k) of hours with demand >= k; fixed options cost their price every hour, metered
    ones only f(k) of hours. `spot_share` of metered work may run on spot."""
    d = np.sort(np.asarray(demand, dtype=float))
    hours = len(d)
    metered = on_demand if spot is None else (
        spot_share * spot + (1 - spot_share) * on_demand)
    levels = np.arange(1, int(np.ceil(d[-1])) + 1)
    f = 1.0 - np.searchsorted(d, levels - 1e-9, side="left") / hours   # hours >= k
    n_own = int(np.sum(owned < np.minimum(reserved, metered * f)))
    n_res = int(np.sum((reserved <= owned) & (reserved < metered * f)))
    over = np.clip(d - (n_own + n_res), 0, None).sum()       # unit-hours bought hourly
    cost = {"owned": n_own * owned * hours if n_own else 0.0,
            "reserved": n_res * reserved * hours if n_res else 0.0,
            "metered": over * metered}
    return Plan(n_own, n_res, float(over), cost)

The optimum in Figure 26.3 is a hybrid, and it is one because of the shape of the demand, not because of any single price: the base and the nightly batch are used more than 9% of the time and are cheaper owned; the weekend sweeps and bursts, 496 hours of the year, are cheaper rented. A firm whose research runs flat out every night owns; a firm whose research is a few enormous sweeps a year rents; most are in between.

26.4 Spot capacity and the cost of interruption

Spot capacity is 59% cheaper than on demand for this instance, and the provider’s own advisor places it in the highest band of interruption frequency, above 20% of instances reclaimed in the trailing month — at least 0.03% an hour. Chapter 13’s sweep of 10 000 backtests, forty of them seven and a half hours long, runs on 24 instances, each billed until its last task ends.

def checkpointed(tasks: list, chunk: float, overhead: float) -> list:
    """Split each task into chunks of `chunk` seconds (plus `overhead` per checkpoint),
    chained so that a preempted chunk restarts from the last checkpoint."""
    out, nid = [], max(t.tid for t in tasks) + 1
    for t in tasks:
        k = max(1, int(np.ceil(t.duration / chunk)))
        prev = None
        for i in range(k):
            d = t.duration / k + (overhead if k > 1 else 0.0)
            tid = t.tid if i == 0 else nid
            if i > 0:
                nid += 1
            deps = (prev,) if prev is not None else ()
            out.append(J.Task(tid, d, t.estimate / k, t.team, deps=deps))
            prev = tid
    return out

On demand, the sweep costs $203 and takes 7.5 hours, the length of its longest backtest. On spot without interruptions it costs $82. What interruption costs is mostly time, and it falls on the long tasks (Figure 26.4): a 7.5-hour backtest interrupted after six hours starts again, so a single interruption of one of the forty long tasks nearly doubles the sweep’s length. Cutting each backtest into 15-minute chunks that write a checkpoint — the training checkpoint of Book 12, chapter 23, applied to backtests — bounds the loss at one chunk, and the sweep costs the same at every interruption rate the chart shows. Spot capacity is for work that can be interrupted, and checkpointing is what makes work interruptible.

26.5 Data gravity and data-transfer charges

The provider’s data-transfer charge (Book 14, chapter 16) is asymmetric: data coming into the region is free, data leaving it to the internet costs 9 cents a gigabyte for the first 10 terabytes of a month, falling in tiers to 5 cents above 150 terabytes. Keeping a 500-terabyte tick store in the cloud’s standard object storage (chapter 2’s price) costs $11 776 a month; taking a full copy out once costs $29 491, about twenty months of an owned server. So the tick store’s location decides where research runs: once it is in the cloud, research follows it, and results — a few gigabytes a sweep — are what should cross the boundary. A hybrid design keeps one authoritative copy and sends the computation to it, or keeps two copies and pays once to fill the second.

26.6 Security, control and regulators

Most cloud incidents at financial firms are on the customer’s side of that line: a storage bucket left public, a key committed to a repository, a permission granted too widely — the subjects of chapter 29. Regulators add a second line. In the European Union the EBA’s guidelines on outsourcing, which apply since 30 September 2019 and absorbed its earlier recommendation on cloud outsourcing, require a documented exit strategy for every critical or important function outsourced: a plan, tested, for moving the function elsewhere if the provider fails or the relationship ends. For a research cluster that is easy — the owned half of a hybrid is the exit. For an order path or a clearing system, it is a second full implementation, which is part of why a move like CME’s is a ten-year programme. Concentration — many firms relying on the same few providers — is the operational risk (Book 6, chapter 28) that supervisors watch most.

26.7 Tutorial: own, rent or both

26.8 Build: the cloud cost model

26.9 Exercises

26.10 Problem: Own, Rent or Both

26.11 Interview questions