9Colocation Products and How They Are Sold

European rules require a trading venue that sells co-location to give every user of the same service access to its network under the same conditions, and the rule’s list of conditions includes, between cooling and access to data, the length of the cable. In New Jersey, Nasdaq has told the SEC that it is rebuilding the cabling of its data-centre campus so that every telecom carrier’s connection reaches its customers over an equal distance, and it has named the work the “Equalization Project”. The customer whose cabinet stands next to the matching engine and the customer at the far end of the hall are meant to wait the same nanoseconds for the same bits, and where the room does not make the cables equal, a coil of fibre does.

Part III of this book is about where the exchanges are and what it costs to be near them. This chapter describes what a venue’s data centre sells and on what terms: space, power, cooling and hands; cross-connects and the meet-me room; the fairness rules; the connections into the venue’s own network; and the fee schedules, which in the United States are public filings.

9.1 What colocation sells: space, power, cooling and hands

A venue’s data centre is an industrial building. ICE’s Mahwah data centre, which houses the NYSE exchanges, is a building of 398 000 square feet on a 28-acre site 34 miles from Wall Street, with 28 megawatts of electrical power; its colocation cabinets come in 4, 8 and 12 kilowatts, scalable to 15, with partial cabinets of 1 to 2 kilowatts, each fed by two independent power circuits. Cabinets are sold by power density because power and the cooling that removes it are the building’s scarce resources: chapter 8’s arithmetic of servers per cabinet is the customer’s side of the same constraint.

9.2 Cross-connects and the meet-me room

Every connection that enters or crosses a colocation hall is a cross-connect that someone pays for each month. ICE’s operating policies for Mahwah are typical of the rules: carriers may only sit in the meet-me rooms, with at most 16 kilowatts each; they reach a customer only through a cross-connect, for which a monthly carrier connection fee is charged; and a customer trading on the venue must use the venue’s network, not a direct cross-connect to the venue’s systems. Nasdaq’s filings describe the same structure in Carteret: a carrier cage, a patch panel, cabling managed by the exchange to a distribution point, then to the customer’s cabinet, each piece “color-coded, inventoried, and auditable”.

9.3 Equal cable lengths and the fairness rules

A cabinet 5 metres from the engine in a hall whose longest run is 150 metres therefore gives up $707\,\mathrm{n}\mathrm{s}$, the price of fairness paid by the customer who would otherwise have won by standing close. The European rule makes the principle law for venues in its scope, adding that the venue must monitor “all connections and latency measurements” and must sell each co-location service on its own rather than in bundles. In the United States, colocation services are exchange services whose fees are filed with the SEC, and exchanges describe their equalisation in those filings. What equalisation removes is the race for the nearest rack; what it leaves is every other race: faster servers, faster switches, better code and, for firms that trade across venues, the routes between buildings of chapters 10 to 14.

9.4 Port speeds and connectivity products

A customer’s cabinet reaches the venue’s systems through the venue’s own network, over ports that the venue sells in several speeds and kinds: market-data ports and order-entry ports, one or ten or more gigabits, a standard network and, at some venues, a lower-latency one at a higher price, and connections to the venue’s disaster-recovery site (chapter 28). The price of a connection is set by filing, like the cabinet’s. The fastest product of Box 9.1, an options exchange’s 10-gigabit ultra-low-latency connection, costs ten times its 1-gigabit connection. Some venues also sell connectivity between data centres (the wireless links of chapter 14 among them), and the rule of equal conditions applies to what they sell within each product.

9.5 Fee schedules and how a footprint is bought

The chart shows where the money goes. A cabinet at Nasdaq costs about 90 000 USD a year once its installation is spread over three years; two ultra-low-latency connections to an options exchange cost 360 000 USD a year, with nothing to install. The building is the cheap part; the venue’s fastest connections are not, and a firm pays for them at every venue it trades.

def bill(schedule, footprint, term_months=36):
    lines, monthly, one_time = [], 0.0, 0.0
    for key, qty in footprint:
        it = schedule.items[key]
        lines.append((it.name, qty, qty * it.monthly, qty * it.one_time))
        monthly += qty * it.monthly
        one_time += qty * it.one_time
    amort = one_time / term_months
    return {"monthly": monthly, "one_time": one_time, "amortised": amort, "total_monthly": monthly + amort,
            "annual": 12 * (monthly + amort), "lines": lines}


def stale(schedule, today, days=60):
    t = dt.date.fromisoformat(today)
    return [k for k, it in schedule.items.items() if (t - dt.date.fromisoformat(it.as_of)).days > days]

9.6 Tutorial: pricing a footprint

9.7 Build: the colocation bill

9.8 Exercises

9.9 Problem: The Cage That Paid for Itself

9.10 Interview questions