13Long-Haul Fibre

In August 2010 a new fibre route between Chicago and New Jersey went into service: shorter than the routes before it, with lower-latency equipment, and quoted at $6.65\,\mathrm{m}\mathrm{s}$ one way between the Cermak building in Chicago and Nasdaq’s data centre in Carteret, some $13.3\,\mathrm{m}\mathrm{s}$ round trip. Its cost was publicly estimated at about 300 million dollars. Before it, prices in New Jersey had responded to Chicago within 7.25 to $7.95\,\mathrm{m}\mathrm{s}$; around its opening, the response times drop by 1 to $1.5\,\mathrm{m}\mathrm{s}$. Within two years microwave networks estimated at 4.2 to $5.2\,\mathrm{m}\mathrm{s}$ had beaten it, and the cable’s buyers had learnt what chapter 10’s floors say: glass is slower than air by 46%, and no amount of digging changes that.

Fibre did not lose. It carries everything that is not the fastest few messages: market data by the gigabit, orders when the weather closes the radio links (chapter 14), the transatlantic routes that no tower can cross, and every connection from a firm’s cabinet to its carrier. This chapter takes a long-haul route apart: the glass and what happens to light in it, the path and how straight it is, the equipment, the subsea cables, the ways of buying fibre, and the new glass that is nearly as fast as air.

13.1 Light in glass: attenuation, amplifiers and dispersion

Standard single-mode fibre at $1550\,\mathrm{n}\mathrm{m}$ loses about $0.16\,\mathrm{dB}$ per kilometre (Corning’s datasheet for its ultra-low-loss fibre gives 0.16 typical, 0.17 maximum), so a signal loses half its power every 19 kilometres and a thousandth of it every 190. Over the $1359\,\mathrm{k}\mathrm{m}$ of glass that the 2010 route must contain (below), the loss would be $217\,\mathrm{dB}$: no receiver hears that. Long-haul fibre is therefore cut into spans, with an amplifier at the end of each.

The amplifier costs little time (the doped fibre is metres, the pass-through a fraction of a microsecond), which is why the route model below charges $0.1\,\text{µ}\mathrm{s}$ per span, an assumption, and why amplification is not the latency problem. Dispersion is.

The same datasheet bounds dispersion at $18\,\mathrm{p}\mathrm{s}$ per nanometre per kilometre at $1550\,\mathrm{n}\mathrm{m}$: over 1 359 kilometres, a spread of about $24\,\mathrm{n}\mathrm{s}$ per nanometre of spectrum, far more than a bit at ten gigabits ($0.1\,\mathrm{n}\mathrm{s}$). Compensating it optically adds glass to the path: a spool in every span, which the model sets at a fifth of the span’s length. Compensating it electronically adds processing time at the ends instead, and leaves the path alone.

FEC lets a link run with a noisier signal, hence longer spans or higher speeds, at the price of a delay per terminal. Laughlin, Aguirre and Grundfest note the same trade-off in microwave radios, whose latency is “dominated by” FEC buffering and interleaving. For low-latency fibre the choices are therefore the ones the model exposes: short spans or long ones, optical or electronic dispersion compensation, heavy FEC or light.

13.2 Route straightness: the Chicago–New York story

Chapter 10’s floor for 350 Cermak to Carteret is $5507\,\text{µ}\mathrm{s}$ in straight fibre; the 2010 route was quoted at $6650\,\text{µ}\mathrm{s}$. The difference, $1143\,\text{µ}\mathrm{s}$, is path and equipment, and inverting the published figure separates them once an equipment time is assumed. With two terminals of $10\,\text{µ}\mathrm{s}$ and an amplifier every 80 kilometres at $0.1\,\text{µ}\mathrm{s}$ (the model’s assumptions, $21.7\,\text{µ}\mathrm{s}$ in all), the route must contain 1 359 kilometres of glass: 230 kilometres more than the geodesic, a path factor of 1.20. The later figure of $6.55\,\mathrm{m}\mathrm{s}$ implies 1 339 kilometres, 20 fewer, or faster equipment.

def components(route):
    n = route.spans[0].n_g
    glass = _us(route.geodesic_km, n)
    path = sum(_us(s.km, s.n_g) for s in route.spans) - glass
    dcf = sum(_us(s.dcf_km, s.n_g) for s in route.spans)
    amps = sum(s.amp_us for s in route.spans)
    term = 2 * route.terminal_us
    return {"glass": glass, "path": path, "dcf": dcf, "amps": amps, "terminals": term,
            "total": glass + path + dcf + amps + term}


def invert(published_us, geodesic_km, n_g=1.462, equipment_us=0.0):
    """A published one-way latency, less assumed equipment time, as the glass the route must have."""
    glass_us = published_us - equipment_us
    path_km = glass_us * 1e-6 * C0 / n_g / 1e3
    return {"path_km": path_km, "factor": path_km / geodesic_km,
            "excess_km": path_km - geodesic_km}

Figure 13.2 shows why the conclusion is robust: whatever the equipment, most of the millisecond is path. A cable must follow rights of way it can obtain; the 2010 route bought straightness at 300 million dollars and still ran 17 to 20% longer than the geodesic. The microwave networks that beat it had route factors near 1.01 in air (chapter 10) and a medium 46% faster.

13.3 Subsea cables

A subsea route is a terrestrial route at each end, a landing station on each coast, and the ocean between. Hibernia Express, the first new transatlantic cable in over twelve years when it opened in 2015, published a round trip under $58.95\,\mathrm{m}\mathrm{s}$ between Equinix’s NY4 in Secaucus and LD4 in Slough, more than five milliseconds better than the existing cables, over a fibre pair of Corning’s pure-silica-core fibre. The geodesic between the two buildings is $5551\,\mathrm{k}\mathrm{m}$; half the round trip, $29.475\,\mathrm{m}\mathrm{s}$, implies about 6 040 kilometres of glass, a path factor of 1.09. Across an ocean the geodesic is almost available, apart from the landings and the tails to the data centres; on land, the rights of way cost the Chicago route twice as much in proportion.

13.4 Dark fibre against leased wavelengths

An internet carrier’s annual report shows the form in practice: most of its dark fibre is held “in the form of long-term leases under indefeasible rights of use”, it relies on the fibre’s owner to maintain it, and it amortises the IRUs over generally 15 to 20 years. For a trading firm, the choice between dark fibre and a lit service is a choice between control and cost. Dark fibre lets the firm choose its own equipment (light FEC, electronic dispersion compensation, no switches in the path) and costs an IRU, the equipment and yearly maintenance; a lit service costs a monthly fee and gives the seller’s latency.

The break-even arithmetic is simple and its inputs are not: IRU prices and lit fees are negotiated, not published, which is why the figure uses illustrative numbers. With them, dark fibre costs 0.675 million a year spread over twenty years against 1.08 million for the lease, and pays back its up-front outlay after 9.6 years. A firm that expects to keep the route for less than that, or whose strategy may move to microwave, leases; a firm that needs to control every microsecond of the path buys, whatever the break-even says.

def dark(iru, years, om_year, capex, capex_years):
    """Annual cost of dark fibre: the IRU over its term, O&M, and the firm's own
    optical equipment over its life."""
    return iru / years + om_year + capex / capex_years


def lit(monthly):
    return 12 * monthly


def break_even_years(iru, om_year, capex, monthly):
    """Years after which dark fibre (IRU and equipment up front, O&M yearly) has cost less
    in total than a lit service at `monthly`; infinite if the lease is always cheaper."""
    saving = 12 * monthly - om_year
    return float("inf") if saving <= 0 else (iru + capex) / saving

13.5 Faster glass: hollow-core fibre

For decades hollow-core fibre was faster and useless: it lost too much light to cover long distances. A 2025 paper from Microsoft’s fibre team and the University of Southampton reports a hollow-core fibre with a loss of $0.091\,\mathrm{dB}$ per kilometre at $1550\,\mathrm{n}\mathrm{m}$, below the $0.14\,\mathrm{dB}$ of the best solid silica fibre, with transmission speeds 50% higher, “a 30% reduced latency” and six times less chromatic dispersion than standard telecom fibre. If the loss holds in cabled routes, the fibre floor moves to nearly the air floor.

Figure 13.4 rebuilds the 2010 route four ways with the model (a group index of 1.003 for the hollow core is an assumption consistent with the paper’s 30%). As inverted it takes $6.650\,\mathrm{m}\mathrm{s}$. With a dispersion-compensating spool of a fifth of each span, it would take $7.976\,\mathrm{m}\mathrm{s}$: optical compensation costs more than a millisecond on this route, every reason for a low-latency route to compensate electronically. In hollow-core fibre on the same path it would take $4.569\,\mathrm{m}\mathrm{s}$, $2.08\,\mathrm{m}\mathrm{s}$ less; in standard fibre along the geodesic, $5.528\,\mathrm{m}\mathrm{s}$. The new glass would save more than any straightening could.

13.6 Tutorial: taking a route apart

13.7 Build: the fibre route model

13.8 Exercises

13.9 Problem: Thirteen Milliseconds

13.10 Interview questions