15Protocols I: FIX

An order sent this morning from a fund to its broker most likely travelled as a line of text: 35=D for a new order, 55=ESZ6 for the instrument, 54=1 for a buy, 38=5 for the quantity, each pair ended by a control character. The format was first written in 1992 so that Fidelity Investments and Salomon Brothers could exchange equity trading information electronically, and it became the common language of orders between the buy side, brokers and many venues: the Financial Information eXchange protocol. It is old, verbose, slow to parse naively, and everywhere. This chapter builds a FIX engine: the codec that reads and writes the text without copying it, the session layer that keeps two firms’ views of a conversation identical through disconnections, and the measurements that say what the protocol costs.

15.1 Tag-value encoding and the message

Three fields open every message, in a fixed order: BeginString (tag 8, the protocol version), BodyLength (9) and MsgType (35). One closes it: CheckSum (10), whose field and trailing SOH mark the end of the message. Between them come the rest of the standard header (the sender’s and target’s identifiers 49 and 56, the sequence number 34, the sending time 52 in UTC) and the body. BodyLength counts the bytes from the start of the MsgType field up to, not including, the CheckSum field, delimiters included; CheckSum is the sum of every byte before it modulo 256, written as three digits. Figure 15.1 takes apart the example that the protocol’s usual description works through, a logon whose body is 65 bytes long and whose checksum is 062; the build’s tests reproduce both numbers.

The checksum is a weak check, and knowing how weak tells a reader what it is for.

The transport underneath, TCP, already carries its own checksums on every segment. The FIX checksum and body length are therefore less a defence against corruption on the wire than a check on the framing: a program that has lost its place in a byte stream, or a message cut short or stitched together by a faulty gateway, is caught at the next message. Framing itself uses the first two fields: once 8= and 9= are read, the message’s full length is known, the body plus the seven bytes of 10=nnn and its delimiter.

15.2 The session layer: sequence numbers, heartbeats, resend and gap fill

Sequence numbers (One Quant Book 1, chapter 28, for market data) are the session layer’s whole mechanism. Each side keeps two counters: the number of its next outgoing message and the number it expects next from the other side. A message with the expected number is processed and the counter moves on. A higher number means messages were lost: the receiver sends a ResendRequest (MsgType 2) from the expected number to “infinity” (EndSeqNo 0), and holds the later messages until the gap is filled. A lower number is either a duplicate, marked with PossDupFlag (43) and ignored, or a fatal error: the two sides no longer agree, and the receiver logs out. The answer to a resend request contains the application messages again, marked PossDupFlag with their original sending time (122), and replaces administrative messages (heartbeats, test requests, resend requests themselves) by SequenceReset messages with GapFillFlag (123) set: a heartbeat from half an hour ago means nothing now. Figure 15.2 draws the golden conversation of the build around its gap, and Listing 15.1 is the same conversation as the engine traces it.

14:30:31.000 BROKER -> Heartbeat(0) seq 3
14:31:01.500 BROKER -> ExecutionReport(8) seq 6  39=2
14:31:01.500          gap expected=4 received=6
14:31:01.500 FIRM   -> ResendRequest(2) seq 3  7=4 16=0
14:31:01.510 BROKER -> ExecutionReport(8) seq 4  43=Y 39=1
14:31:01.510          deliver 4 8
14:31:01.510 BROKER -> SequenceReset(4) seq 5  43=Y 123=Y 36=6
14:31:01.510          gap fill to 6
14:31:01.510          deliver 6 8
14:31:01.510          gap closed next_in=7
14:31:01.511 BROKER -> ExecutionReport(8) seq 6  43=Y 39=2
14:31:01.511          duplicate 6 ignored
14:31:10.000 FIRM   -> OrderCancelRequest(F) seq 4
14:31:10.100 BROKER -> ResendRequest(2) seq 7  7=3 16=0
14:31:10.100 FIRM   -> SequenceReset(4) seq 3  43=Y 123=Y 36=4
14:31:10.100 FIRM   -> OrderCancelRequest(F) seq 4  43=Y

Heartbeats make silence meaningful. At logon both sides agree an interval, HeartBtInt (108), here 30 seconds. A side that has sent nothing for that long sends a Heartbeat (0). A side that has heard nothing for somewhat longer (the build waits 1.2 intervals) sends a TestRequest (1) with an identifier that the answering heartbeat must carry, and if that answer does not come within another interval it declares the connection dead. The detection time is the price of a long interval: with 30 seconds, a silently dead connection is noticed after about 66, during which the counterparty’s messages go nowhere and must be resent.

        const std::int64_t seq = m.seq();
        if (seq > next_in) {
            if (t == "2") resend(m, now);
            queue_[seq] = std::string(p, n);
            if (!resend_asked_) {
                resend_asked_ = true;
                log(now, "gap expected=" + std::to_string(next_in) + " received=" + std::to_string(seq));
                emit("2", field(7, std::to_string(next_in)) + field(16, "0"), now);
            }
            return Error::ok;
        }
        if (seq < next_in) {
            if (m.get(43) == "Y") {
                log(now, "duplicate " + std::to_string(seq) + " ignored");
                return Error::ok;
            }
            log(now, "seq " + std::to_string(seq) + " too low, expected " + std::to_string(next_in));
            state = "logout_sent";
            emit("5", field(58, "MsgSeqNum too low, expecting " + std::to_string(next_in) + " but received " +
                                    std::to_string(seq)), now);
            disconnect(now, "sequence too low");
            return Error::ok;
        }

15.3 The application layer: orders and execution reports

On top of the session, each application message carries one business event. A NewOrderSingle (35=D) carries the firm’s order identifier ClOrdID (11), the instrument (55), the side (54), the quantity (38), the order type (40) and, for a limit order, the price (44); an OrderCancelRequest (35=F) names the order to cancel by its original identifier (41) and gives the request its own new one. The answer to each is an execution report (35=8), which the specification uses “to confirm the receipt of an order”, to confirm changes, to relay status and fills and to reject orders; One Quant Book 10, chapter 26, describes it from the venue’s side. Its OrdStatus (39) walks the order lifecycle of One Quant Book 7, chapter 17: 0 new, 1 partially filled, 2 filled, 4 cancelled, 8 rejected; LastQty (32) and LastPx (31) describe the latest fill, CumQty (14) and LeavesQty (151) the totals. A partial fill of 2 of 5 contracts at 5723.25 therefore reads 39=1, 32=2, 31=5723.25, 14=2, 151=3.

The session and application layers meet in one place: resent messages. An execution report that arrives with PossDupFlag may or may not have been processed before (the original might have arrived and only its successor been lost). The application must therefore make processing idempotent, recognising a fill it has already booked by its execution identifier (17), or it books a fill twice. The session layer delivers each sequence number once; it cannot know what the application did with the original.

15.4 Parsing fast

A naive parser splits the message on SOH, converts each tag to an integer and copies each value into a map from tag to string: several allocations per field (chapter 6), a tree insertion per field, and one pass per byte. The build’s parser does none of that. It finds every delimiter of the message with the vector scan of chapter 14, parses each tag in place, and records for each field only its tag, offset and length in a fixed array: the values stay where they are, in the receive buffer, and are read as string views. Body length and checksum are verified in the same pass.

    Error parse(const char* p, std::size_t n) {
        p_ = p;
        n_ = n;
        count_ = 0;
        if (n < 5 || p[0] != '8' || p[1] != '=' || p[n - 1] != SOH) return Error::framing;
        std::uint32_t soh[kMaxFields];
        const std::size_t k = simdscan::positions(p, n, SOH, soh, kMaxFields);
        if (k == kMaxFields && soh[k - 1] != n - 1) return Error::too_many_fields;
        std::size_t start = 0;
        for (std::size_t i = 0; i < k; ++i) {
            std::uint64_t tag = 0;
            std::size_t used = 0;
            if (!simdscan::parse_uint(p + start, soh[i] - start, tag, used) || p[start + used] != '=') return Error::tag;
            fields_[count_++] = FieldRef{static_cast<int>(tag), static_cast<std::uint32_t>(start + used + 1),
                                         static_cast<std::uint32_t>(soh[i] - start - used - 1)};
            start = soh[i] + 1;
        }
        if (count_ < 4 || fields_[0].tag != 8 || fields_[1].tag != 9 || fields_[2].tag != 35 ||
            fields_[count_ - 1].tag != 10)
            return Error::order;
        for (std::size_t i = 3; i + 1 < count_; ++i)  // found by chapter 25's fuzzer: a CheckSum inside the body
            if (fields_[i].tag == 10 || fields_[i].tag == 8 || fields_[i].tag == 9) return Error::order;
        const std::size_t body_start = fields_[1].off + fields_[1].len + 1;
        const std::size_t body_end = fields_[count_ - 1].off - 3;          // first byte of "10="
        std::uint64_t declared = 0;
        std::size_t used = 0;
        simdscan::parse_uint(p + fields_[1].off, fields_[1].len, declared, used);
        if (used != fields_[1].len || declared != body_end - body_start) return Error::body_length;
        const std::string_view ck = value(count_ - 1);
        std::uint64_t c = 0;
        if (ck.size() != 3 || !simdscan::parse_uint(ck.data(), 3, c, used) || used != 3 || c != checksum(p, body_end))
            return Error::checksum;
        return Error::ok;
    }

Figure 15.3 measures both on execution reports of about 205 bytes. The zero-copy view parses one in about $80\,\mathrm{n}\mathrm{s}$, some twelve million a second on one core; the map-based parser takes about $900\,\mathrm{n}\mathrm{s}$, eleven times longer; the Python reference about $5\,\text{µ}\mathrm{s}$. Building is symmetric: the fixed-buffer builder, which writes digits itself and computes the length and checksum in place, produces a new order in about $60\,\mathrm{n}\mathrm{s}$, against about $330\,\mathrm{n}\mathrm{s}$ for the same message assembled by string concatenation.

Parsing is rarely the whole cost of a FIX connection: the session layer writes each sent message to its store, often on disk, before or after sending it, and the counterparty’s engine, network and matching take far longer than $80\,\mathrm{n}\mathrm{s}$. But on a gateway that carries a broker’s whole flow, or on a market maker’s order path through a venue that speaks only FIX, the difference between $80\,\mathrm{n}\mathrm{s}$ and a microsecond per message is the difference between a component that disappears from the latency budget of chapter 1 and one that owns part of it.

15.5 Where FIX is still used

FIX is the language of order routing between firms: from asset managers’ order management systems to brokers, from brokers’ algorithms to venues, and for direct market access (One Quant Book 1, chapter 4), where a client’s orders reach the venue through the broker’s infrastructure. Some venues accept it for order entry and drop copies, as the dated box below shows for one of them. Where speed is the product, venues have moved to binary protocols with fixed-offset fields (chapter 16), and the FIX Trading Community itself has standardised binary alternatives: Simple Binary Encoding for the messages, and FIXP, a “lightweight point-to-point protocol” for the session, whose own introduction names as its basis the FIX session layer and exchange protocols such as SoupBinTCP. The tag-value engine remains the one every trading firm needs somewhere.

15.6 Tutorial: a FIX engine in three languages

15.7 Build: the FIX engine

15.8 Exercises

15.9 Problem: The Weekend Gap

15.10 Interview questions