How FIX protocol works Monday March 7th, 2022 – Posted in: Forex trading

Introduction

The history of trading in stock markets has changed dramatically since its inception. The first stock exchanges appeared about four hundred years ago. Back then, deals were done verbally, trading rules were just beginning. But the world did not stand still, and, in keeping with the times, stock exchanges took on every possible technical tool that could have been available at the time.

Thus, in the mid-19th century, trade applications at some distance from the exchange connected telegraph lines. Then the telephone replaced him. And eventually came the moment when the first trading requests of traders ran through the communications of the Internet.

In fairness, trading through the phone sometimes still occurs in modern times at some brokers. And a certain part of them is the only guaranteed way of making a transaction. Also from the beginning of the first exchanges up to the beginning of the 21st-century transactions directly on the territory of exchanges transactions were carried out orally. Such trading floors were called stock pits and represented a spectacle familiar to us in Hollywood films on similar subjects.

FIX protocol History

One of the oldest digital trading protocols is the FIX protocol. Stands for “Financial Information Protocol” or “FInancial eXchange protocol”, and both definitions are found. The FIX Protocol Specification was created in 1992 to provide information on share trading between Fidelity Investments and Salomon Brothers. Brokers and investment funds wanted to speed up the process of trading on the exchange. The result is an open standard for transmitting information electronically that no major organization controls. Today, the FIX has become the industry standard used by financial market participants in different countries to connect their products.

Initially, this communication protocol was used only between the two exchanges mentioned above, was called SBX (Salomon Brothers Exchange), and was used only for stock trading. Then, as the story goes, the leaders of these companies, realizing the importance of their invention, offered to join the development of Goldman Sachs and Putman. The protocol has since been given its current name.

Initially, the protocol supported only 17 message types and 103 tags. The idea was so successful that, by 1995, more than 70% of all US brokerages used the protocol. This was due to cost savings, lower costs, and lower trading errors. After all, the human factor has a very big influence when trading through the phone. Over time, the protocol has expanded significantly. Support for derivatives, bond, and currency trading was added. The current FIX specification describes 168 messages, 7,868 tags, and covers all classes of securities. In 1998, the development of the FIX protocol and its specifications, as well as the development of related technologies, was transferred to the FIX Protocol Ltd consortium. The consortium was later renamed FIX Trading Community. More than 270 leading financial institutions are now members of the organization.

How FIX protocol works

The FIX protocol runs over TCP. Currently, the FIX protocol model is defined at two levels – session or control level (session creation, data delivery work), and application-level (data content description). The control layer is responsible for the basic parameters of the FIX session: open and close the connection, recover lost messages. The application layer controls the sending and receiving of data: Requests, executions, rejections, market data, and requests for status information. The protocol itself is text, that is, it uses characters from the ASCII table. It is divided into 2 representations – traditional, of the form Tag=Value(Tag=Value or Key=Value), and in the XML(FIXML) representation. The latest version is currently 5.0, but the most common is version 4.2 and 4.4. Different brokers and exchanges can use different versions of the protocol, and sometimes several at the same time.

Syntax Tag=Value

The message is similar to the following:

8=FIX.4.2 | 9=178 | 35=D | 34=1234567 | 49=TESTER | 56=PHLX | 52=20071123-05:30:00.000 | 11=ATOMNOCCC9990900 | 54=1 | 167=FUT | 55=MSFT | 38=15 | 40=2 | 44=15 | 59=0 | 10=128 |

This is a text string that consists of some parts separated by a vertical bar. These parts are called fields. In turn, each of the parts consists of a key-value pair separated by a sign equal. To the right of the sign is the key, to the left is the value. In the FIX specification, keys are called tags. A tag is always a positive integer that is essentially a pointer to a field name. All field names and their possible values are described in the documentation for a particular exchange. Most of these fields are standard and can be found in the BOM of any exchange. There are also custom fields, which are defined and have meaning only within a particular exchange. If the description of standard fields can be found in the general FIX protocol specification, the custom fields must be described in the exchange FIX protocol specification. There are also required, optional, and conditionally-required fields (required depending on the presence of some other fields in the message). The SoH (Start of Header) symbol is used as a separator between the fields. It is actually not visible, so it is replaced with a vertical bar in this example. In the UNICODE encoding, this character has the code “\u0001”.

In turn, while the message is divided into fields, they, in turn, define 3 more components:

Header (Message Header)
Body (Message Body)
Message Trainer (checksum)

In FIX1.1/FIX5.0, the header contains 5 required fields and one optional field: 8 (BeginString), 9 (BodyLength), 35 (MsgType), 49 (SenderCompID), 56 (DestComppID), and 1128 (ApplVerID – if present, must be at 6th position). There are two main groups of messages – administrative and applied. Admin messages handle the basics of a FIX session. They allow you to start and end a session, as well as restore missed messages. The application messages are related to the sending and receiving of trade-related information, such as a request for a warrant or information about the current status and subsequent execution of that warrant.

We will analyze this message:

8=FIX.4.2 – BeginString. The protocol version is used. Tells the trader program which rules to apply to parse the fields of this message.
9=178 – BodyLength. The message body length is 178 bytes, including the header. Calculated as the number of characters from tag 35 (inclusive) to tag 10 (exclusive). The SoH splitter is also considered.
35=D – MsgType. Message type. In this case, D means placing a trade request. For example, a field of 35=V indicates that market data is obtained about the instrument.
34=1234567 – MsgSeqNum. Message number. If MsgSeqNum messages are not 1 different in the sequence sent, the server returns an error and does not process the message.
49=TESTER – SenderCompID. Sender ID. Issued by the exchange. This can be a username, or a broker’s name if the message is sent by a broker.
56=PHLX – TargetCompID. Recipient ID to send to. Also issued by the exchange.
52=20071123-05:30:00.000 – SendingTime. Date and time when the message was sent.
11=ATOMNOCCC9990900 – ClOrdID. Application number in the broker trading system.
54=1 – Side. Asset purchase action
167=FUT – SecurityType. The asset is a future
55=MSFT – Symbol. Microsoft Promotions. The tickers may vary from exchange to exchange.
38=15 – OrderQty. The deal volume is 15 lots or contracts.
40=2 – OrdType. Limited Price Purchase, Limited Order.
44=15 – Price. Purchase price – 15
59=0 – TimeInForce. The deadline for the application is the end of the working day.
10=128 – CheckSum. Message checksum. Always the last field in the message. It is calculated using special formula and described in the protocol specification. It is set by summing the ASCII values of all characters in the message except the checksum field characters. The value is then divided by 256 and the remainder of the compartment is taken. The checksum must always be three characters long. So, if the modulus is 22, then zero – “022” is added before the number.

If you describe this message in general, it is an action to buy Microsoft stock futures in the number of 15 contracts at the capped price of 15, with the expiration date to the end of the day.

To provide more flexibility in the FIX, the protocol contains what is known as user-defined fields, User Defined Fields. They are used for data transfer between cooperating financial organizations. Tag numbers from 5000 to 9999 were reserved for custom fields, which could be reserved on the official website of the standard. These numbers were subsequently used, so a new interval was allocated, from 20,000 to 39,999.

FIXML syntax

Work on the syntax began in 1998, and the first version was released in 1999. It is semantically equivalent to messages encoded with tags, but uses XML parsing technology. The new application for the operation in the format FIXML is as follows:

<FIXML>

<Order ClOrdID=”123456″ Side=”2″ TransactTm=”2001-09-11T09:30:47-05:00″ OrderTyp=”2″ Px=”93.25″

Acct=”26522154″>

</Order>

</FIXML>

In XML format, a specific block of code is framed in a so-called tag (another tag, not the one used by the protocol itself). For example, there is a <FIXML> top-order tag. Any tag breaks off a specific piece of code or message and closes it with the same tag, but inside the left edge, there is a slash character. Thus, the mentioned tag will close the block as follows: </FIXML>.

Next, within this tag, we see the following tag:

<Order ClOrdID=”123456″ Side=”2″ TransactTm=”2001-09-11T09:30:47-05:00″ OrderTyp=”2″ Px=”93.25″

Acct=”26522154″>.

Here we see the tag name – Order, and inside the braces, there is additional information (the same fields, key=value pair as the syntax Tag=Value).

In this case:

ClOrdID=”123456″ – order id in the broker trading system

Side=”2″ – Sales Order

TransactTm=”2001-09-11T09:30:47-05:00″ date and time of transaction

OrdTyp=”2″ – Claim Type – Limit Order

Px=”93.25″ – The price of an asset to buy or sell

Acct=”26522154″ – user account number

Like the previous XML tag, the XML tag is closed with a slash – </Order>

The content between the opening and closing XML tag is called the tag body. Thus, the Order tag is the body of the FIXML tag. Inside the Order tag as a body, there are two more tags: Instrmt and OrdQty.

Next tag:

Here we can see that it is both closing and closing at the same time, at the end of it there is an oblique line. This is the case if the tag has nobody, and there is no point in making a separate framing tag. This tag contains information about the ticker of the tool, its id, etc.

In the <OrdQty Qty=”1000″/> tag we can see information about the volume of the transaction, it is also the closing tag without a body.

FIXML is typically used for back-office and clearing applications, not for trading.

FIX session layer

Previous examples have described the application layer. The specification defines special field types that are responsible for the session-level between the client and the exchange:

35=A – Logon. Used to authenticate the user to the exchange system when connecting. This message is sent first, and it signals the exchange to the beginning of the data session. A reply message arrives on successful submission. If the connection fails, an error is returned.
35=5 – Logout. This message is used to deauthorize with the server and close the trading session with the exchange.
35=0 – Heartbeat. Heart rate, or pulse. This message is sent from both sides of the connection to each other and indicates that both sides are ready to receive and send messages. Heartbeat frequency is set in the first Logon message.
35=1 – Test Request. This message is a test message and is sent when the counterparty has not sent a heartbeat message within a set period. The session will be closed if this message remains unanswered.
35=2 – Resend Request. This message is a request to resend the message if any information has been missed.
35=3 – Reject, or reject. Sent by the counterparty if the previous message is malformed or cannot be executed.
35=4 – Sequence Reset, or reset the message sending sequence. It can have two forms.

– 123=”Y” – There is a GapFillFlag tag with the specified value. Indicates a request to ignore administrative messages if they are being sent again.

– used to zero the MsgSeqNum counter

Implementing FAST as an Enhancement to FIX API

Large amounts of FIX data were subject to significant delays in processing them, making traders less likely to develop effective trading strategies. Soon after realizing such a problem, the first steps toward correcting the situation were taken. The goal of the FAST protocol was to allow large amounts of data to be transmitted, avoiding delays in obtaining information. FAST (FIX Adapted for STreaming, which translates to FIX, which is adapted for streaming) is a technology standard developed by FIX Protocol Ltd. that is specifically designed to optimize network presentation. It is essentially a binary format of the FIX protocol and allows for the transfer of large amounts of information about transactions and the market environment in a compact form. This allows its use in high-speed trading systems requiring low transmission delays. Fast was developed at the California Institute of Technology in Pasadena. Currently, the latest version of FAST 1.2. The most common application – connection directly to exchanges, bypassing the trading system of brokers. The FAST protocol is the standardized market information distribution protocol and the most popular in the world. Previously, all Internet traffic was based on the Transmission Control Protocol (TCP) system, developed in the seventies of the last century. TCP is a socket connection that requires acknowledgment of packet delivery. It breaks down large files and messages into 1500-byte packets, each containing the source and destination addresses of the packet. The sender waits for confirmation that the sent packet was successfully delivered, and only then sends the next one. If the delivery fails or the delivery signal does not arrive within the specified timeout, the packet is sent again, but at a lower rate. And it can go on as long as you like, with the velocity going down exponentially, until the signal that the delivery is successful comes. It becomes apparent that even minor line problems can severely affect the data rate, as each failed send is a loss of precious milliseconds. Therefore, FAST introduced a workaround – the system constantly monitors the packets and received messages for good delivery in order to detect problems on the line immediately and set the next send to a stable, albeit slightly reduced rate. This approach eliminates the need to resend packets in most cases, as it does not take time to probe for slow performance. In some exchanges, the connection interface is implemented through two types of connection: UDP and TCP. Sending all data on the UDP socket makes no sense, as such connection does not require confirmation of delivery, and the packets will simply disappear, and neither party will know about it. Therefore, most of the data is transmitted over the UDP socket, and only the sets of undelivered data are transmitted over TCP. In fast-moving markets, the delay associated with retransmission is often undesirable, leading to missed opportunities or bad deals.

How FAST works

As we already know, the FIX protocol is a sequence of fields that in turn consist of key=value pairs. In addition to the fact that the string format is essentially redundant, you may notice that each message has duplicate elements, such as field names and “=” characters. FAST eliminates redundancy by using a template that describes the entire message structure. This method is called implicit tagging. Since the FIX tags in the transmitted data are only “implied”, the syntax of Tag = Value changes as follows:

A template describes a structured set of fields with their operators;
The field sequence in the message matches the tag sequence in the template;
Only data changes are sent in the message.

Each tag value has a fixed length in bytes. Knowing the length of a specific field and the order in which the fields are placed in a message, it is not difficult to identify the values of these fields in a message. Therefore, templates allow only the data itself to be transmitted directly, without the need to transfer redundant characters. This encoding is called simple binary encoding, or SBE. Thus, message encoding and decoding have much lower latency than character protocols, because it is not necessary to translate data into a format that computers can use. Since the corresponding fields are in a fixed position in the message, it is much easier to access the desired value. You only need to know its offset relative to the beginning of the message, and the length of its value. Unlike the tag and FIXML, an SBE message is not self-describing. Only data with a minimal header is sent over the network to identify the template that controls the message. Metadata (templates) describing the message structure is exchanged outside the main communication channel. After receiving the necessary templates, the trading server knows in which format to send data to the client within the framework of this connected session. Thus, before transferring data, the systems somehow agree with each other on how to communicate with them in the future. The message schema is typically sent in XML format. It can contain any number of message templates. The template describes the fields that compose the message. In addition, the schema provides a list of simple and complex data types that can be reused in any number of fields.

Open implementations for different languages for the FIX protocol

Java – QuickFIX/J – http://www.quickfixj.org/

.NET/C# – QuickFIX/N – http://quickfixn.org/

Free on-line parser of FIX-messages – FIX Parser – https://fix.aprics.net/

Open Language Implementations for FAST

For C – Reference implementation from FPL – www.fixprotocol.org/fastdownload

For C# – Reference implementation from FPL – www.fixprotocol.org/fastdownload

Java л д code – OpenFAST – www.openfast.org

www.sourceforge.net/projects/openfastdotnet/ Д я

Для C++ – QuickFAST – www.quickfast.org

Для Golang – goFAST – www.github.com/co11ter/goFAST

Other Protocol FIX Encodings

The FIX community also developed standard mappings between the FIX protocol and other data serialization protocols such as:

JSON
Google (Protocol buffers)

|ASN.1

FIX/FAST Protocol Application Areas

As a rule, an abbreviation occurs exactly together of two words. Used to obtain various financial information from trade exchanges: Tables of financial instruments (prices, volumes, etc.), quotations, all transactions, indices, as well as information about all impersonal applications. The set of data obtained may differ depending on the exchange. The primary FIX/FAST is:

Private traders for trading
Programmers create products based on this technology. Such as – FIX API Trading platforms, connectors to different exchanges, trading systems, data collection systems, and statistics.
Exchange of information between exchanges
Obtaining the most up-to-date data for various information agencies

The most common use is for high-frequency traders to gain an advantage over other market players via Direct Market Access (DMA). This protocol can be incorporated into the trading ecosystem by stock exchanges, banks, and forex brokers. Cryptocurrency exchanges, according to commonly known data, do not use this technology. Historically, these exchanges mostly use WebSocket connections to obtain market information and HTTP rest API connections to conduct trading operations to provide direct connections to traders’ trading servers. If the WebSocket technology is still comparable in speed to a TCP connection, then HTTP rest API loses a lot.