How to send FIX logon message with Python to GDAX/Coinbase

Question

I'm trying to establish a FIX 4.2 session to fix.gdax.com (docs: https://docs.gdax.com/#fix-api or https://docs.prime.coinbase.com/?python#logon-a) using Python 3.5 and stunnel. Everything is working apart from my logon message which is rejected and the session is closed by the server with no response making it difficult to debug what's going wrong. My Python code is as follows:

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 4197)) # address and port specified in stunnel config file

# generate a signature according to the gdax protocol for signing a message:
timestamp = str(time.time())
message   = [timestamp, "A", "0", "f3e85389ffb809650c367d42b37e0a80", "Coinbase", "password-goes-here"] # these are the components of the pre-hash string as specified in the docs for a logon message
message   = bytes("|".join(message), 'utf-8') # add the field separator

hmac_key  = base64.b64decode(r"api-secret-goes-here")
signature = hmac.new(hmac_key, message, hashlib.sha256)
sign_b64  = base64.b64encode(signature.digest()).decode()
# in the above line the .decode() is not included when used to authenticate messages to the REST API and those are working successfully.
#The reason I've included it here is to allow a string to be passed into the variable 'body' below:

msgType    = "A"
t          = str(datetime.utcnow()).replace("-","").replace(" ", "-")[:-3] # format the timestamp into YYYYMMDD-HH:MM:SS.sss as per the FIX standard

body       = '34=1|52=%s|49=f3e85389ffb809650c367d42b37e0a80|56=Coinbase|98=0|108=30|554=password-goes-here|96=%s|8013=Y|' % (t, sign_b64)
bodyLength = len(body.encode('utf-8')) # length of the message in bytes
header     = '8=FIX.4.2|9=%s|35=%s|' % (bodyLength, msgType)
msg        = header + body

# generate the checksum:
def check_sum(s):
    sum = 0
    for char in msg:
        sum += ord(char)
    sum = str(sum % 256)
    while len(sum) < 3:
        sum = '0' + sum
    return sum

c_sum = check_sum(msg)
logon = msg + "10=%s" % c_sum # append the check sum onto the message
logon = logon.encode('ascii') # create a bytes object to send over the socket
print(logon)

s.sendall(logon)
print(s.recv(4096))

The results of those two print statements are:

b'8=FIX.4.2|9=159|35=A|34=1|52=20171104-11:13:53.331|49=f3e85389ffb809650c367d42b37e0a80|56=Coinbase|98=0|108=30|554=password-goes-here|96=G7yeX8uQqsCEhAjWDWHoBiQz9lZuoE0Q8+bLJp4XnPY=|8013=Y|10=212'
b''

There are a lot of variables here that could be wrong and the process of trial and error is getting a bit tedious. Can anyone see what is wrong with the logon message?

Aforetime

It looks like you're using | as a field separator. The FIX field separator is ASCII character 1 (I believe that's \x01 in Python), therefore your messages don't follow the FIX protocol, which may explain the lack of response. I'd highly recommend using an existing FIX library rather than trying to implement your own. – Chinook

@Chinook I have tried that as well, to no avail :( I'll try a library instead, thanks! – Aforetime

I have almost the same code as yours, and get a result with a msgSeqNum (34) value > 1. However, it still does not logon. – Matos

Answer 1

I made some modifications to your code and put comments where it differs from yours (corrects your errors):

import socket
import base64
import time, datetime
import hmac
import hashlib

PASSPHRASE = "your passphrase"
API_KEY = "your api key"
API_SECRET = "your secret"

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 4197))

seq_num = "1" # Correction: using the same MsgSeqNum for signed text and for the field 34


# Correction: t is the same in both signed RawData and in SendingTime (52)
timestamp = str(time.time())
t = str(datetime.datetime.utcnow()).replace("-","").replace(" ", "-")[:-3]

# Correction: '|' is not a valid separator for FIX, it must be '\u0001'
message   = "\u0001".join([t, "A", seq_num, API_KEY, "Coinbase", PASSPHRASE]).encode("utf-8")

hmac_key  = base64.b64decode(API_SECRET)
signature = hmac.new(hmac_key, message, hashlib.sha256)
sign_b64  = base64.b64encode(signature.digest()).decode()

msgType = "A"

body = "34={}|52={}|49={}|56=Coinbase|98=0|108=30|554={}|96={}|8013=Y|".format(seq_num, t, API_KEY, PASSPHRASE, sign_b64) # using the same time (t) and seq_num as in signed text

# Correction: bodyLength is the number of characters, not bytes, also it must include everything after "8=FIX.4.2|9={}|" i.e. the "35=A|" part of the header
bodyLength = len("35={}|".format(msgType)) + len(body)
header     = "8=FIX.4.2|9={}|35={}|".format(bodyLength, msgType)
msg        = header + body

msg = msg.replace('|', '\u0001') # Correction: '|' is not a valid separator for FIX, it must be '\u0001'

# generate the checksum:
def check_sum(s):
    sum = 0
    for char in msg:
        sum += ord(char)
    sum = str(sum % 256)
    while len(sum) < 3:
        sum = '0' + sum
    return sum

c_sum = check_sum(msg)

logon = msg + "10={}\u0001".format(c_sum)
logon = logon.encode('ascii')
print(logon)

s.sendall(logon)
print(s.recv(4096))

For me this corrected code now returns the Logon message from the server instead of just 0 bytes as it was in your case. Can you confirm that it also works for you and that you can successfully send other transactions after logon is done?

Tishtisha

There is still an error in this script - python interprets 'u\0001' as a 6-character string. You need a u in front of the string (u'\u0001') to get the single-char separator that FIX requires. Easier yet would be to convert to hex notation '\x01' which works in non-unicode contexts - at that point, I think none of the ascii/utf-8 encoding calls would be necessary. – Micmac

Thanks for noting that. I forgot to mention that I tested my code in Python 3 where strings are unicode by default. And since they are unicode by default, I think, one always needs to encode/decode when sending/receiving strings via sockets. In Python 2, probably, it's not necessary, but I didn't test it. – Tishtisha

Ah, thanks for the comment. I'm on python 2, so that must be the difference. – Micmac

How does this connect to GDAX in the first place? The socket is only connected to the localhost. – Alkalify

Answer 2

Nothing new to add just wanted to rephrase the above solution in a more function based way without tunneling:

import socket
import base64
import time, datetime
import hmac
import hashlib
import ssl

host = 'fix.gdax.com'
#sandbox_host = 'fix-public.sandbox.gdax.com'
port = 4198
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssl_sock = context.wrap_socket(s, server_hostname=host)
ssl_sock.connect((host, port))

def check_sum(s):
    sum = 0
    for char in s:
        sum += ord(char)
    sum = str(sum % 256)
    while len(sum) < 3:
        sum = '0' + sum
    return sum

def sign(t, msg_type, seq_num, api_key, password, secret):
    message   = "\x01".join([t, msg_type, seq_num, api_key, "Coinbase", password]).encode("utf-8")
    hmac_key  = base64.b64decode(secret)
    signature = hmac.new(hmac_key, message, hashlib.sha256)
    return base64.b64encode(signature.digest()).decode()

def wrap_fix_string(msg_type, body):
    bodyLength = len("35={}|".format(msg_type)) + len(body)
    header     = "8=FIX.4.2|9=00{}|35={}|".format(bodyLength, msg_type)
    msg        = header + body
    return msg

def generate_login_string(seq_num, t, api_key, password, secret):
    msgType = "A"
    sign_b64 = sign(t, msgType, seq_num, api_key, password, secret)
    body = f"49={api_key}|554={password}|96={sign_b64}|8013=S|52={t}|56=Coinbase|98=0|108=30|34={seq_num}|9406=N|" # using the same time (t) and seq_num as in signed text    
    msg = wrap_fix_string(msgType, body)
    msg = msg.replace('|', '\x01')
    c_sum = check_sum(msg)
    return msg + "10={}\x01".format(c_sum)    

PASSPHRASE = "your passphrase"
API_KEY = "your api key"
API_SECRET = "your secret"
seq_num = "1"
t = str(datetime.datetime.utcnow()).replace("-","").replace(" ", "-")[:-3]
logon =  generate_login_string(seq_num, t, API_KEY, PASSPHRASE, API_SECRET)
logon = logon.encode('ascii')
print(f'logon: {logon}')

ssl_sock.sendall(logon)
print('GETTING')
print(ssl_sock.recv(4096))

Frayne

You're missing this line to create the context: context = ssl.create_default_context() – Kenlee

Answer 3

sudo yum install python3-devel

nohup pip3 install quickfix &

Then you will be able to use quickfix with flexible functions like auto-heartbeat, auto-reconnect, resend, rewind, etc. These functions will make your application more stable.

Goldia