Python 3 hash HMAC-SHA512 [duplicate]

Question

I'm writing a bot for https://poloniex.com/support/api/

The public methods all work fine, but the Trading API Methods require some extra tricks:

All calls to the trading API are sent via HTTP POST to https://poloniex.com/tradingApi and must contain the following headers:
Key - Your API key.
Sign - The query's POST data signed by your key's "secret" according to the HMAC-SHA512 method.
Additionally, all queries must include a "nonce" POST parameter. The nonce parameter is an integer which must always be greater than the previous nonce used.
All responses from the trading API are in JSON format.

My code for returnBalances looks like this:

import hashlib
import hmac
from time import time

import requests


class Poloniex:
    def __init__(self, APIKey, Secret):
        self.APIKey = APIKey
        self.Secret = Secret

    def returnBalances(self):
        url = 'https://poloniex.com/tradingApi'
        payload = {
            'command': 'returnBalances',
            'nonce': int(time() * 1000),
        }

        headers = {
            'Key': self.APIKey,
            'Sign': hmac.new(self.Secret, payload, hashlib.sha512).hexdigest(),
        }

        r = requests.post(url, headers=headers, data=payload)
        return r.json()

trading.py:

APIkey = 'AAA-BBB-CCC'
secret = b'123abc'

polo = Poloniex(APIkey, secret)
print(polo.returnBalances())

And I got the following error:

Traceback (most recent call last):
  File "C:/Python/Poloniex/trading.py", line 5, in <module>
    print(polo.returnBalances())
  File "C:\Python\Poloniex\poloniex.py", line 22, in returnBalances
    'Sign': hmac.new(self.Secret, payload, hashlib.sha512).hexdigest(),
  File "C:\Users\Balazs91\AppData\Local\Programs\Python\Python35-32\lib\hmac.py", line 144, in new
    return HMAC(key, msg, digestmod)
  File "C:\Users\Balazs91\AppData\Local\Programs\Python\Python35-32\lib\hmac.py", line 84, in __init__
    self.update(msg)
  File "C:\Users\Balazs91\AppData\Local\Programs\Python\Python35-32\lib\hmac.py", line 93, in update
    self.inner.update(msg)
TypeError: object supporting the buffer API required

Process finished with exit code 1

I've also tried to implement the following, but it didn't help: https://mcmap.net/q/1328655/-hash_hmac-sha512-authentication-in-python

Any help is highly appreciated!

Paillette

Your payload in returnBalances is a dict, which does not support the buffer API (read: a cheap bytes-like representation). You probably want `payload = str(dict(command='returnBalances', nonce=...))' as the string representation of a dict like that happens to equal the JSON-representation; it's bytes-encoded form can be sent to to HMAC. – Holocaine

@Holocaine The string representation of a Python dict isn't a valid query string, so that's not going to work. – Hyperphysical

Answer 1

The payload you pass to requests.post has to be either a valid query string or a dict corresponding to that query string. Normally, it's more convenient just to pass a dict and get requests to build the query string for you, but in this case we need to construct an HMAC signature from the query string, so we use the standard urlib.parse module to build the query string.

Annoyingly, the urlib.parse.urlencode function returns a text string, so we need to encode it into a bytes string in order to make it acceptable to hashlib. The obvious encoding to use is UTF-8: encoding a text string that only contains plain ASCII as UTF-8 will create a byte sequence that's identical to the equivalent Python 2 string (and of course urlencode will only ever return plain ASCII), so this code will behave identically to the old Python 2 code on the Poloniex API page you linked.

from time import time
import urllib.parse
import hashlib
import hmac

APIkey = b'AAA-BBB-CCC'
secret = b'123abc'

payload = {
    'command': 'returnBalances',
    'nonce': int(time() * 1000),
}

paybytes = urllib.parse.urlencode(payload).encode('utf8')
print(paybytes)

sign = hmac.new(secret, paybytes, hashlib.sha512).hexdigest()
print(sign)

output

b'command=returnBalances&nonce=1492868800766'
3cd1630522382abc13f24b78138f30983c9b35614ece329a5abf4b8955429afe7d121ffee14b3c8c042fdaa7a0870102f9fb0b753ab793c084b1ad6a3553ea71

And then you can do something like

headers = {
    'Key': APIKey,
    'Sign': sign,
}

r = requests.post(url, headers=headers, data=paybytes)

Hyperphysical

Thank you very much, it works now! There was a little typo in your code, last line should be: r = requests.post(url, headers=headers, data=payload) – Oscilloscope

@BalázsMagyar That's not a typo! We should be sending the exact query string paybytes that corresponds to sign. If you pass the payload dict to requests.post there's the chance that it will put the query parameters in a different order, because dicts don't have an inherent order. And that would mean that the HMAC calculated by the server for that query string wouldn't match the one we send in sign. – Hyperphysical

@BalázsMagyar OTOH, it shouldn't make a difference whether you pass requests.post the text version of the query string (i.e., urllib.parse.urlencode(payload)) or the UTF-8 encoded version, but I think it's cleaner to pass it a bytes string rather than a Unicode string. – Hyperphysical

I see your point, however with data=paybytes I get the following error: {'error': 'Invalid command.'} – Oscilloscope

@BalázsMagyar Weird. How about if you send the text string instead of the bytes string, like I mentioned in the previous comment, the output of urllib.parse.urlencode(payload)? If you're running on Python 3.6 then dicts do retain their insertion order, but it's currently an implementation detail and we're not supposed to write software that relies on it. So I'm really not comfortable with you passing the payload as a dict. – Hyperphysical

Just tried the text string as you mentioned, but i got the same error. It's pretty interesting, I'll do some research about that problem. Actually the order of insertion doesn't matter, because first I create the payload dict, then I calculate the hash from that, then I pass the payload as a dict. If the order is different, then the HMAC calculation will be different too, which is good. – Oscilloscope

@BalázsMagyar Good point. Since you aren't modifying payload after the HMAC is calculated, requests.post will get the payload items in the same order that hmac.new got them. So everything should be fine and I was worrying needlessly. ;) Still, it'd be nice to know why it doesn't work like it should when you send paybytes. – Hyperphysical

FWIW, I was just reading the requests docs, and the data arg is supposed to be a dict, a bytes string, or a file-like object. You shouldn't pass it a plain text string in Python 3. – Hyperphysical

Great, now everything is clear. Thank you very much again your time and help! – Oscilloscope