Why doesn't Python hash function give the same values when run on Android implementation?

Question

I believed that hash() function works the same in all python interpreters. But it differs when I run it on my mobile using python for android. I get same hash value for hashing strings and numbers but when I hash built-in data types the hash value differs.

PC Python Interpreter (Python 2.7.3)

>>> hash(int)
31585118
>>> hash("hello sl4a")
1532079858
>>> hash(101)
101

Mobile Python Interpreter (Python 2.6.2)

>>> hash(int)
-2146549248
>>> hash("hello sl4a")
1532079858
>>> hash(101)
101

Can any one tell me is it a bug or I misunderstood something.

Dolomite

I don't know why it differs for hash() but maybe you could use base64 instead: docs.python.org/2/library/base64.html – Derryberry

@rednaw thanks, but I just want to know is it normal to have different hash values. – Dolomite

You should never rely on the hash value being constant between different interpreters. There's nothing in the spec to guarantee that behavior. the only guarantee is that the hash value will always be the same on a particular run of a particular interpreter. – Stonehenge

Is this a duplicate of #794261 ? – Hekate

Answer 1

for old python (at least, my Python 2.7), it seems that

hash(<some type>) = id(<type>) / 16

and for CPython id() is the address in memory - http://docs.python.org/2/library/functions.html#id

>>> id(int) / hash(int)                                                     
16                                                                              
>>> id(int) % hash(int)                                                 
0                                                                               

so my guess is that the Android port has some strange convention for memory addresses?

anyway, given the above, hashes for types (and other built-ins i guess) will differ across installs because functions are at different addresses.

in contrast, hashes for values (what i think you mean by "non-internal objects") (before the random stuff was added) are calculated from their values and so likely repeatable.

PS but there's at least one more CPython wrinkle:

>>> for i in range(-1000,1000):
...     if hash(i) != i: print(i)
...
-1

there's an answer here somewhere explaining that one...

Ferule

Yes I accept with you but how hash will work for non-internal objects. – Dolomite

In android >>> id(int) / hash(int) gives -2L and >>> id(int) % hash(int) gives -2144680448L – Dolomite

@andrewcooke wow hash(-1)=-2 really irritated me. In case someone is wondering, the question regarding it is here: #10130954 – Epigynous

Answer 2

hash() is randomised by default each time you start a new instance of recent versions (Python3.3+) to prevent dictionary insertion DOS attacks

Prior to that, hash() was different for 32bit and 64bit builds anyway.

If you want something that does hash to the same thing every time, use one of the hashes in hashlib

>>> import hashlib
>>> hashlib.algorithms
('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')

Banwell

For strings ... I'm not sure that they've added that randomization to integers or other builtin types .. (but I was about to make this point as well) – Stonehenge

But, none of the hashlib algorithms hashes data types. – Dolomite

How can you convert them to strings? pickle perhaps? – Banwell

The hash functions in hashlib are cryptographic and due to performance are not a good solution in all situations. – Erda

Answer 3

Hashing of things like int relies on id(), which is not guaranteed constant between runs or between interpreters. That is, hash(int) will always produce the same result during a program's run, but might not compare equal between runs, either on the same platform or on different platforms.

BTW, while hash randomization is available in Python, it's disabled by default. Since your strings and numbers are hashing equally, clearly it's not the issue here.

Horvitz

Hash randomization is only disabled by default on old versions of Python. For Python 3.3 and later it is enabled by default. – Underpass

@Horvitz Why python returns the same hash value for objects of same value but different id's? I would expect x='hello world' and y='hello world' which have different id's to have different hash values. – Orison

@adosar "just hash the id" is the default behavior of hash for things like Object and type because there's no more generally useful default behavior. For things which can have their value hashed in a reasonable fashion (like strings and integers) it hashes the value, not the identity. – Horvitz

Remember, hash hashing strings by value is the only reason you can reliably do something like mymap["foo"] = 3 and expect to access the value at mymap["foo"] later. – Horvitz

@Horvitz In the case mymap["foo"] = 3 why the value must be used? For example, if a dictionary has a key "foo" then the id is unique, so why the value must be used instead? I mean the key is not variable that maybe later point to some other object. – Orison

Becuase the literal string "foo" which you use on line 10 is not (guaranteed to be) the same object as the literal string "foo" which you use on line 20. If strings were not hashed by value, each instance of "foo" in your code could have a different hash. – Horvitz

Answer 4

With CPython, for efficiency reason hash() on internal objects returns the same value as id() which in its turn return the memory location ("address") of the object.

From one CPython-based interpreter to an other memory location of such object is subject to change. Depending on your OS, this could change from one run to an other.

Knp

This is not true anymore, see #11324771. – Fireback

Answer 5

From Python 3.3 the default hash algorithm has created hash values which are salted with a random value which is different even between different python processes on the same machine.

Hash randomization only is implemented currently for strings - since it was considered to be the most likely data type captured from outside that could be attacked.

The same frozenset consistently produces the same hash value across different machines or even different processes

Source: https://www.quora.com/Do-two-computers-produce-the-same-hash-for-identical-objects-in-Python

Rosie