Are Decimal 'dtypes' available in NumPy?

Question

Are Decimal data type objects (dtypes) available in NumPy?

>>> import decimal, numpy
>>> d = decimal.Decimal('1.1')
>>> s = [['123.123','23'],['2323.212','123123.21312']]
>>> ss = numpy.array(s, dtype=numpy.dtype(decimal.Decimal))
>>> a = numpy.array(s, dtype=float)
>>> type(d)
<class 'decimal.Decimal'>
>>> type(ss[1,1])
<class 'str'>
>>> type(a[1,1])
<class 'numpy.float64'>

I suppose numpy.array doesn't support every dtype, but I sort of thought that it would at least let a dtype propagate as far as it could as long as the right operations were defined. Am I missing something? Is there some way for this to work?

Polybius

There are three answer below at the moment I am writing this and all are useful. I've marked one as correct. Depending on what you want, it seems that you can first 'cast' the array as dtype=float and then dtype=Decimal and it will function properly. If you need to go from string to Decimal, then I think it's an element wise conversion that is needed. – Polybius

Before reading all that, ask to yourself "Do I really need more than, let's say, 10 digits precision ?". If not, float64 is good and you can avoid most of workaround and, as underlined here, you'll take full advantage of numpy. – Menswear

This is the subject of a meta question. – Subclavian

Answer 1

Important caveat: this is a bad answer

You would probably do best to skip to the next answer.

---

It seems that Decimal is available:

>>> import decimal, numpy
>>> d = decimal.Decimal('1.1')
>>> a = numpy.array([d,d,d],dtype=numpy.dtype(decimal.Decimal))
>>> type(a[1])
<class 'decimal.Decimal'>

I'm not sure exactly what you are trying to accomplish. Your example is more complicated than is necessary for simply creating a decimal NumPy array.

Gudrun

You're just creating an array with dtype dtype('object') here; NumPy doesn't know anything special about the Decimal type. – Baldric

The performance difference is about 100 times slower: %timeit(float_arr * 2) 1.34 µs per loop, %timeit(dec_arr * 2) 141 µs per loop. (given float_arr = np.array([1.1] * 1000); from decimal import Decimal; dec_arr = np.array([Decimal(1.1)] * 1000, dtype=np.dtype(Decimal))) – Leanaleanard

p.s. using numpy==1.10.4 – Leanaleanard

Answer 2

NumPy doesn't recognize decimal.Decimal as a specific type. The closest it can get is the most general dtype, object. So when converting the elements to the desired dtype, the conversion is a no operation.

>>> ss.dtype
dtype('object')

Keep in mind that because the elements of the array are Python objects, you won't get much of a speedup using them. For example, if you try to add this to any other array, the other elements will have to be boxed back into Python objects and added via the normal Python addition code. You might gain some speed in that the iteration will be in C, but not that much.

Leatrice

Answer 3

Unfortunately, you have to cast each of your items to Decimal when you create the numpy.array. Something like

s = [['123.123','23'],['2323.212','123123.21312']]
decimal_s = [[decimal.Decimal(x) for x in y] for y in s]
ss = numpy.array(decimal_s)

Capetian