Count all values in a matrix less than a value

Question

I have to count all the values in a matrix (2-d array) that are less than 200.

The code I wrote down for this is:

za=0   
p31 = numpy.asarray(o31)   
for i in range(o31.size[0]):   
    for j in range(o32.size[1]):   
        if p31[i,j]<200:   
            za=za+1   
print za

o31 is an image and I am converting it into a matrix and then finding the values.

Is there a simpler way to do this?

Explication

Doesn't this just print the number of values that are less than 200, and not the actual values? – Rhesus

Yes.. I just need the total of all the values instead of the actual values themselves. – Explication

You aren't getting the total here either. Set za to an empty list za = [], then za.append(p31[i,j]), finally out of your for loop, print sum(za); but I'm sure there is a better way since you are using numpy. – Rhesus

Answer 1

The numpy.where function is your friend. Because it's implemented to take full advantage of the array datatype, for large images you should notice a speed improvement over the pure python solution you provide.

Using numpy.where directly will yield a boolean mask indicating whether certain values match your conditions:

>>> data
array([[1, 8],
       [3, 4]])
>>> numpy.where( data > 3 )
(array([0, 1]), array([1, 1]))

And the mask can be used to index the array directly to get the actual values:

>>> data[ numpy.where( data > 3 ) ]
array([8, 4])

Exactly where you take it from there will depend on what form you'd like the results in.

Terminal

Thanks... But I need the total number of values, not the values themselves.. should I do sum(numpy.where(data<200))? – Explication

If you want the number OF values, not the sum, you would do len(numpy.where(data<200)) – Intranuclear

if condition changed to > 0, then it will return (array([0, 0, 1, 1]), array([0, 1, 0, 1])), what does it mean? or is it a bug? – Economics

This method is unnecessarily complex. @neonneo's method is much more straightforward and does a better job of answering the question. – Brookhouse

@Intranuclear for me np.where returns a tuple, gotta do len(np.where(...)[0]) – Cadastre

Answer 2

This is very straightforward with boolean arrays:

p31 = numpy.asarray(o31)
za = (p31 < 200).sum() # p31<200 is a boolean array, so `sum` counts the number of True elements

Voe

I think this should be the top answer. The np.where also works obviously, but gives additional information which is not required here and makes the answer a bit less straightforward, and performance a bit slower – Sharpeyed

Answer 3

There are many ways to achieve this, like flatten-and-filter or simply enumerate, but I think using Boolean/mask array is the easiest one (and iirc a much faster one):

>>> y = np.array([[123,24123,32432], [234,24,23]])
array([[  123, 24123, 32432],
       [  234,    24,    23]])
>>> b = y > 200
>>> b
array([[False,  True,  True],
       [ True, False, False]], dtype=bool)
>>> y[b]
array([24123, 32432,   234])
>>> len(y[b])
3
>>>> y[b].sum()
56789

Update:

As nneonneo has answered, if all you want is the number of elements that passes threshold, you can simply do:

>>>> (y>200).sum()
3

which is a simpler solution.

---

Speed comparison with filter:

### use boolean/mask array ###

b = y > 200

%timeit y[b]
100000 loops, best of 3: 3.31 us per loop

%timeit y[y>200]
100000 loops, best of 3: 7.57 us per loop

### use filter ###

x = y.ravel()
%timeit filter(lambda x:x>200, x)
100000 loops, best of 3: 9.33 us per loop

%timeit np.array(filter(lambda x:x>200, x))
10000 loops, best of 3: 21.7 us per loop

%timeit filter(lambda x:x>200, y.ravel())
100000 loops, best of 3: 11.2 us per loop

%timeit np.array(filter(lambda x:x>200, y.ravel()))
10000 loops, best of 3: 22.9 us per loop

*** use numpy.where ***

nb = np.where(y>200)
%timeit y[nb]
100000 loops, best of 3: 2.42 us per loop

%timeit y[np.where(y>200)]
100000 loops, best of 3: 10.3 us per loop

Forwhy

timeit y[b] omits half the code from the computation done in filter. %timeit y[y>200] is equivalent-ish. – Selsyn

Answer 4

Here's a variant that uses fancy indexing and has the actual values as an intermediate:

p31 = numpy.asarray(o31)
values = p31[p31<200]
za = len(values)

Banderilla

This is just regular boolean indexing. – Selsyn

Answer 5

To count the number of values larger than x in any numpy array you can use:

n = len(matrix[matrix > x])

The boolean indexing returns an array that contains only the elements where the condition (matrix > x) is met. Then len() counts these values.

Multinuclear

You probably need to explain why this works. Overloaded __gt__ I guess, but that's not obvious. – Blatman

Answer 6

You can use numpy.count_nonzero, converting the whole into a one-liner:

za = numpy.count_nonzero(numpy.asarray(o31)<200)  #as written in the code

Pizza

You could avoid an explicit call to asarray with np.count_nonzero(np.less_than(o3, 200)). – Selsyn