Nearest Neighbor Search in Python without k-d tree

Question

I'm beginning to learn Python coming from a C++ background. What I am looking for is a quick and easy way to find the closest (nearest neighbor) of some multidimensional query point in an 2D (numpy) array of multidimensional points (also numpy arrays). I know that scipy has a k-d tree, but I don't think this is what I want. First of all, I will be changing the values of the multidimensional points in the 2D array. Secondly, the position (coordinates) of each point in the 2D array matters as I will also be changing their neighbors.

I could write a function that goes through the 2D array and measures the distance between the query point and the points in the array while keeping track of the smallest one (using a scipy spatial distance function to measure distance). Is there is a built in function that does this? I am trying to avoid iterating over arrays in python as much as possible. I will also have numerous query points so there would be at least two "for loops" - one to iterate through the query points and for each query, a loop to iterate through the 2D array and find the minimum distance.

Thanks for any advice.

Rori

Answer 1

If concise is your goal, you can do this one-liner:

In [14]: X = scipy.randn(10,2)

In [15]: X
Out[15]: 
array([[ 0.85831163,  1.45039761],
       [ 0.91590236, -0.64937523],
       [-1.19610431, -1.07731673],
       [-0.48454195,  1.64276509],
       [ 0.90944798, -0.42998205],
       [-1.17765553,  0.20858178],
       [-0.29433563, -0.8737285 ],
       [ 0.5115424 , -0.50863231],
       [-0.73882547, -0.52016481],
       [-0.14366935, -0.96248649]])

In [16]: q = scipy.array([0.91, -0.43])

In [17]: scipy.argmin([scipy.inner(q-x,q-x) for x in X])
Out[17]: 4

If you have several query points:

In [18]: Q = scipy.array([[0.91, -0.43], [-0.14, -0.96]])

In [19]: [scipy.argmin([scipy.inner(q-x,q-x) for x in X]) for q in Q]
Out[19]: [4, 9]

Selfreliant

Answer 2

Broadcasting is very useful for this kind of thing. I'm not sure if this is what you need, but here I use broadcasting to find the displacement between p (one point in 3 space) and X (a set of 10 points in 3-space).

import numpy as np

def closest(X, p):
    disp = X - p
    return np.argmin((disp*disp).sum(1))

X = np.random.random((10, 3))
p = np.random.random(3)

print X
#array([[ 0.68395953,  0.97882991,  0.68826511],
#       [ 0.57938059,  0.24713904,  0.32822283],
#       [ 0.06070267,  0.06561339,  0.62241713],
#       [ 0.93734468,  0.73026772,  0.33755815],
#       [ 0.29370809,  0.76298588,  0.68728743],
#       [ 0.66248449,  0.6023311 ,  0.76704199],
#       [ 0.53490144,  0.96555923,  0.43994738],
#       [ 0.23780428,  0.75525843,  0.46067472],
#       [ 0.84240565,  0.82573202,  0.56029917],
#       [ 0.66751884,  0.31561133,  0.19244683]])
print p
#array([ 0.587416 ,  0.4181857,  0.2539029])
print closest(X, p)
#9

Gambell

Answer 3

You can compute all distances scipy.spatial.distance.cdist( X, Y ) or use RTree for dynamic data: http://gispython.org/rtree/docs/class.html .

Tokay

I like the first suggestion, but I am doing one query at a time and updating values in the array (similar to SOM). I could use cdist(X,Y) where X is just one query and update the array and move on to the next query. Rtree seems like it might be OK, but I'm a bit unsure of how to use it in my situation. I wonder if there are any graph packages that would allow for a nearest neighbor search with an outside point? I could use a graph package to make a lattice where each node is an multidimensional point. Some of the other features of a graph package would come in handy in my program – Rori

Answer 4

For faster search and support for dynamic item insertion, you could use a binary tree for 2D items where greater and less than operator is defined by distance to a reference point (0,0).

def dist(x1,x2):
    return np.sqrt( (float(x1[0])-float(x2[0]))**2 +(float(x1[1])-float(x2[1]))**2 )

class Node(object):

    def __init__(self, item=None,):
        self.item = item
        self.left = None
        self.right = None

    def __repr__(self):
        return '{}'.format(self.item)

    def _add(self, value, center):
        new_node = Node(value)
        if not self.item:
            self.item = new_node        
        else:
        vdist = dist(value,center)
        idist = dist(self.item,center)
            if vdist > idist:
                self.right = self.right and self.right._add(value, center) or new_node
            elif vdist < idist:
                self.left = self.left and self.left._add(value, center) or new_node
            else:
                print("BSTs do not support repeated items.")

        return self # this is necessary!!!

    def _isLeaf(self):
        return not self.right and not self.left

class BSTC(object):

    def __init__(self, center=[0.0,0.0]):
        self.root = None
    self.count = 0
    self.center = center

    def add(self, value):
        if not self.root:
            self.root = Node(value)
        else:
            self.root._add(value,self.center)
    self.count += 1

    def __len__(self): return self.count

    def closest(self, target):
            gap = float("inf")
            closest = float("inf")
            curr = self.root
            while curr:
                if dist(curr.item,target) < gap:
                    gap = dist(curr.item, target)
                    closest = curr
                if target == curr.item:
                    break
                elif dist(target,self.center) < dist(curr.item,self.center):
                    curr = curr.left
                else:
                    curr = curr.right
            return closest.item, gap


import util

bst = util.BSTC()
print len(bst)

arr = [(23.2323,34.34535),(23.23,36.34535),(53.23,34.34535),(66.6666,11.11111)]
for i in range(len(arr)): bst.add(arr[i])

f = (11.111,22.2222)
print bst.closest(f)
print map(lambda x: util.dist(f,x), arr)

Nival

Hi @Nival . Thank you for posting the solution. Is it possible to revise the code, as indentation of some of the lines, seems to be off. I tried to fix it, but then I get the wrong results. Can you check the indentation of your code please? Thank you for this. – Asperity