Fastest moving object recognition and tracking in Android

Question

I am working on an augment reality game, which need to recognize and track a fast moving object. I have tried the following Image processing libraries,

1. Opencv

2. BoofCv

3. FastCv

I have tried TLD algorithm to track the object, tracking was successful but the performance was really needed to be improved. If the object is moving faster the result takes time, because of the processing time taken by the algorithms. I have also tried circulant,mean-shift like algorithms with boofcv.

Check these demos :

OpenTLD using FastCv

Boofcv Demonstration

Object tracking in these two demos seems to be good but the calculation takes time.

Can i go with the following scenario to do this faster,

1. Extract the r,g,b matrix of object to be tracked

2. Take camera frames and convert it into a r,g,b matrix and search the tracked object matrix with in the camera frame.

Is there is any better way to do this ??

Papule

I have found template matching algorithms in OpenCv. #17001583. – Papule

What FPS are you getting and what do you need? It's going to be hard to run faster than mean-shift and mean-shift is probably not robust enough for general purpose tracking. – Isotope

I am getting around 9 to 11 Fps. – Papule

@Papule Did you find any solution ? – Guitarfish

Answer 1

I suggest using gray scale instead of RGB, like it is usually done in image processing that way your computation is reduced to 1 matrix instead of 3.

If you need to check for color, just use rgb when needed but not through out the whole computation.

Tracking fast moving objects is always difficult. Try using a camera that can take more frames per second although you need to process more images and I suppose you are on a mobile device

Also what you can do is to reduce the image size being processed to a smaller window based on the previous object position you can estimate the next position and limit it to a certain vecinity and only process that bits of the image. In brief perform optical flow only in certain section of the image (use gray scale).

Phenocryst

From my experience color information is really vital if you're doing histogram tracking. Unless you have a special case, e.g. bright object with a dark background. With gray scale too many objects look similar and the track tends to randomly attach itself to other objects. – Isotope

Well, most of the image processing algorithms and techniques works in gray scale and most of the times it is enough to solve most of the problems. Many medical image processing is done in grayscale and if you are careful enough and use certain techniques you can identify very complex objects or forms. It really depends what you are doing, that is why I said "just use rgb when needed". Needing color information does not means you have to use the RGB through the entire process. – Phenocryst

Answer 2

I think SIFT and SURF algorithms are best suited for this purpose. SIFT or SURF can by used like any other feature detector and extractor:

FeatureDetector detector = FeatureDetector.create(FeatureDetector.ORB); 
// May be SIFT,   SURF, etc 
detector.detect(mat, keypoints); 
DescriptorExtractor extractor =     DescriptorExtractor.create(DescriptorExtractor.ORB); 
// May be SIFT, SURF, etc 
extractor.compute(mat, keypoints, features);

Use openCv to learn more. And ofcourse your way may be find solutions for this. Keep trying.

Entebbe

They can only be used to track objects if the object is fairly large visually. For example, all of those detector/descriptors will fail to track a person juggling a red ball. In that scenario, it is likely that at most one interest point will be found (lack of texture) and it won't be consistently triggered. Basically the object needs to be large enough to create numerous interest points so that if a few fail it can still work. – Isotope