stitching microscope images of a microchip

Question

So, I'm trying to stitch images taken by a microscope of a microchip, but it's very hard to have all the features aligned. I already have a 50% overlap between two adjacent images, but even with that, it's not always a good fit.

I'm using SURF with OpenCV to extract the keypoints and find the homographic matrix. But still, it's far from being an acceptable result.

My objective is to be able to stitch perfectly 2x2 images, so this way, I can repeat that process recursively until I have the final image.

Do you have any suggestion ? A nice algorithm to approach this problem. Or maybe a way to transform the images to be able to extract better keypoints from them. Play with the threshold (a smaller one to get more keypoints, or a larger one?).

Right now, my approach is to first stitch two 2x1 images and then, stitch these two together. It's close from what we want, but still not acceptable. Also, the problem might be that the image used to be the "source" (while the second image is transform with the matrix to overlap that one) might not be a bit misaligned or there's a small angle on that image that affects the whole result.

Any help or suggestion is appreciated. Specially any solution that would allow to use OpenCV and SURF (even if I'm not totally against other libraries... it's just that most of the project has been developed with that).

Thanks!

Willin

Answer 1

I found using TurboReg during image registration development to be a helpful comparison tool. It is a free ImageJ plugin, and has many different fitting types.

Have you taken a look at the new OpenCV stitching samples: stitching.cpp and stitching_detailed.cpp?

EDIT : I forgot this was cutting edge OpenCV because I'm using the trunk at home :) To get access to these new samples, you'll need to check out the OpenCV trunk from SVN like this:

svn co https://code.ros.org/svn/opencv/trunk/opencv opencv-trunk

Unfortunately, you'll need to recompile it, but you should be able to use the new stitching code :) If you haven't built OpenCV from source before, here is a good little tutorial to get you started. I will mention that OpenCV has a lot more options that can be enabled/disabled than are mentioned in the tutorial, so you might want to use the cmake-gui to get a look at all of the options. You can apt-get it with this command:

> sudo apt-get install cmake-qt-gui

Also, if you're more concerned with quality, and you don't mind slower performance; you might consider using the Lucas-Kanade method for image registration. Here is a lecture, and here is a paper on the topic that might be helpful to you.

Tso

stitching_detail.cpp looks quite interesting... problem is I can't compile it. Do you happen to know how ? I don't know where I can find the files in "opencv2/stitching/"... I'm on Ubuntu and cannot tell if I miss any packages, cannot find any clear information about that anywhere! – Willin

It's a bit complicated, but thanks for the answer... right now, I'm trying to compile the source code to use the stitching library... just hoping it will work! – Willin

Should work just fine. Just make sure you apt-get all of the necessary dependencies as mentioned in the tutorial. – Tso

As for if the stitching sample meets your needs, time will tell I suppose :) Stitching is a difficult problem to solve well as I'm sure you're finding out :D – Tso

any tips to install the stitching part in the OpenCV/Trunk ? It's like the only library that doesn't compile properly. All the other ones, like calib3d end-up creating a library called opencv_calib3d... but there's nothing for stitching and when I try to compile, there's a lot of linking errors! (really tired of working on that thing... need it to work :( ) – Willin

Did the opencv_stitching binary build OK? This built correctly on my MinGW build. – Tso

nah, it's not there... also, in the include directory opencv2/stitching... the subdirectory detail is not there, all the files are in the stitching directory which cause some conflit too... why couldn't it be simple :( I'm on Linux, but MinGW is suppose to simulate a linux environment – Willin

Actually, I noticed I have OpenCV 2.3.1 on my Windows box. I'll have to take a look at my Ubuntu box as that is where I have the trunk built. So, you may not have opencv_stitching in the trunk anymore. – Tso

if it's not there, is there a way to use it or compile "stitching_detailed.cpp" ? Because that piece of code seams to do a lot of things that might help me to solve my problem, but first I need to see how it works and how well it does... and from that, start integrating some part of it in my project. – Willin

Thanks a lot for sending me toward stitching_detail.cpp... now I managed to compile it and run it. It's actually quite close to what I need. Wished I found that program sooner (but unfortunately, I discarded researching everything with opencv2 because the computer at work don't run it... so I tried at home and it's running here) – Willin

Answer 2

The Fiji's stitching plugin handles this situation of alignment error propagation with 2D mosaicing. We use it in daily use for microscopic stitching, and I must say it is perfect.

Internationalist

There's some people at my job using it to stitch, but they're not always satisfied with the result! – Willin

You may have to deal with thresholds and other parameters. We're using it in as in our commercial microscope software, and it's really good. Do you have knowledge of displacement between tiles? If yes it is the perfect case. – Internationalist

Actually, I coded a program that stitched perfectly (by allowing to play with parameters)! – Willin