How to implement pan/zoom on gigapixel bitmaps?
Asked Answered
V

3

7

In my project, I'm using (uncompressed 16-bit grayscale) gigapixel images which come from a high resolution scanner for measurement purposes. Since these bitmaps can not be loaded in memory (mainly due to memory fragmentation) I'm using tiles (and tiled TIFF on disc). (see StackOverflow topic on this)

I need to implement panning/zooming in a way like Google Maps or DeepZoom. I have to apply image processing on the fly before presenting it on screen, so I can not use a precooked library which directly accesses an image file. For zooming I intend to keep a multi-resolution image in my file (pyramid storage). The most useful steps seem to be +200%, 50% and show all.

My code base is currently C# and .NET 3.5. Currently I assume Forms type, unless WPF gives me great advantage in this area. I have got a method which can return any (processed) part of the underlying image.

Specific issues:

  • hints or references on how to implement this pan/zoom with on-demand generation of image parts
  • any code which could be used as a basis (preferably commercial or LGPL/BSD like licenses)
  • can DeepZoom be used for this (i.e. is there a way that I can provide a function to provide a tile at the right resulution for the current zoom level?) ( I need to have pixel accurate addressing still)
Viewy answered 10/2, 2010 at 7:31 Comment(1)
DeepZoom seems like the approach. You will have to create low-resolution versions of the images with something like imagemagick.org/script/index.php. From my poking around, it appears this is a patent minefield, so....Aiguille
V
3

I decided to try something myself. I came up with a straightforward GDI+ code, which uses the tiles I've already got. I just filter out the parts which are relevant for current clipping region. It works like magic! Please find my code below. (Form settings double buffering for the best results)

 protected override void OnPaint(PaintEventArgs e)
    {
        base.OnPaint(e);
        Graphics dc = e.Graphics;
        dc.ScaleTransform(1.0F, 1.0F);
        Size scrollOffset = new Size(AutoScrollPosition);

        int start_x = Math.Min(matrix_x_size, 
                             (e.ClipRectangle.Left - scrollOffset.Width) / 256);
        int start_y = Math.Min(matrix_y_size, 
                             (e.ClipRectangle.Top - scrollOffset.Height) / 256);
        int end_x = Math.Min(matrix_x_size, 
                        (e.ClipRectangle.Right - scrollOffset.Width + 255) / 256);
        int end_y = Math.Min(matrix_y_size, 
                      (e.ClipRectangle.Bottom - scrollOffset.Height + 255) / 256);

        // start * contain the first and last tile x/y which are on screen 
        // and which need to be redrawn.
        // now iterate trough all tiles which need an update 
        for (int y = start_y; y < end_y; y++)
            for (int x = start_x; x < end_x; x++)
            {  // draw bitmap with gdi+ at calculated position.
                dc.DrawImage(BmpMatrix[y, x], 
                           new Point(x * 256 + scrollOffset.Width, 
                                     y * 256 + scrollOffset.Height));
            }
    }

To test it, I've created a matrix of 80x80 of 256 tiles (420 MPixel). Of course I'll have to add some deferred loading in real life. I can leave tiles out (empty) if they are not yet loaded. In fact, I've asked my client to stick 8 GByte in his machine so I don't have to bother about performance too much. Once loaded tiles can stay in memory.

public partial class Form1 : Form
{
    bool dragging = false;
    float Zoom = 1.0F;
    Point lastMouse;
    PointF viewPortCenter;

    private readonly Brush solidYellowBrush = new SolidBrush(Color.Yellow);
    private readonly Brush solidBlueBrush = new SolidBrush(Color.LightBlue);
    const int matrix_x_size = 80;
    const int matrix_y_size = 80;
    private Bitmap[,] BmpMatrix = new Bitmap[matrix_x_size, matrix_y_size];
    public Form1()
    {
        InitializeComponent();

        Font font = new Font("Times New Roman", 10, FontStyle.Regular);
        StringFormat strFormat = new StringFormat();
        strFormat.Alignment = StringAlignment.Center;
        strFormat.LineAlignment = StringAlignment.Center;
        for (int y = 0; y < matrix_y_size; y++)
            for (int x = 0; x < matrix_x_size; x++)
            {
                BmpMatrix[y, x] = new Bitmap(256, 256, PixelFormat.Format24bppRgb);
                //                    BmpMatrix[y, x].Palette.Entries[0] = (x+y)%1==0?Color.Blue:Color.White;

                using (Graphics g = Graphics.FromImage(BmpMatrix[y, x]))
                {
                    g.FillRectangle(((x + y) % 2 == 0) ? solidBlueBrush : solidYellowBrush, new Rectangle(new Point(0, 0), new Size(256, 256)));
                    g.DrawString("hello world\n[" + x.ToString() + "," + y.ToString() + "]", new Font("Tahoma", 8), Brushes.Black,
                        new RectangleF(0, 0, 256, 256), strFormat);
                    g.DrawImage(BmpMatrix[y, x], Point.Empty);
                }
            }

        BackColor = Color.White;

        Size = new Size(300, 300);
        Text = "Scroll Shapes Correct";

        AutoScrollMinSize = new Size(256 * matrix_x_size, 256 * matrix_y_size);
    }   

Turned out this was the easy part. Getting async multithreaded i/o done in the background was a lot harder to acchieve. Still, I've got it working in the way described here. The issues to resolve were more .NET/Form multithreading related than to this topic.

In pseudo code it works like this:

after onPaint (and on Tick)
   check if tiles on display need to be retrieved from disc
       if so: post them to an async io queue
       if not: check if tiles close to display area are already loaded
           if not: post them to an async io/queue
   check if bitmaps have arrived from io thread
      if so: updat them on screen, and force repaint if visible

Result: I now have my own Custom control which uses roughly 50 MByte for very fast access to arbitrary size (tiled) TIFF files.

Viewy answered 12/2, 2010 at 14:26 Comment(1)
Make sure to add a compiler flag saying it is 64bit only, else it becomes a outofmemory paradise.Photoconduction
B
3

This CodeProject article: Generate...DeepZoom Image Collection might be a useful read since it talks about generating a DeepZoom image source.

This MSDN article has a section Dynamic Deep Zoom: Supplying Image Pixels at Run Time and links to this Mandelbrot Explorer which 'kinda' sounds similar to what you're trying to do (ie. he is generating specific parts of the mandelbrot set on-demand; you want to retrieve specific parts of your gigapixel image on-demand).

I think the answer to "can DeepZoom be used for this?" is probably "Yes", however as it is only available in Silverlight you will have to do some tricks with an embedded web browser control if you need a WinForms/WPF client app.

Sorry I can't provide more specific answers - hope those links help.

p.s. I'm not sure if Silverlight supports TIFF images - that might be an issue unless you convert to another format.

Burl answered 10/2, 2010 at 8:13 Comment(3)
Silverlight doesn't currently support TIFF images, but it does have a WriteableBitmap ClassVeinule
Thanks. +1 for the answer and the links. I'm afraid for the complexity of adding a Silverlight-via-a-webserver-in-wpf just for the pan/zoom. (I currently have no server or browser part in my application; it is desktop). As for the TIFF: this is no problem; I can stream PNG from the TIFF internally. Just discard 50% of the bits ;-)Viewy
From what I've learned so far about DeepZoom, it seems that things like accurate addressing (synchronization of zoom/position between multiple windows in my application) will be very hard. The functionality offered by Deepzoom does not justify the architectural complexity it'll bring (adding Silverlight and a server just for the pan/zoom). I'm making a custom (one of) application where the development hours are more important than the bells & whistles. I'll keep an eye on it for future applications (when it comes to .NET x.x it'll be interesting!Viewy
V
3

I decided to try something myself. I came up with a straightforward GDI+ code, which uses the tiles I've already got. I just filter out the parts which are relevant for current clipping region. It works like magic! Please find my code below. (Form settings double buffering for the best results)

 protected override void OnPaint(PaintEventArgs e)
    {
        base.OnPaint(e);
        Graphics dc = e.Graphics;
        dc.ScaleTransform(1.0F, 1.0F);
        Size scrollOffset = new Size(AutoScrollPosition);

        int start_x = Math.Min(matrix_x_size, 
                             (e.ClipRectangle.Left - scrollOffset.Width) / 256);
        int start_y = Math.Min(matrix_y_size, 
                             (e.ClipRectangle.Top - scrollOffset.Height) / 256);
        int end_x = Math.Min(matrix_x_size, 
                        (e.ClipRectangle.Right - scrollOffset.Width + 255) / 256);
        int end_y = Math.Min(matrix_y_size, 
                      (e.ClipRectangle.Bottom - scrollOffset.Height + 255) / 256);

        // start * contain the first and last tile x/y which are on screen 
        // and which need to be redrawn.
        // now iterate trough all tiles which need an update 
        for (int y = start_y; y < end_y; y++)
            for (int x = start_x; x < end_x; x++)
            {  // draw bitmap with gdi+ at calculated position.
                dc.DrawImage(BmpMatrix[y, x], 
                           new Point(x * 256 + scrollOffset.Width, 
                                     y * 256 + scrollOffset.Height));
            }
    }

To test it, I've created a matrix of 80x80 of 256 tiles (420 MPixel). Of course I'll have to add some deferred loading in real life. I can leave tiles out (empty) if they are not yet loaded. In fact, I've asked my client to stick 8 GByte in his machine so I don't have to bother about performance too much. Once loaded tiles can stay in memory.

public partial class Form1 : Form
{
    bool dragging = false;
    float Zoom = 1.0F;
    Point lastMouse;
    PointF viewPortCenter;

    private readonly Brush solidYellowBrush = new SolidBrush(Color.Yellow);
    private readonly Brush solidBlueBrush = new SolidBrush(Color.LightBlue);
    const int matrix_x_size = 80;
    const int matrix_y_size = 80;
    private Bitmap[,] BmpMatrix = new Bitmap[matrix_x_size, matrix_y_size];
    public Form1()
    {
        InitializeComponent();

        Font font = new Font("Times New Roman", 10, FontStyle.Regular);
        StringFormat strFormat = new StringFormat();
        strFormat.Alignment = StringAlignment.Center;
        strFormat.LineAlignment = StringAlignment.Center;
        for (int y = 0; y < matrix_y_size; y++)
            for (int x = 0; x < matrix_x_size; x++)
            {
                BmpMatrix[y, x] = new Bitmap(256, 256, PixelFormat.Format24bppRgb);
                //                    BmpMatrix[y, x].Palette.Entries[0] = (x+y)%1==0?Color.Blue:Color.White;

                using (Graphics g = Graphics.FromImage(BmpMatrix[y, x]))
                {
                    g.FillRectangle(((x + y) % 2 == 0) ? solidBlueBrush : solidYellowBrush, new Rectangle(new Point(0, 0), new Size(256, 256)));
                    g.DrawString("hello world\n[" + x.ToString() + "," + y.ToString() + "]", new Font("Tahoma", 8), Brushes.Black,
                        new RectangleF(0, 0, 256, 256), strFormat);
                    g.DrawImage(BmpMatrix[y, x], Point.Empty);
                }
            }

        BackColor = Color.White;

        Size = new Size(300, 300);
        Text = "Scroll Shapes Correct";

        AutoScrollMinSize = new Size(256 * matrix_x_size, 256 * matrix_y_size);
    }   

Turned out this was the easy part. Getting async multithreaded i/o done in the background was a lot harder to acchieve. Still, I've got it working in the way described here. The issues to resolve were more .NET/Form multithreading related than to this topic.

In pseudo code it works like this:

after onPaint (and on Tick)
   check if tiles on display need to be retrieved from disc
       if so: post them to an async io queue
       if not: check if tiles close to display area are already loaded
           if not: post them to an async io/queue
   check if bitmaps have arrived from io thread
      if so: updat them on screen, and force repaint if visible

Result: I now have my own Custom control which uses roughly 50 MByte for very fast access to arbitrary size (tiled) TIFF files.

Viewy answered 12/2, 2010 at 14:26 Comment(1)
Make sure to add a compiler flag saying it is 64bit only, else it becomes a outofmemory paradise.Photoconduction
S
3

I guess you can address this issue following the steps below:

  1. Image generation:

    • segment your image in multiple subimages (tiles) of a small resolution, for instace, 500x500. These images are depth 0
    • combine a series of tiles with depth 0 (4x4 or 6x6), resize the combination generating a new tile with 500x500 pixels in depth 1.
    • continue with this approach until get the entire image using only a few tiles.
  2. Image visualization

    • Start from the highest depth
    • When user drags the image, load the tiles dynamically
    • When the user zoom a region of the image, decrease the depth, loading the tiles for that region in a higher resolution.

The final result is similar to Google Maps.

Substrate answered 19/11, 2012 at 17:7 Comment(2)
I'm having a hard time seeing how the link you have provided is helpful; it does not offer any code/pseudocode to describe how to do what the OP was asking. I am removing the link so that no one mistakes this answer for spam.Affricative
Actually, my application needed mainly zoom levels of 25-100%, and for that using a single layer with fixed 256x256 tiles worked very well. It's just that GDI+ can't really handle large images and needs some help (dynamic display of tiles) and loading of TIFF files (Tifflib.net). Your method is indeed required for terrabit images or if any zoom factor is needed. Thanks for your contribution.Viewy

© 2022 - 2024 — McMap. All rights reserved.