Is storing Graphics objects a good idea?
Asked Answered
L

4

44

I'm currently in the process of writing a paint program in java, designed to have flexible and comprehensive functionalities. It stemmed from my final project, that I wrote overnight the day before. Because of that, it's got tons and tons of bugs, which I've been tackling one by one (e.g. I can only save files that will be empty, my rectangles don't draw right but my circles do...).

This time, I've been trying to add undo/redo functionality to my program. However, I can't "undo" something that I have done. Therefore, I got an idea to save copies of my BufferedImage each time a mouseReleased event was fired. However, with some of the images going to 1920x1080 resolution, I figured that this wouldn't be efficient: storing them would probably take gigabytes of memory.

The reason for why I can't simply paint the same thing with the background colour to undo is because I have many different brushes, which paint based on Math.random(), and because there are many different layers (in a single layer).

Then, I've considered cloning the Graphics objects that I use to paint to the BufferedImage. Like this:

ArrayList<Graphics> revisions = new ArrayList<Graphics>();

@Override
public void mouseReleased(MouseEvent event) {
    Graphics g = image.createGraphics();
    revisions.add(g);
}

I haven't done this before, so I have a couple questions:

  • Would I still be wasting pointless memory by doing this, like cloning my BufferedImages?
  • Is there necessarily a different way I can do this?
Lynnlynna answered 12/7, 2015 at 17:12 Comment(0)
B
45

No, storing a Graphics object is usually a bad idea. :-)

Here's why: Normally, Graphics instances are short-lived and is used to paint or draw onto some kind of surface (typically a (J)Component or a BufferedImage). It holds the state of these drawing operations, like colors, stroke, scale, rotation etc. However, it does not hold the result of the drawing operations or the pixels.

Because of this, it won't help you achieve undo-functionality. The pixels belongs to the component or image. So, rolling back to a "previous" Graphics object will not modify the pixels back to the previous state.

Here's some approaches I know works:

  • Use a "chain" of commands (command pattern) to modify the image. Command pattern works very nice with undo/redo (and is implemented in Swing/AWT in Action). Render all commands in sequence, starting from the original. Pro: The state in each command is usually not so large, allowing you to have many steps of undo-buffer in memory. Con: After a lot of operations, it becomes slow...

  • For every operation, store the entire BufferedImage (as you originally did). Pro: Easy to implement. Con: You'll run out of memory fast. Tip: You could serialize the images, making undo/redo taking less memory, at the cost of more processing time.

  • A combination of the above, using command pattern/chain idea, but optimizing the rendering with "snapshots" (as BufferedImages) when reasonable. Meaning you won't need to render everything from the beginning for each new operation (faster). Also flush/serialize these snapshots to disk, to avoid running out of memory (but keep them in memory if you can, for speed). You could also serialize the commands to disk, for virtually unlimited undo. Pro: Works great when done right. Con: Will take some time to get right.

PS: For all of the above, you need to use a background thread (like SwingWorker or similar) to update the displayed image, store commands/images to disk etc in the background, to keep a responsive UI.

Good luck! :-)

Bradstreet answered 14/7, 2015 at 19:52 Comment(11)
awt action and image serialization... Checking it out right now :)Lynnlynna
@Zizouz212: Start out with the AbstractAction class (and read up on command pattern if needed!). And for image serialization, you could use ImageIO and a lossless format (like BMP, PNG or TIFF). Or you could just store the byte or int backing arrays of the BufferedImage to disk, assuming your program uses a standardized color model.Bradstreet
Oh my god, I completely forgot about the bounty, and I was totally going to award it to you. I hope 75 points is okay, but for the record, I give this 150 points. :) I feel really bad now...Lynnlynna
No worries.. Not much to do about that, I guess. You could perhaps accept the answer.Bradstreet
I'm not joking when you literally save my life and give me hopes back in my app :DLynnlynna
There's another option, too. Any two versions of an image are going to be very similar. You could store the difference between the two images, rather than the entire image, similar to a loss-less video compression algorithm.Marlomarlon
@EricJ. It's an interesting idea. It's probably more computationally expensive than just storing the parameters of a command or snapshot, but likely to save memory compared to a full image, so it's probably worth investigating. Also, finding the bounding rectangle of the change will help, to store less for each change.Bradstreet
I'm not sure whether it would be more computationally expensive. It might depend on what the commands did. Replaying a simple line draw command will be quick, but a floodfill operation costs a bit more and applying a complex filter to part of an image yet more. If you want to support a very lengthy undo queue, it might cost more to replay dozens of commands vs. perhaps store a "keyframe" (full image, with optional loss-less compression) every N undo steps and applying differences from the last keyframe. Then again you could use the keyframe idea with command replay too. Interesting problem.Marlomarlon
@EricJ. I think we're looking at it from different sides. Storing a flood fill, is just a point and a color/gradient/pattern, and thus the command is much cheaper to store than creating a keyframe diff. Replaying the full flood fill command on the other hand might be more expensive, compared to just apply the diff. :-)Bradstreet
Yes, that was my point that replaying a lot of commands might be more expensive than my thought. Then again, it might not be.Marlomarlon
@Lynnlynna Thank you! :-)Bradstreet
T
9

Idea #1, storing the Graphics objects simply wouldn't work. The Graphics should not be considered as "holding" some display memory, but rather as a handle to access an area of display memory. In the case of BufferedImage, each Graphics object will be always the handle to the same given image memory buffer, so they all will represent the same image. Even more importantly, you can't actually do anything with the stored Graphics: As they do not store anything, there is no way whatsoever they could "re-store" anything.

Idea #2, cloning the BufferedImages is a much better idea, but you'll indeed be wasting memory, and quickly run out of it. It helps only to store those parts of the image affected by the draw, for example using rectangular areas, but it still costs a lot of memory. Buffering those undo images to disk could help, but it will make your UI slow and unresponsive, and that's bad; furthermore, it makes you application more complex and error-prone.

My alternative would be to store store the image modifications in a list, rendered from first to last on top of the image. An undo operation then simply consists of removing the modification from the list.

This requires you to "reify" the image modifications, i.e. create a class that implements a single modification, by providing a void draw(Graphics gfx) method which performs the actual drawing.

As you said, random modifications pose an additional problem. However, the key problem is your use of Math.random() to create random numbers. Instead, perform each random modification with a Random created from a fixed seed value, so that the (pseudo-)random number sequences are the same on each invocation of draw(), i.e., each draw has exactly the same effects. (That's why they are called "pseudo-random" -- the generated numbers look random, but they are just as deterministic as any other function.)

In contrast to the image storing technique, which has memory problems, the problem with this technique is that many modifications may make the GUI slow, especially if the modifications are computationally intensive. To prevent this, the simplest way would be to fix an appropriate maximum size of the list of undoable modifications. If this limit would be exceeded by adding a new modification, remove the oldest modification the list and apply it to the backing BufferedImage itself.

The following simple demo application shows that (and how) this all works together. It also includes a nice "redo" feature for redoing undone actions.

package stackoverflow;

import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.image.BufferedImage;
import java.util.LinkedList;
import java.util.Random;
import javax.swing.*;

public final class UndoableDrawDemo
        implements Runnable
{
    public static void main(String[] args) {
        EventQueue.invokeLater(new UndoableDrawDemo()); // execute on EDT
    }

    // holds the list of drawn modifications, rendered back to front
    private final LinkedList<ImageModification> undoable = new LinkedList<>();
    // holds the list of undone modifications for redo, last undone at end
    private final LinkedList<ImageModification> undone = new LinkedList<>();

    // maximum # of undoable modifications
    private static final int MAX_UNDO_COUNT = 4;

    private BufferedImage image;

    public UndoableDrawDemo() {
        image = new BufferedImage(600, 600, BufferedImage.TYPE_INT_RGB);
    }

    public void run() {
        // create display area
        final JPanel drawPanel = new JPanel() {
            @Override
            public void paintComponent(Graphics gfx) {
                super.paintComponent(gfx);

                // display backing image
                gfx.drawImage(image, 0, 0, null);

                // and render all undoable modification
                for (ImageModification action: undoable) {
                    action.draw(gfx, image.getWidth(), image.getHeight());
                }
            }

            @Override
            public Dimension getPreferredSize() {
                return new Dimension(image.getWidth(), image.getHeight());
            }
        };

        // create buttons for drawing new stuff, undoing and redoing it
        JButton drawButton = new JButton("Draw");
        JButton undoButton = new JButton("Undo");
        JButton redoButton = new JButton("Redo");

        drawButton.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
                // maximum number of undo's reached?
                if (undoable.size() == MAX_UNDO_COUNT) {
                    // remove oldest undoable action and apply it to backing image
                    ImageModification first = undoable.removeFirst();

                    Graphics imageGfx = image.getGraphics();
                    first.draw(imageGfx, image.getWidth(), image.getHeight());
                    imageGfx.dispose();
                }

                // add new modification
                undoable.addLast(new ExampleRandomModification());

                // we shouldn't "redo" the undone actions
                undone.clear();

                drawPanel.repaint();
            }
        });

        undoButton.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
                if (!undoable.isEmpty()) {
                    // remove last drawn modification, and append it to undone list
                    ImageModification lastDrawn = undoable.removeLast();
                    undone.addLast(lastDrawn);

                    drawPanel.repaint();
                }
            }
        });

        redoButton.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
                if (!undone.isEmpty()) {
                    // remove last undone modification, and append it to drawn list again
                    ImageModification lastUndone = undone.removeLast();
                    undoable.addLast(lastUndone);

                    drawPanel.repaint();
                }
            }
        });

        JPanel buttonPanel = new JPanel(new FlowLayout());
        buttonPanel.add(drawButton);
        buttonPanel.add(undoButton);
        buttonPanel.add(redoButton);

        // create frame, add all content, and open it
        JFrame frame = new JFrame("Undoable Draw Demo");
        frame.getContentPane().add(drawPanel);
        frame.getContentPane().add(buttonPanel, BorderLayout.NORTH);
        frame.pack();
        frame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
        frame.setLocationRelativeTo(null);
        frame.setVisible(true);
    }

    //--- draw actions ---

    // provides the seeds for the random modifications -- not for drawing itself
    private static final Random SEEDS = new Random();

    // interface for draw modifications
    private interface ImageModification
    {
        void draw(Graphics gfx, int width, int height);
    }

    // example random modification, draws bunch of random lines in random color
    private static class ExampleRandomModification implements ImageModification
    {
        private final long seed;

        public ExampleRandomModification() {
            // create some random seed for this modification
            this.seed = SEEDS.nextLong();
        }

        @Override
        public void draw(Graphics gfx, int width, int height) {
            // create a new pseudo-random number generator with our seed...
            Random random = new Random(seed);

            // so that the random numbers generated are the same each time.
            gfx.setColor(new Color(
                    random.nextInt(256), random.nextInt(256), random.nextInt(256)));

            for (int i = 0; i < 16; i++) {
                gfx.drawLine(
                        random.nextInt(width), random.nextInt(height),
                        random.nextInt(width), random.nextInt(height));
            }
        }
    }
}
Trimly answered 16/7, 2015 at 23:59 Comment(2)
Instead of maximum-undo-depth, you could store complete images every X steps, in addition to the actions taken. This would allow fast painting (because you would only replay the actions since the last snapshot) while allowing for unlimited undo.Ahouh
Great work actually implementing a demo app for this! You could probably have used the Swing Edit and UndoManager classes for some of it though. :-)Bradstreet
E
7

Most games(or programs) saves only the necessary parts and thats what you should do.

  • a rectangle can be represented by width, height, background color, stroke, outline etc. So you can just save these parameters instead of the actual rectangle. "rectangle color:red width: 100 height 100"

  • for the random aspects of your program( random color on brushes) you can either save the seed or save the result. "random seed: 1023920"

  • if the program allows user to import images then you should copy and save the images.

  • fillters and effects(zoom/transformation/glow) can all be represented by parameters just like shapes. eg. "zoom scale: 2" "rotate angle: 30"

  • so you save all these parameters in a list and when then you need to undo you can mark the parameters as deleted( but dont actually delete them since you want to be able to redo as well). Then you can erase the whole canvas and recreate the image based on the parameters minus those that were marked as deleted.

*for stuff like lines you can just store their locations in a list.

El answered 22/1, 2016 at 23:23 Comment(1)
I like this answer, as it fleshes out and adds some detail on how to go on storing the parameters on each drawing command. It's basically the same idea as I describe as a "chain of commands".Bradstreet
Z
4

You are going to want to try to compress your images (using PNG is a good start, it has some nice filters along with zlib compression that really help). I think the best way to do this is to

  • make a copy of the image before you modify it
  • modify it
  • compare the copy with the new modified image
  • for every pixel you didn't change, make that pixel a black, transparent pixel.

That should compress really, really well in PNG. Try black and white and see if there is a difference (I don't think there will be, but make sure you set the rgb values to the same thing, not just the alpha value, so it will compress better).

You might get even better performance out of cropping the image to the part that was changed, but I'm not sure how much you gain from that, considering the compression (and the fact that you will have to now save and remember the offset).

Then, since you have an alpha channel, if they undo, you can just put the undo image back on top of the current image and you're set.

Zigrang answered 16/7, 2015 at 3:43 Comment(1)
You can XOR the image with the previous image to get rid of all the pixels which have not changed before compressing.Mother

© 2022 - 2024 — McMap. All rights reserved.