Drawing an image using sub-pixel level accuracy using Graphics2D

Question

I am currently attempting to draw images on the screen at a regular rate like in a video game.

Unfortunately, because of the rate at which the image is moving, some frames are identical because the image has not yet moved a full pixel.

Is there a way to provide float values to Graphics2D for on-screen position to draw the image, rather than int values?

Initially here is what I had done:

BufferedImage srcImage = sprite.getImage ( );
Position imagePosition = ... ; //Defined elsewhere
g.drawImage ( srcImage, (int) imagePosition.getX(), (int) imagePosition.getY() );

This of course thresholds, so the picture doesn't move between pixels, but skips from one to the next.

The next method was to set the paint color to a texture instead and draw at a specified position. Unfortunately, this produced incorrect results that showed tiling rather than correct antialiasing.

g.setRenderingHint ( RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON );

BufferedImage srcImage = sprite.getImage ( );

g.setPaint ( new TexturePaint ( srcImage, new Rectangle2D.Float ( 0, 0, srcImage.getWidth ( ), srcImage.getHeight ( ) ) ) );

AffineTransform xform = new AffineTransform ( );

xform.setToIdentity ( );
xform.translate ( onScreenPos.getX ( ), onScreenPos.getY ( ) );
g.transform ( xform );

g.fillRect(0, 0, srcImage.getWidth(), srcImage.getHeight());

What should I do to achieve the desired effect of subpixel rendering of an Image in Java?

Woodenhead

Answer 1

You can use a BufferedImage and AffineTransform, draw to the buffered image, then draw the buffered image to the component in the paint event.

    /* overrides the paint method */
    @Override
    public void paint(Graphics g) {
        /* clear scene buffer */
        g2d.clearRect(0, 0, (int)width, (int)height);

        /* draw ball image to the memory image with transformed x/y double values */
        AffineTransform t = new AffineTransform();
        t.translate(ball.x, ball.y); // x/y set here, ball.x/y = double, ie: 10.33
        t.scale(1, 1); // scale = 1 
        g2d.drawImage(image, t, null);

        // draw the scene (double percision image) to the ui component
        g.drawImage(scene, 0, 0, this);
    }

Check my full example here: http://pastebin.com/hSAkYWqM

Hower

Hey Lawren! Thank you for responding. Thank you so much for showing me how to do this! – Woodenhead

why t.scale(1, 1); ?? AFAIK that instruction does nothing? (scale to its actual size?) – Liborio

Answer 2

You can composite the image yourself using sub-pixel accuracy, but it's more work on your part. Simple bilinear interpolation should work well enough for a game. Below is psuedo-C++ code for doing it.

Normally, to draw a sprite at location (a,b), you'd do something like this:

for (x = a; x < a + sprite.width; x++)
{
    for (y = b; y < b + sprite.height; y++)
    {
        *dstPixel = alphaBlend (*dstPixel, *spritePixel);
        dstPixel++;
        spritePixel++;
    }
    dstPixel += destLineDiff; // Move to start of next destination line
    spritePixel += spriteLineDiff; // Move to start of next sprite line
}

To do sub-pixel rendering, you do the same loop, but account for the sub-pixel offset like so:

float xOffset = a - floor (a);
float yOffset = b - floor (b);
for (x = floor(a), spriteX = 0; x < floor(a) + sprite.width + 1; x++, spriteX++)
{
    for (y = floor(b), spriteY = 0; y < floor (b) + sprite.height + 1; y++, spriteY++)
    {
        spriteInterp = bilinearInterp (sprite, spriteX + xOffset, spriteY + yOffset);
        *dstPixel = alphaBlend (*dstPixel, spriteInterp);
        dstPixel++;
        spritePixel++;
    }
    dstPixel += destLineDiff; // Move to start of next destination line
    spritePixel += spriteLineDiff; // Move to start of next sprite line
}

The bilinearInterp() function would look something like this:

Pixel bilinearInterp (Sprite* sprite, float x, float y)
{
    // Interpolate the upper row of pixels
    Pixel* topPtr = sprite->dataPtr + ((floor (y) + 1) * sprite->rowBytes) + floor(x) * sizeof (Pixel);
    Pixel* bottomPtr = sprite->dataPtr + (floor (y) * sprite->rowBytes) + floor (x) * sizeof (Pixel);

    float xOffset = x - floor (x);
    float yOffset = y - floor (y);

    Pixel top = *topPtr + ((*(topPtr + 1) - *topPtr) * xOffset;
    Pixel bottom = *bottomPtr + ((*(bottomPtr + 1) - *bottomPtr) * xOffset;
    return bottom + (top - bottom) * yOffset;
}

This should use no additional memory, but will take additional time to render.

Cliffhanger

How would I go about doing such operations in java? Is there optimizations that can be made? I have many objects, and computing something like this seems like it may take some time... – Woodenhead

I'm not very fluent in java, but presumably your sprites have some structure, right? In C-based languages, usually there's a pointer to the first pixel and a number of bytes per row. I imagine in Java that you probably have an array of pixels with no gaps between rows. So a sprite may literally be just a very large array of red, green, blue, alpha (or possibly alpha, red, green, blue). So instead of calculating the address of a pixel structure, you'd simply get the offsets you need. In bilinearInterp, the first 2 lines are just calculating the index of the 4 pixels around the desired point. – Cliffhanger

That's correct, but to compute for each frame in my code may not be the best way. I was hoping there may be some sort of built-in function as part of the Java API that performs this. – Woodenhead

Answer 3

I successfully solved my problem after doing something like lawrencealan proposed.

Originally, I had the following code, where g is transformed to a 16:9 coordinate system before the method is called:

private void drawStar(Graphics2D g, Star s) {

    double radius = s.getRadius();
    double x = s.getX() - radius;
    double y = s.getY() - radius;
    double width = radius*2;
    double height = radius*2;

    try {

        BufferedImage image = ImageIO.read(this.getClass().getResource("/images/star.png"));
        g.drawImage(image, (int)x, (int)y, (int)width, (int)height, this);

    } catch (IOException ex) {
        Logger.getLogger(View.class.getName()).log(Level.SEVERE, null, ex);
    }

}

However, as noted by the questioner Kaushik Shankar, turning the double positions into integers makes the image "jump" around, and turning the double dimensions into integers makes it scale "jumpy" (why the hell does g.drawImage not accept doubles?!). What I found working for me was the following:

private void drawStar(Graphics2D g, Star s) {

    AffineTransform originalTransform = g.getTransform();

    double radius = s.getRadius();
    double x = s.getX() - radius;
    double y = s.getY() - radius;
    double width = radius*2;
    double height = radius*2;

    try {

        BufferedImage image = ImageIO.read(this.getClass().getResource("/images/star.png"));

        g.translate(x, y);
        g.scale(width/image.getWidth(), height/image.getHeight());

        g.drawImage(image, 0, 0, this);

    } catch (IOException ex) {
        Logger.getLogger(View.class.getName()).log(Level.SEVERE, null, ex);
    }

    g.setTransform(originalTransform);

}

Seems like a stupid way of doing it though.

Robenarobenia

Thanks for posting; I ended up doing something similar as well; its definitely unfortunate that drawImage has no support for that. – Woodenhead

Amazing thanks a lot!! Also, I used g.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR); to make it look even better. – Nogging

Answer 4

Change the resolution of your image accordingly, there's no such thing as a bitmap with sub-pixel coordinates, so basically what you can do is create an in memory image larger than what you want rendered to the screen, but allows you "sub-pixel" accuracy.

When you draw to the larger image in memory, you copy and resample that into the smaller render visible to the end user.

For example: a 100x100 image and it's 50x50 resized / resampled counterpart:

See: http://en.wikipedia.org/wiki/Resampling_%28bitmap%29

Hower

This also implements a double buffer, so you can do multiple revisions to the in memory image while not updating the UI which can be processor intense – Hower

Image a ball moving across the screen, but its moving very slowly. Its moving 1 pixel every second. If I draw 2 times to the screen every second, then the second time, the drawing will be exactly the same as the first time. The third time will show the ball that would have moved one pixel in one step. What I want instead is for the ball to be moved partially so that it can be drawn as if it were between the two pixels. Is this functionality available in java? – Woodenhead

You have to emulate it using the above process. So say your render area is 800x600, you need an in-memory image that is 2x that -- 1600x1200 -- and instead of moving at the floating point level, you move across whole pixels... so instead of moving a ball 0.5 px you move it 1px in the larger memory based image, and when you resample it to the viewable area (800x600) it will be emulated sub-pixel accuracy.. – Hower

Oh! I see what you mean. Is that memory efficient? – Woodenhead

There must be some other way to get the anti-aliasing effect without having to double each dimension! – Woodenhead