How to scale down the size and quality of an BufferedImage?

Question

I'm working on a project, a client-server application named 'remote desktop control'. What I need to do is take a screen capture of the client computer and send this screen capture to the server computer. I would probably need to send 3 to 5 images per second. But considering that sending BufferedImage directly will be too costly for the process, I need to reduce the size of the images. The image quality need not to be loss less.

How can I reduce the byte size of the image? Any suggestions?

Haemocyte

Answer 1

You can compress it with ZIP very easily by using GZIPInputStream and its output counterpart on the other end of the socket.

Edit:

Also note that you can create delta images for transmission, you can use a "transpartent color" for example (magic pink #FF00FF) to indicate that no change was made on that part of the screen. On the other side you can draw the new image over the last one ignoring these magic pixels.

Note that if the picture already contains this color you can change the real pink pixels to #FF00FE for example. This is unnoticable.

An other option is to transmit a 1-bit mask with every image (after painting the no-change pixels to an arbitrary color. For this you can change the color which is mostly used in the picture to result in the best compression ratio (optimal huffman-coding).

Gild

wow, thanks! i guess i have to work with a bit more, but i got the main theme alright – Haemocyte

Answer 2

Vbence's solution of using a GZIPInputStream is a good suggestion. The way this is done in most commercial software - Windows Remote Desktop, VNC, etc. is that only changes to the screen-buffer are sent. So you keep a copy on the server of what the client 'sees', and with each consecutive capture you calculate what is different in terms of screen areas. Then you only send these screen areas to the client along with their top-left coords, width, height. And update the server copy of the client 'view' with just these new areas.

That will MASSIVELY reduce the amount of network data you use, while I have been typing this answer, only 400 or so pixels (20x20) are changing with each keystroke. This on a 1920x1080 screen is just 1/10,000th of the screen, so clearly worth thinking about.

The only expensive part is how you calculate the 'difference' between one frame and the next. There are plenty of libraries out there to do that cheaply, most of them very mathematical (discrete cosine transform type stuff, way over my head), but it can be done relatively cheaply.

Jeffers

"The only expensive part is how you calculate the 'difference' between one frame and the next. There are plenty of libraries out there to do that cheaply, most of them very mathematical (discrete cosine transform type stuff, way over my head), but it can be done relatively cheaply." - can you give me the names of some libraries for this? – Haemocyte

Its actually not all that difficult to do yourself. Look at motion detection for security cameras for ideas codeproject.com/Articles/10248/Motion-Detection-Algorithms, given the fact that you know a frame buffer has no 'noise' you can optimise hugely. – Jeffers

It's a good approach but takes effort to implement. You generally don't want to transmit a single rectangle, but calculate the best combination of rectangles. With a single rect for example an animated tray icon can lead to transmitting almost the whole screen every time. – Gild

Answer 3

See this thread for how to encode to JPG with controllable compression/quality. The slider on the left is used to control the level.

Ultimately it would be better to encode the images directly to a video codec that can be streamed, but I am a little hazy on the details.

Damn

thanks for the link. i'll look into it. can you tell me a bit more about encoding image to video codec? how do i do that? any link would be helpful too – Haemocyte

Though I've used JMF to turn JPEG images to a MOV, I've not delved into generating data intended to be streamed. Or in other words, 'no'. Besides that, I think that lynks has provided a much better strategy (in only transmitting changes to the previous frame). – Damn

Answer 4

One way would be to use ImageIO API

ImageIO.write(buffimg, "jpg", new File("buffimg.jpg"));  

As for the quality and other parameters- I'm not sure, but it should be possible, just dig deeper.

Condense

"just dig deeper" Or see the code linked in my answer. ;) – Damn

i've tried it. my screen resolution is 1366*768 and a screen-capture produces an image of approximately 135KB. so, still it's way too costly! – Haemocyte