.NET Image.Save occasionally generates a PNG with a bad IDAT chunk

Question

I have a C#/.NET utility I wrote that loads PNG images from disk

Bitmap b = Bitmap.FromStream(new MemoryStream(File.ReadAllBytes(filename))) as Bitmap;

performs several transformations on them (rotation, scaling, alpha) and then saves the resulting PNG images back to disk with different file names based on the transformations applied

b.Save(outputName, ImageFormat.Png);

I've successfully written thousands of PNGs using the utility. However, occasionally one of the PNGs fails to load in a separate program which uses libpng. In that program, libpng gives the error "Too many IDATs found"

Looking into the PNG file reveals a 'rogue' IDAT chunk at the end of the file just before the IEND chunk. One such IDAT chunk (and the following IEND chunk) looks like this in a hex editor. These are the final 24 bytes in the file.

IDAT: 0x00 0x00 0xFF 0xF4 0x49 0x44 0x41 0x54 0x35 0xAF 0x06 0x1E    
IEND: 0x00 0x00 0x00 0x00 0x49 0x45 0x4e 0x44 0xAE 0x42 0x60 0x82

The IDAT chunk length is shown as 0xFFF4. However, as is obvious, there aren't that many bytes in the IDAT chunk (or even the file for that matter.)

Has anyone else come across this problem? I can fix the problem in one of several ways. I can hand edit the PNG file to remove that last IDAT chunk (or set its size to 0.) I can run a secondary program which fixes broken PNGs. However, I'd like a C#/.NET solution which I can easily add to my original program. Ideally, I'd like a solution which doesn't require me to re-open the PNG as a binary file; check for the bad IDAT chunk; and re-write the PNG. However, I'm beginning to think that's what I'll need to do.

Prochronism

Do your images vary in size/row stride? Have you been able to narrow down if there is something unique (width, height, stride, format) to the images that are broken? – Unstopped

I think you're going to have to isolate the problem more. If you use C# to immediately read back each file you save, can you repro the error? If so, you need to track down the precise set of circumstances that led to that broken image and go from there. – Pianoforte

@Unstopped The image sizes and row strides are all over the spectrum. They are all Format32bppArgb. I've found nothing unique about the failing PNG. – Prochronism

If you rerun your code with the same inputs, do you get the same results? Same set of broken files? – Unstopped

@KirkWoll I added code in my program to re-load (Image.FromStream) the bad PNG just after it is save to disk. I then immediately re-save (Image.Save) it to disk with a different filename. The re-saved version still has the rogue IDAT chunk. – Prochronism

@Unstopped Yes, same inputs, same results. The one output PNG that is bad comes from an input PNG - let's call it X.png. X.png is a valid PNG. I take X.png and apply 73 different rotations to it; saving each rotated version to a separate file (X-rrr.png where rrr is the rotation in degrees.) 72 of those rotated versions are fine. One is bad. That happens to be 195 degrees of rotation, but I don't that that matters. – Prochronism

It seems other users have stumbled onto the same problem. Some code seem to recover from the problem and some don't (like libpng). Still doesn't explain where that extra chunk comes from though. – Unstopped

And you can find nothing that sets that specific image apart from the others? Must be some bug in the encoder, very strange. – Unstopped

@Unstopped Thanks for the input and for finding that libpng reference. Though libpng gives an error (or is it a warning) GIMP will successfully load the PNG with the rogue IDAT. It still spits the message out to the console. However, some apps which use libpng treat such an error/warning as a error and refuse to load the PNG instead of just skipping the bad IDAT. I do believe there is some error in the encoder used by .NET System.Drawing namespace. I have tried mucking around with the version of Save that takes encoder parameters, but with no luck. – Prochronism

System.Drawing.Bitmap uses GDI+, so this much be a major bug in windows (unless it's actually allowed by the png spec). One option would be to use a 3rd party encoder, google says there are a few around (freeimage.net, etc). – Unstopped

If you can reproduce it, you should fill a bug report. – Subtract

I have had a similar problem with this files below – Ewell

Original File docs.google.com/file/d/0B4RkktioBqoYNXUwZ0hvRWFqUXM/… – Ewell

Bad File docs.google.com/file/d/0B4RkktioBqoYYjV1MVhKUTZIckk/… – Ewell

I understand this is 4 years old, but am working on root causing this now. Am curious what OS you were on at the time, if you remember. Given the timeframe, presume this was Windows 7/Server 2012? – Dance

Yes, the OS at the time was Windows 7 64-bit. – Prochronism

I do not have an answer to your question, but in addressing the exact same issue we found a way to reproduce it. We figured out that the size of the resulting PNG-file is a key indicator of the problem being present. If the size of the file (in bytes) is 0x1001C + n * 0x10000 with n 0, 1, 2, 3, 4 (and probably larger values, but I cannot confirm that), the problem is consistently present. I posted a question with some code to reproduce the issue: #52101203 I hope this helps. – Forsake

What .NET framework is it on (aka name and version - .NET Framework, net core, mono, with version) as well as Windows version? Have you built a collection of these bad behaving images? This seems like Bitmap.Save() has a bug, or, more likely GDI+ PNG encoder that is responsible for transforming your bitmap to PNG, which is Windows dependent, including it's version and installed service packs. Though it is likely problem doesn't often surface. Also can you share some image for repro? – Lailaibach

Answer 1

Old question.

.NET is notoriously poor at handling images. The codecs are old win32 ones with many bugs.

.NET does not always free up the OS resources used when reading/writing image files even if you follow the recommended dispose and/or using methods.

Sesquiplane

Answer 2

This isn't really a complete answer since I'm unable to reproduce the image. But you could try loading the image using another one of the Bitmap constructors:

http://msdn.microsoft.com/en-us/library/0cbhe98f.aspx

Which would change your code into:

Bitmap b = new Bitmap(filename);

Because that constructor uses some native GDI+ functions to load the image into memory, instead of you reading the raw bytes:

http://dotnetframework.org/default.aspx/4@0/4@0/DEVDIV_TFS/Dev10/Releases/RTMRel/ndp/fx/src/CommonUI/System/Drawing/Bitmap@cs/1305376/Bitmap@cs

There might be a difference, even though the problem seems to be when writing the image to disk.

Orate

Thanks for the input. However, I've tried the other Bitmap constructors with no luck. I specifically chose the FromStream version because others can leave a file handle open longer than I want. – Prochronism