XDocument: saving XML to file without BOM

Question

I'm generating an utf-8 XML file using XDocument.

XDocument xml_document = new XDocument(
                    new XDeclaration("1.0", "utf-8", null),
                    new XElement(ROOT_NAME,                    
                    new XAttribute("note", note)
                )
            );
...
xml_document.Save(@file_path);

The file is generated correctly and validated with an xsd file with success.

When I try to upload the XML file to an online service, the service says that my file is wrong at line 1; I have discovered that the problem is caused by the BOM on the first bytes of the file.

Do you know why the BOM is appended to the file and how can I save the file without it?

As stated in Byte order mark Wikipedia article:

While Unicode standard allows BOM in UTF-8 it does not require or recommend it. Byte order has no meaning in UTF-8 so a BOM only serves to identify a text stream or file as UTF-8 or that it was converted from another format that has a BOM

Is it an XDocument problem or should I contact the guys of the online service provider to ask for a parser upgrade?

Cubitiere

Answer 1

Use an XmlTextWriter and pass that to the XDocument's Save() method, that way you can have more control over the type of encoding used:

var doc = new XDocument(
    new XDeclaration("1.0", "utf-8", null),
    new XElement("root", new XAttribute("note", "boogers"))
);
using (var writer = new XmlTextWriter(".\\boogers.xml", new UTF8Encoding(false)))
{
    doc.Save(writer);
}

The UTF8Encoding class constructor has an overload that specifies whether or not to use the BOM (Byte Order Mark) with a boolean value, in your case false.

The result of this code was verified using Notepad++ to inspect the file's encoding.

Sandasandakan

When you open it with Notepad++ is it still in utf-8 even using new UTF8Encoding(false)? – Cubitiere

I thought you wanted it in UTF-8, just without the BOM? – Sandasandakan

yep, that's correct. I was just asking if new UTF8Encoding(false) could have some other implication. – Cubitiere

Nope, the boolean value passed to the UTF8Encoding's constructor simply controls whether it includes a BOM. true to include, false to omit. – Sandasandakan

Consider adding writer.Formatting = Formatting.Indented; – Limb

Kevin, that would depend entirely on whether the file was intended to be viewed by humans, otherwise it's just wasted bytes. The question did not provide enough details to presume either way. – Sandasandakan

Warning: Dercsár's solution is better. "Starting with the .NET Framework 2.0, we recommend that you create XmlWriter instances by using the XmlWriter.Create method and the XmlWriterSettings class to take advantage of new functionality.". Source: XmlTextWriter Constructor (String, Encoding) (System.Xml) – Jat

Answer 2

First of all: the service provider MUST handle it, according to XML spec, which states that BOM may be present in case of UTF-8 representation.

You can force to save your XML without BOM like this:

XmlWriterSettings settings = new XmlWriterSettings();
settings.Encoding = new UTF8Encoding(false); // The false means, do not emit the BOM.
using (XmlWriter w = XmlWriter.Create("my.xml", settings))
{
    doc.Save(w);
}

(Googled from here: http://social.msdn.microsoft.com/Forums/en/xmlandnetfx/thread/ccc08c65-01d7-43c6-adf3-1fc70fdb026a)

Bartels

BOM may be present in case of UTF-8 representation can you point me to that specific document? – Cubitiere

Here you go: w3.org/TR/2006/REC-xml-20060816/#charencoding First paragraph: "All XML processors MUST be able to read entities in both the UTF-8 and UTF-16 encodings." UTF-8 encoding enables (though not requires) BOM (see Joe's comment below), therefore XML processors must be able process UTF-8 files with BOM. – Ostosis

"While Unicode standard allows BOM in UTF-8, it does not require or recommend it. Byte order has no meaning in UTF-8" - en.wikipedia.org/wiki/Byte_order_mark – Sandasandakan

Warning: doing this instead of just doc.Save(filename) has a side-effect: everything is written on one line. If you'd like your file to remain human-readable, consider adding settings.Indent = true; in this answer's code. – Jat

Answer 3

The most expedient way to get rid of the BOM character when using XDocument is to just save the document, then do a straight File read as a file, then write it back out. The File routines will strip the character out for you:

        XDocument xTasks = new XDocument();
        XElement xRoot = new XElement("tasklist",
            new XAttribute("timestamp",lastUpdated),
            new XElement("lasttask",lastTask)
        );
        ...
        xTasks.Add(xRoot);
        xTasks.Save("tasks.xml");

        // read it straight in, write it straight back out. Done.
        string[] lines = File.ReadAllLines("tasks.xml");
        File.WriteAllLines("tasks.xml",lines);

(it's hoky, but it works for the sake of expediency - at least you'll have a well-formed file to upload to your online provider) ;)

Primary

Just use XmlWriter.Create with the XmlWriterSettings.Indent = true;. Here you can format your output just as you see fit. – Baranowski

Answer 4

By UTF-8 Documents

String XMLDec = xDoc.Declaration.ToString();
StringBuilder sb = new StringBuilder(XMLDec);
sb.Append(xDoc.ToString());
Encoding encoding = new UTF8Encoding(false); // false = without BOM
File.WriteAllText(outPath, sb.ToString(), encoding); 

Harmonie

As already answered multiple times and its also vary vague why you're using a StringBuilder here while the document can save itself, as shown in other answers. Explain. – Hass