Is there a way to escape a CDATA end token in xml?

Question

I was wondering if there is any way to escape a CDATA end token (]]>) within a CDATA section in an xml document. Or, more generally, if there is some escape sequence for using within a CDATA (but if it exists, I guess it'd probably only make sense to escape begin or end tokens, anyway).

Basically, can you have a begin or end token embedded in a CDATA and tell the parser not to interpret it but to treat it as just another character sequence.

Probably, you should just refactor your xml structure or your code if you find yourself trying to do that, but even though I've been working with xml on a daily basis for the last 3 years or so and I have never had this problem, I was wondering if it was possible. Just out of curiosity.

Edit:

Other than using html encoding...

Messere

First, i accept the answer as correct but note: Nothing precludes someone from encoding > as > within CData to ensure embedded ]]> will not be parsed as CDEnd. It simply means it's unexpected and that & must FIRST be encoded as & too so that the data can be properly decoded. Users of the document must know to decode this CData too. It's not unheard of since part of the purpose of CData is to contain content that a specific consumer understands how to handle. Such a CData just can't be expected to be interpreted properly by any generic consumer. – Bluepencil

@nix, CDATA just provides an explicit way to declare text node content such that language tokens within (other than ]]>) do not get parsed. It specifically does not expand entity references like &gt; for this reason, so in a CDATA block, that just means those four characters, not '>'. To put it in perspective: in the xml spec, all text content is called "cdata", not just these sequences ("character data"). Also it's not about specific consuming agents. (Such a thing does exist though -- processing instructions (<?target instruction?>). – Hypogeous

(I should add, even if this sort of thing runs contrary to the original intent of the node, all is fair in the long & torturous battle with XML. I just feel it could be useful for readers to know that <![CDATA[]]> was not actually designed for that purpose.) – Hypogeous

@Hypogeous CDATA was designed to allow anything: they are used to escape blocks of text containing characters which would otherwise be recognized as markup That implies CDATA too since it is also markup. But, in fact, you don't need the double encoding I implied. ]]> is an acceptable means of encoding a CDEnd within a CDATA. – Bluepencil

True, you wouldn't need double encoding -- but you would still need the agent to have special knowledge, since the parser wouldn't parse &gt; as >. That's what you mean though, I think? That you could replace them as you see fit, after parsing? – Hypogeous

Answer 1

You cannot escape a CDATA end sequence. Production rule 20 of the XML specification is quite clear:

[20]    CData      ::=      (Char* - (Char* ']]>' Char*))

EDIT: This product rule literally means "A CData section may contain anything you want BUT the sequence ']]>'. No exception.".

EDIT2: The same section also reads:

Within a CDATA section, only the CDEnd string is recognized as markup, so that left angle brackets and ampersands may occur in their literal form; they need not (and cannot) be escaped using "<" and "&". CDATA sections cannot nest.

In other words, it's not possible to use entity reference, markup or any other form of interpreted syntax. The only parsed text inside a CDATA section is ]]>, and it terminates the section.

Hence, it is not possible to escape ]]> within a CDATA section.

EDIT3: The same section also reads:

2.7 CDATA Sections

[Definition: CDATA sections may occur anywhere character data may occur; they are used to escape blocks of text containing characters which would otherwise be recognized as markup. CDATA sections begin with the string "<![CDATA[" and end with the string "]]>":]

Then there may be a CDATA section anywhere character data may occur, including multiple adjacent CDATA sections inplace of a single CDATA section. That allows it to be possible to split the ]]> token and put the two parts of it in adjacent CDATA sections.

ex:

<![CDATA[Certain tokens like ]]> can be difficult and <invalid>]]> 

should be written as

<![CDATA[Certain tokens like ]]]]><![CDATA[> can be difficult and <valid>]]> 

Dodecagon

Indeed. Well, I'm not an academic type but as I said in the question, I'm just curious about this. To be honest, I'll just take your word on this, because I can barely make sense out of the syntax used for the rule. Thanks for your answer. – Messere

It reads like this: Char* (the set of all character sequences) - (except) Char* ']]>' Char* (the set of all character sequences that include the substring ']]>'). – Dodecagon

Thanks for the extra clarification. I'm accepting your answer as the one that better addresses the question I asked. (S. Lott's answer provides a work-around, which is fine, although it doesn't specifically deal with an actual escape char or sequence. – Messere

This is not an academic question. Think about an RSS feed of a blog post that contains a discussion about CDATA. – Insuppressible

I meant "academic" in the sense: "interesting to discuss, but without practical use". Generally, CDATA is not useful, it's just a way to serialize XML text, and it's semantically equivalent to escaping special chars using character entities &lt; &gt; and &quot;. Characters entities is the simplest, most robust and most general solution, so use that instead of CDATA sections. If you use a proper XML library (instead of building XML out of strings) you don't even have to think about it. – Dodecagon

I just got bitten by this one because I am trying to encode some compressed Javascript into a <script> tag like: <script>/*<![CDATA[*/javascript goes here/*]]>*/</script> and my javascript includes just that sequence! I like the idea of splitting into multiple CDATA sections ... – Grand

If you were to add a CDATA code snippet in Sublime Text, it would require that you escape the ending sequence (configuration of Sublime is done almost exclusively through JSON and XML files). – Ichthyic

@NickT Instead of escaping the ending text in Sublime, you can do this: ]${1:Delete me then move along--required to escape CDATA end-tag}]>. Tools > New Snippet... annoys me, because it prints the snippet template into a new file. I don't want it a new file, so I just duplicated the blank snippet text itself into another snippet file...hence the need. – Parve

I experienced this in the real world. While reading the wikipedia dump and writing another xml file I encountered this on the page for the National Transportation Safety Board. It contained US$>100 million (2013) for the budget in the infobox. The source xml contained [[United States dollar|US$]]>100 million (2013) which was translated to [[United States dollar|US$]]>100 million (2013) by the reader and the writer opted to use CDATA to escape the text and failed. – Clemen

@Dodecagon re: it's just a way to serialize XML text or binary (unprintable) data. re: Characters entities is the simplest, most robust and most general solution for text that might confuse the XML parser, but if there are lots of them, it may be more space efficient to use CDATA. – Centuple

re: If you use a proper XML library and a proper library will have methods for adding CDATA (printable or unprintable) which will deal with the escape for you, if it needs to. Using a proper library is definitely the way to go. – Centuple

Re @jesse-chisholm: I am not sure what you are trying to say. CDATA might be more space efficient, but not in a way that should matter, since nobody should be transferring xml data that is not gzipped. After parsing, the memory usage should be the same. – Dodecagon

@ddaa: I was referring to the comment Characters entities is the simplest, most robust and most general solution, so use that instead of CDATA sections. If you use a proper XML library (instead of building XML out of strings) you don't even have to think about it. I was agreeing that using a proper library was better than building XML by hand, but disagreeing that entities are always the most robust, because if you have lots of them, then a CDATA is more efficient. Either way a proper library will handle it for you. And gzip makes the data binary which really needs CDATA. – Centuple

so, the answer is obvious: ]]> must be replaced with: ]]>]]&gt;<![CDATA[, in other words: close the current CDATA, type a "normal" ]]> but escaping the closing > and then open another CDATA. This would to the trick. – Bs

The answer is correct. CDATA sections do not escape content. I disagree whether this is academic though. If you are using XML format to store content in CDATA sections, then you can't store any XML content since it cannot tell the difference between content and markup. For this reason, the design of XML is broken. It fails the fundamental rule of parsing and delimiters: that you can embed delimiters in content with escaping. The design of CDATA breaks this rule. There are plenty other things wrong with XML as well, like how it's entitled to mess with whitespace in content. Use JSON. – Diadelphous

My point is that CDATA is useless in XML. It adds no expressiveness (everything you can do with CDATA you can do without it) and it provides an idiom that invites incorrect an fragile patterns: producing XML by string interpolation, and consuming XML without a proper parser. Therefore CDATA must be avoided. Therefore limitations in CDATA are "academic". – Dodecagon

Good answer, though I'd actually call escaping replacing ]]> with ]]]]><![CDATA[>, which, as you demonstrated, works. – Elena

Answer 2

You have to break your data into pieces to conceal the ]]>.

Here's the whole thing:

<![CDATA[]]]]><![CDATA[>]]>

The first <![CDATA[]]]]> has the ]]. The second <![CDATA[>]]> has the >.

Gambetta

Answer 3

simply replace ]]> with ]]]]><![CDATA[>

Linette

Answer 4

You do not escape the ]]> but you escape the > after ]] by inserting ]]><![CDATA[ before the >, think of this just like a \ in C/Java/PHP/Perl string but only needed before a > and after a ]].

BTW,

S.Lott's answer is the same as this, just worded differently.

Commonwealth

This way of saying it gives people the wrong idea. This is not escaping. ]]]]><![CDATA[> isn't some magical sequence for ]]>. ]]]]> has ]] characters as data, and ]]> ends the current CDATA section. <![CDATA[> starts a new CDATA section and puts > in it. They are actually two different elements and will be treated differently when working with a DOM parser. You should be aware of that. This way of doing it is similar to ]]]><![CDATA[]>, except it puts ] in the first and ]> in the second CDATA. The difference remains. – Preclude

The difference is overstated, since CDATA content is treated as a literal span of escaped text. Only when messing with the DOM does it really matter, and at that level you're dealing with other invisible boundaries anyway like text, comment, and processing instruction nodes. – Acetanilide

Answer 5

S. Lott's answer is right: you don't encode the end tag, you break it across multiple CDATA sections.

How to run across this problem in the real world: using an XML editor to create an XML document that will be fed into a content-management system, try to write an article about CDATA sections. Your ordinary trick of embedding code samples in a CDATA section will fail you here. You can imagine how I learned this.

But under most circumstances, you won't encounter this, and here's why: if you want to store (say) the text of an XML document as the content of an XML element, you'll probably use a DOM method, e.g.:

XmlElement elm = doc.CreateElement("foo");
elm.InnerText = "<[CDATA[[Is this a problem?]]>";

And the DOM quite reasonably escapes the < and the >, which means that you haven't inadvertently embedded a CDATA section in your document.

Oh, and this is interesting:

XmlDocument doc = new XmlDocument();

XmlElement elm = doc.CreateElement("doc");
doc.AppendChild(elm);

string data = "<![[CDATA[This is an embedded CDATA section]]>";
XmlCDataSection cdata = doc.CreateCDataSection(data);
elm.AppendChild(cdata);

This is probably an ideosyncrasy of the .NET DOM, but that doesn't throw an exception. The exception gets thrown here:

Console.Write(doc.OuterXml);

I'd guess that what's happening under the hood is that the XmlDocument is using an XmlWriter produce its output, and the XmlWriter checks for well-formedness as it writes.

Oxidation

Well, I had an almost "real world" example. I usually load Xml from Flash that contains html markup within CDATA sections. Having a way to escape it could be useful, I guess. But anyway, in that case, the CDATA content is usually valid XHTML, and so the "outer" CDATA could be avoided altogether. – Messere

CDATA can nearly always be avoided altogether. I find that people who struggle with CDATA very frequently don't understand what they're really trying to do and/or how the technology they're using really works. – Oxidation

Oh, I should also add that the only reason that the CMS I alluded to in my answer used CDATA was that I wrote it, and I didn't understand what I was really trying to do and/or how the technology works. I didn't need to use CDATA. – Oxidation

If you're using .net, the preceding comment about CDATA being avoidable is spot on - just write the content as a string and the framework will do all the escaping (and unescaping on read) for you from the real world....... xmlStream.WriteStartElement("UnprocessedHtml"); xmlStream.WriteString(UnprocessedHtml); xmlStream.WriteEndElement(); – Stines

Answer 6

Here's another case in which ]]> needs to be escaped. Suppose we need to save a perfectly valid HTML document inside a CDATA block of an XML document and the HTML source happens to have it's own CDATA block. For example:

<htmlSource><![CDATA[ 
    ... html ...
    <script type="text/javascript">
        /* <![CDATA[ */
        -- some working javascript --
        /* ]]> */
    </script>
    ... html ...
]]></htmlSource>

the commented CDATA suffix needs to be changed to:

        /* ]]]]><![CDATA[> *//

since an XML parser isn't going to know how to handle javascript comment blocks

Eckstein

This is not a special case. Simply replace ]]> with ]]]]><![CDATA[> still applies here. The fact that it's JavaScript, or commented is not important. – Linette

Answer 7

In PHP: '<![CDATA['.implode(explode(']]>', $string), ']]]]><![CDATA[>').']]>'

Hubsher

Answer 8

A cleaner way in PHP:

   function safeCData($string)
   {
      return '<![CDATA[' . str_replace(']]>', ']]]]><![CDATA[>', $string) . ']]>';
   }

Don't forget to use a multibyte-safe str_replace if required (non latin1 $string):

   function mb_str_replace($search, $replace, $subject, &$count = 0)
   {
      if (!is_array($subject))
      {
         $searches = is_array($search) ? array_values($search) : array ($search);
         $replacements = is_array($replace) ? array_values($replace) : array ($replace);
         $replacements = array_pad($replacements, count($searches), '');
         foreach ($searches as $key => $search)
         {
            $parts = mb_split(preg_quote($search), $subject);
            $count += count($parts) - 1;
            $subject = implode($replacements[$key], $parts);
         }
      }
      else
      {
         foreach ($subject as $key => $value)
         {
            $subject[$key] = mb_str_replace($search, $replace, $value, $count);
         }
      }
      return $subject;
   }

Bharal

Can you explain your downvote? Saying that I did a mistake isn't as useful as explaining where it is. – Bharal

There is no need to do multibyte safe replace if you are using UTF-8. I didn't downvote though:) – Bolshevism

Answer 9

I'd just like to add that it also works if you break the CDATA end tag ]]> between the ]], like this: ] ]]><![CDATA[ ]>

ex.

<![CDATA[Certain tokens like ]]]><![CDATA[]> can be difficult and <valid> but <unconventional>]]> 

However, it is the globally accepted convention to break the ]]> before the > as shown in the other answers here.

<![CDATA[Certain tokens like ]]]]><![CDATA[> can be difficult and <valid> and <conventional>]]> 

Saleme

Answer 10

See this structure:

<![CDATA[
   <![CDATA[
      <div>Hello World</div>
   ]]]]><![CDATA[>
]]>

For the inner CDATA tag(s) you must close with ]]]]><![CDATA[> instead of ]]>. Simple as that.

Lexical