How to tell lxml.etree.tostring(element) not to write namespaces in python?

Question

I have a huge xml file (1 Gig). I want to move some of the elements (entrys) to another file with the same header and specifications.

Let's say the original file contains this entry with tag <to_move>:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE some SYSTEM "some.dtd">
<some>
...
<to_move date="somedate">
    <child>some text</child>
    ...
...
</to_move>
...
</some>

I use lxml.etree.iterparse to iterate through the file. Works fine. When I find the element with tag <to_move>, let's assume it is stored in the variable element I do

new_file.write(etree.tostring(element))

But this results in

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE some SYSTEM "some.dtd">
<some>
...
<to_move xmlns:="some" date="somedate">  # <---- Here is the problem. I don't want the namespace.
    <child>some text</child>
    ...
...
</to_move>
...
</some>

So the question is: How to tell etree.tostring() not to write the xmlns:="some". Is this possible? I struggeled with the api-documentation of lxml.etree, but I couldn't find a satisfying answer.

This is what I found for etree.trostring:

tostring(element_or_tree, encoding=None, method="xml",
xml_declaration=None, pretty_print=False, with_tail=True,
standalone=None, doctype=None, exclusive=False, with_comments=True)

Serialize an element to an encoded string representation of its XML tree.

To me every one of the parameters of tostring() does not seem to help. Any suggestion or corrections?

Bogey

Answer 1

I often grab a namespace to make an alias for it like this:

someXML = lxml.etree.XML(someString)
if ns is None:
      ns = {"m": someXML.tag.split("}")[0][1:]}
someid = someXML.xpath('.//m:ImportantThing//m:ID', namespaces=ns)

You could do something similar to grab the namespace in order to make a regex that will clean it up after using tostring.

Or you could clean up the input string. Find the first space, check if it is followed by xmlns, if yes, delete the whole xmlns bit up to the next space, if no delete the space. Repeat until there are no more spaces or xmlns declarations. But don't go past the first >.

Zestful

Answer 2

There is a way to remove namespaces with XSLT:

import io
import lxml.etree as ET


def remove_namespaces(doc):
    # http://wiki.tei-c.org/index.php/Remove-Namespaces.xsl
    xslt='''<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="xml" indent="no"/>

    <xsl:template match="/|comment()|processing-instruction()">
        <xsl:copy>
          <xsl:apply-templates/>
        </xsl:copy>
    </xsl:template>

    <xsl:template match="*">
        <xsl:element name="{local-name()}">
          <xsl:apply-templates select="@*|node()"/>
        </xsl:element>
    </xsl:template>

    <xsl:template match="@*">
        <xsl:attribute name="{local-name()}">
          <xsl:value-of select="."/>
        </xsl:attribute>
    </xsl:template>
    </xsl:stylesheet>
    '''

    xslt_doc = ET.parse(io.BytesIO(xslt))
    transform = ET.XSLT(xslt_doc)
    doc = transform(doc)
    return doc

doc = ET.parse('data.xml')
doc = remove_namespaces(doc)
print(ET.tostring(doc))

yields

<some>

<to_move date="somedate">
    <child>some text</child>
</to_move>

</some>

Recitativo

I understand, the great idea behind namespaces. :-) But in this special case of mine there is no profit in it at all. That does not mean I will always ignore them. :-) Thanks! – Bogey

Answer 3

This is more in comment to the answer by 'unutbu' in which a suggestion to cleanup namespace was desired without giving example. this might be what you are looking for...

from lxml import objectify
objectify.deannotate(root, cleanup_namespaces=True)

Kettledrum

This won't remove just any namespaces. From the docs: "Recursively de-annotate the elements of an XML tree by removing 'py:pytype' and/or 'xsi:type' attributes and/or 'xsi:nil' attributes." – Kopje

This almost worked for me, I still have a xmlns attribute in the resulting string, is there a way to get rid of that too without using a regexp? – Cotoneaster

This works. Also, generating elements with objectify and parameter annotate=False works too: for instance: myE = objectify.ElementMaker(annotate=False) and then x = myE.rootElem(); you can serialize it using etree.tostring(x, method='xml') and get back just <rootElem/> – Sennit

Answer 4

This is an ancient question, but since 13 years later lxml still doesn't have this obvious function built in, the simplest way to deal with it is with a regex:

def get_text(element: etree.Element) -> str:
    s = etree.tostring(element).decode()
    if m := re.match(r'^<(\w*)[^>]*>((.|\n)+)<\/\1>', s):
        return m.group(2).strip()
    return s

This strips off the entire containing element <description blah blah blah><div>What we want<p>more stuff</p></div></description> and returns just the markup inside the <description>. The regex saves the opening element tag to match at the end (with the \1 backreference) just to be safe, but looking for a closing <\ works fine because the greedy match skips over all the others embedded in the content.

It would probably be better to raise a ValueError if the match fails, but this just returns the original ugly string, namespace and all.

Opinionated