Login/Register
Error 'failed to load external entity' when using Python lxml
Asked Answered
pythonxmllxmlelementtree
F

4

35

I'm trying to parse an XML document I retrieve from the web, but it crashes after parsing with this error:

': failed to load external entity "<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="GreenButtonDataStyleSheet.xslt"?>

That is the second line in the XML that is downloaded. Is there a way to prevent the parser from trying to load the external entity, or another way to solve this? This is the code I have so far:

import urllib2
import lxml.etree as etree

file = urllib2.urlopen("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")
data = file.read()
file.close()

tree = etree.parse(data)
Fink answered 4/5, 2012 at 23:57 Comment(0)
M
38

In concert with what mzjn said, if you do want to pass a string to etree.parse(), just wrap it in a StringIO object.

Example (python2):

from lxml import etree
from StringIO import StringIO

myString = "<html><p>blah blah blah</p></html>"

tree = etree.parse(StringIO(myString))

Example (python3) import from io instead of StringIO:

from lxml import etree
from io import StringIO

myString = "<html><p>blah blah blah</p></html>"

tree = etree.parse(StringIO(myString))

This method is used in the lxml documentation.

Marbleize answered 20/10, 2012 at 1:13 Comment(1)
For python3: from io import StringIO – Veljkov
R
16

etree.parse(source) expects source to be one of

  • a file name/path
  • a file object
  • a file-like object
  • a URL using the HTTP or FTP protocol

The problem is that you are supplying the XML content as a string.

You can also do without urllib2.urlopen(). Just use

tree = etree.parse("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")

Demonstration (using lxml 2.3.4):

>>> from lxml import etree
>>> tree = etree.parse("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")
>>> tree.getroot()
<Element {http://www.w3.org/2005/Atom}feed at 0xedaa08>
>>>

In a competing answer, it is suggested that lxml fails because of the stylesheet referenced by the processing instruction in the document. But that is not the problem here. lxml does not try to load the stylesheet, and the XML document is parsed just fine if you do as described above.

If you want to actually load the stylesheet, you have to be explicit about it. Something like this is needed:

from lxml import etree

tree = etree.parse("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")

# Create an _XSLTProcessingInstruction object
pi = tree.xpath("//processing-instruction()")[0] 

# Parse the stylesheet and return an ElementTree
xsl = pi.parseXSL()
Ro answered 6/5, 2012 at 10:14 Comment(9)
@Duke: Thank you! It's nice to finally get some positive feedback. – Ro
I'm getting this error while parsing a URL. Any idea how to disable loading those "external entities"? I have no interest in stylesheets, just want to parse anchors from the page. – Kweiyang
@MightyPork: This question is not really about "external entities"; the error message is misleading. The problem here is that the OP uses etree.parse() on a string object, which does not work. If you have a related problem, I think you should ask a new question. – Ro
@Ro I asked indeed, but all advice I got was to use different library to fetch the url. I thought you have some more insight into the problem and could help. – Kweiyang
Yes downvoter, this is very probably the problem. It was in my case. – Repel
etree.parse('google.com'): OSError: Error reading file 'google.com': failed to load external entity "google.com" – Flambeau
In my case, I got the error because I tried to open an https url with parse. – Tannate
@ShaneLu: Yes, for some reason HTTPS does not work. Only HTTP and FTP protocols are supported (as stated in the lxml documentation). – Ro
Thanks for the tip. I was getting the "external entity" error with valgrind on a C program. I had to switch from file.nam to file:///file.nam to get the lxml to work with valgrind. Without valgrind, lxml works with file.nam. – Downs
F
5

lxml docs for parse says To parse from a string, use the fromstring() function instead.

parse(...)
    parse(source, parser=None, base_url=None)

    Return an ElementTree object loaded with source elements.  If no parser
    is provided as second argument, the default parser is used.

    The ``source`` can be any of the following:

    - a file name/path
    - a file object
    - a file-like object
    - a URL using the HTTP or FTP protocol

    To parse from a string, use the ``fromstring()`` function instead.

    Note that it is generally faster to parse from a file path or URL
    than from an open file object or file-like object.  Transparent
    decompression from gzip compressed sources is supported (unless
    explicitly disabled in libxml2).
Ferment answered 25/6, 2013 at 20:0 Comment(0)
G
2

You're getting that error because the XML you're loading references an external resource:

<?xml-stylesheet type="text/xsl" href="GreenButtonDataStyleSheet.xslt"?>

LXML doesn't know how to resolve GreenButtonDataStyleSheet.xslt. You and I probably realize that it's going to be available relative to your original URL, http://www.greenbuttondata.org/data/15MinLP_15Days.xml...the trick is to tell lxml how to go about loading it.

The lxml documentation includes a section titled "Document loading and URL resolving", which has just about all the information you need.

Gamut answered 5/5, 2012 at 0:38 Comment(2)
Do you know if it's possible to turn off loading all external resources? I looked in the documentation but couldn't find anything. – Fink
"You're getting that error because the XML you're loading references an external resource". No. That is not why you get the error. Please see my answer. – Ro

© 2022 - 2024 — McMap. All rights reserved.