html parsing with lxml when there's no root tag

Question

I've been building a scaffolding library for sqlalchemy using lxml and formalchemy, and I'm having a hard time getting them to play nicely. specifically, formalchemy.FieldSet.render() returns a fragment of html with no root tag, and I cannot seem to figure out how to get lxml to parse it into something that can be included into an element tree:

what I get:

>>> lxml.etree.fromstring(formalchemy.FieldSet(toyschema.User(), session).render())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lxml.etree.pyx", line 2743, in lxml.etree.fromstring (src/lxml/lxml.etree.c:52665)
  File "parser.pxi", line 1573, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:79932)
  File "parser.pxi", line 1445, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:78709)
  File "parser.pxi", line 920, in lxml.etree._BaseParser._parseUnicodeDoc (src/lxml/lxml.etree.c:75083)
  File "parser.pxi", line 564, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:71739)
  File "parser.pxi", line 645, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:72614)
  File "parser.pxi", line 585, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:71955)
lxml.etree.XMLSyntaxError: Extra content at the end of the document, line 8, column 1

what I Want:

>>> ?????(formalchemy.FieldSet(toyschema.User(), session).render())
[<Element div at 0x1e48e10>, <Element script at 0x1e48cd0>, <Element div at 0x1e48730>, <Element div at 0x1ea1e60>]

My Crude work-around:

>>> lxml.etree.fromstring('<root>%s</root>' % formalchemy.FieldSet(toyschema.User(), session).render()).getchildren()
[<Element div at 0x1e48e10>, <Element script at 0x1e48cd0>, <Element div at 0x1e48730>, <Element div at 0x1ea1e60>]

Jahdai

Answer 1

For parsing HTML you want to use lxml.html, not lxml.etree.

If you want a list of fragments instead of wrapping them in a div you can use fragments_fromstring(string) as described here.

Nineveh

well, that seems to get a little closer, but it conditionally wraps the resulting tree into a div or span, which is not exactly what I had in mind. +1 for suggesting an improvement, though. – Jahdai

This will change tag names to lowercase. It's better to do lxml.etree.fromstring('<a>{}</a>'.format(string)) and then result.getchildren(). – Jerkin