Parsing big XML files using SAX parser (skip some lines/tags)

Question

I am currently developing an app that retrieves data from the internet using SAX. I used it before for parsing simple XML files like Google Weather API. However, the websites that I am interested in take parsing to the next level. The page is huge and looks messy. I only need to retrieve some specific lines; the rest is not useful for me.
Is it possible to skip those useless lines/tags, or do I have to go step by step?

Tawny

I think extended vtd-xml allows you to access xml with random access in a memory efficient fashion, far better than SAX or apache digester – Locally

Answer 1

I like commons-digester. It allows you to specify rules against particular tags. The rule gets executed only when the tag is encountered.

Digester is built over sax and hence has all the sax features plus the specificity that is required for selectively parsing specific tags. It also uses a stack that is pushed with new elements as and when the corresponding tag is encountered and is popped when the element ends.

I use it for parsing all my configuration files.

Check out digester at http://commons.apache.org/digester/

Bangup

Thanks a lot Raja , I will look into this solution !! – Tawny

Answer 2

Yes you can do it, just ignore the tags you are not interested in. But note that the entire document will have to be parsed for this (DefaultHandler impl)

public startElement(String uri, String localName, 
     String qName, Attributes attributes)  {
  if(localName.equals("myInterestingTag") {
     // do your thing....
  }
}

public void endElement(String uri, String localName, String qName) {
  if(localName.equals("myInterestingTag") {
     // do your thing....
  }
}

public void characters(char[] ch, int start, int length) {
  // if parsing myinteresting tag... do some stuff.
}

Danadanae

Thanks , that's exactly how i usually do it ;) ! – Tawny

Answer 3

Yes, you can skip. Just define those tag which you want and it will only fetch those tag values.

Nicoline

Answer 4

You can try to use XPath which will use SAX behind the scene to parse your xml. The downside here is that XML will be parsed on every call of Xpath evaluate method.

Leucas

Thanks for you response i will look into this possibility ! – Tawny

Answer 5

You you want to read specific tags then DOM parser is much faster than SAX parser..SAX parser is useful if you want to parse big XML files..

Initial

SAX parsing is much faster than DOM. DOM also requires that the entire document be brought into memory. – Chavira

Thanks a lot for your answer , i really appreciate your help , i will try to use DOM and see what outputs i get ! – Tawny

Answer 6

You can try a combination of TagSoup for creating a parseable XML document and XPath for fetching the interesting parts.

Busily

sadly using xpath is not that easy on android – Madore

Answer 7

See my answer to a similar question for a strategy of using SAX to skip/ignore tags:

Skipping nodes with sax

It involves switching ContentHandlers on the XMLReader. When you read a porting of the XML document you want to skip you simply swap in a ContentHandler that does nothing with the events. When the end of the section to be ignored is reached it passes control back to the content handler you were using to process the XML content.

Chavira

Your answer will certainly help achieve my goal ! thanks a lot -Cheers – Tawny