Is it possible to extract text by page for word/pdf files using Apache Tika?

Question

All the documentation I can find seems to suggest I can only extract the entire file's content. But I need to extract pages individually. Do I need to write my own parser for that? Is there some obvious method that I am missing?

Twylatwyman

Answer 1

Actually Tika does handle pages (at least in pdf) by sending elements <div><p> before page starts and </p></div> after page ends. You can easily setup page count in your handler using this (just counting pages using only <p>):

public abstract class MyContentHandler implements ContentHandler {
    private String pageTag = "p";
    protected int pageNumber = 0;
    ...
    @Override
    public void startElement (String uri, String localName, String qName, Attributes atts) throws SAXException  {  

        if (pageTag.equals(qName)) {
            startPage();
        }
    }

    @Override
    public void endElement (String uri, String localName, String qName) throws SAXException {  

        if (pageTag.equals(qName)) {
            endPage();
        }
    }

    protected void startPage() throws SAXException {
    pageNumber++;
    }

    protected void endPage() throws SAXException {
    return;
    }
    ...
}

When doing this with pdf you may run into the problem when parser doesn't send text lines in proper order - see Extracting text from PDF files with Apache Tika 0.9 (and PDFBox under the hood) on how to handle this.

Leatri

Just counting <p> tags also counts normal paragraphs, not just pages, at least for me. – Immigration

Answer 2

You'll need to work with the underlying libraries - Tika doesn't do anything at the page level.

For PDF files, PDFBox should be able to give you some page stuff. For Word, HWPF and XWPF from Apache POI don't really do page level things - the page breaks aren't stored in the file, but instead need to be calculated on the fly based on the text + fonts + page size...

Mood

So while Tika uses PDFBox under the hood, it doesn't provide the same breadth of functionality that PDFBox does? I'm especially conerned that from what I see Tika doesn't allow you to set start - end pages the way PDFBox allows you -- as this SO thread demonstrate #6840287 – Contracted

Apache Tika provides common functionality across a very wide range of file formats. It'll never expose everything that each library does, instead it makes life simple and consistent – Mood

So if I want to get page-by-page action with PDF's and the such, Tika won't get me there, and I basically should use PDFBox? – Contracted

Answer 3

You can get the number of pages in a Pdf using the metadata object's xmpTPg:NPages key as in the following:

Parser parser = new AutoDetectParser();
Metadata metadata = new Metadata();
ParseContext parseContext = new ParseContext();
parser.parse(fis, handler, metadata, parseContext);
metadata.get("xmpTPg:NPages");

Philbrick

This doesn't answer the actual question. The question is not about how to get the total number of pages but about how to extract text on a page by page basis. – Topping