Working on tables in pdf using python [duplicate]

Question

I am working on a pdf file. There is number of tables in that pdf.
According to the table names given in the pdf, I wanted to fetch the data from that table using python.

I have worked on html, xlm parsing but never with pdf.
Can anyone tell me how to fetch tables from pdf using python?

Citrin

Answer 1

I think that you need a python parser library. The most famous is PDFMiner.

According to the documentation :

PDFMiner is a tool for extracting information from PDF documents. Unlike other PDF-related tools, it focuses entirely on getting and analyzing text data. PDFMiner allows one to obtain the exact location of text in a page, as well as other information such as fonts or lines. It includes a PDF converter that can transform PDF files into other text formats (such as HTML). It has an extensible PDF parser that can be used for other purposes than text analysis.

Sianna

Answer 2

This is a very complex problem and not solvable in general.

The reason for this is simply that the format PDF is too flexible. Some PDFs are only bitmaps (you would have to do your own OCR then—obviously not our topic here), some are a bunch of letters literally spilled out over the pages; this means that by parsing the text information in the PDF you could get single characters placed on some coordinates. In some cases these come in an orderly fashion (line by line, from left to right), but in some cases you will get rather random-like distributions, most commonly with and stuff, but also special characters, characters in a different font etc. can come way out of line.

The only proper approach is to place all characters according to their coordinates on a page model and then use heuristics to find out what the lines are.

I propose to have a look at your PDFs and the tables therein you want to parse before starting. Maybe they are alike all the time and well-parsable.

Good luck!

Launcelot

Answer 3

I had a similar problem recently, and wrote a library to help solve it: pdfquery.

PDFQuery creates an element tree from the PDF (using pdfminer, with some extra sugar) and lets you fetch elements from the page using JQuery or XPath selectors, based mostly on the text contents or locations of the elements. So to parse a table, you would first find where it is in the document by searching for the label:

label = pdf.pq(':contains("Name of your table")')
left_corner = float(label.attr('x0'))
bottom_corner = float(label.attr('y0'))

Then you would keep searching for lines underneath the table, until the search didn't return results:

page = label.closest('LTPage')
while 1:
    row = pdf.extract( [
             ('column_1', ':in_bbox("%s,%s,%s,%s")' % (left_corner+10, bottom_corner+40, left_corner+50, bottom_corner+20)),
             ('column_2', ':in_bbox("%s,%s,%s,%s")' % (left_corner+50, bottom_corner+40, left_corner+80, bottom_corner+20))
         ], page)
    if not row['column_1'] or row['column_2']:
        break
    print "Got row:", matches
    bottom_corner -= 20

This assumes that your rows are 20 pts high, the first one starts 20 pts below the label, the first column spans from 10 to 50 points from the left edge of the label, and the second column spans from 50 to 80 pts from the left edge of the label.

If you have blank lines or lines with varying heights, this is going to get more annoying. You may also need to use the merge_tags=None option to select individual characters rather than words, if the entries in the table are close enough to make the parser think it's just one line. But hopefully this gets you closer ...

Mikey

Answer 4

You can use Camelot to extract tabular data from your PDF and export it to your favorite format. Currently; CSV, Excel, JSON and HTML are supported. You can check out the documentation at: http://camelot-py.readthedocs.io. It would be helpful if you could post a link to your PDF. Here's a generic code example:

>>> import camelot
>>> tables = camelot.read_pdf('file.pdf')
>>> type(tables[0].df)
<class 'pandas.core.frame.DataFrame'>
>>> tables[0].to_csv('file.csv')

Disclaimer: I'm the author of the library.

Septuplet