Extract / Identify Tables from PDF python [closed]

Question

Are there any open source libraries that support table identification & extraction?

By this I mean:

1. Identify a table structure exists
2. Classify the table from its contents
3. Extract data from the table in a useful output format e.g. JSON / CSV etc.

I have looked through similar questions on this topic and found the following:

• PDFMiner which addresses problem 3, but it seems the user is required to specify to PDFMiner where a table structure exists for each table (correct me if I'm wrong)
• pdf-table-extract which attempts to address problem 1 but according to the To-Do list, cannot currently identify tables that are separated by whitespace. This is a problem as all tables in my PDFs are separated by whitespace!

Currently, I am thinking that I would have to spend a lot of time developing a Machine Learning solution to identify table structures from PDFs. Therefore, any alternative approaches would be more than welcome!

Pericline

If you can use tools beyond python, too, you might want to take a look at tabula. – Shinbone

thanks. Will definitely look into that. I'm keen on finding a solution in python though because of the speed in which python can be written – Pericline

@Alexander McFarlane: Try SLICEmyPDF in 1 of the answers at #56018202 – Poll

See https://mcmap.net/q/340221/-how-can-i-extract-tables-as-structured-data-from-pdf-documents/562769 – Leggat

Answer 1

You should definitely have a look at this answer of mine:

• Extracting table contents from a collection of PDF files

and also have a look at all the links included therein.

Tabula/TabulaPDF is currently the best table extraction tool that is available for PDF scraping.

Lithic

just an update on the effectiveness of this answer... I hacked a solution together using tabula last year to iterate through about 100 PDFs that had a few formats in common. It wasn't pretty but it was the best of the worst and saved significant time. – Pericline

Will pypi.python.org/pypi/pdftable satisfy the requirements? – Wadmal

it just works on textbased pdfs and not on images is there anything similiar to this where it can extract data from pdf images ? – Acherman

@Sundeep: Of course it can only work on text-based PDFs. If you want to extract tables from an image, you have to attempt running a process of OCR (optical character recognition) on the image first and then apply the table extraction on the text. Final result quality will largely depend on success of the OCR step. There is nothing which would be able to extract tables (or texts) directly from image-only PDFs. – Lithic

Iam looking for tools that can do that btw thanks for the info @KurtPfeifle – Acherman

@Sundeep: You could start looking which tools are mentioned here: stackoverflow.com/questions/tagged/ocr – Lithic

Answer 2

After many fruitful hours of exploring OCR libraries, bounding boxes and clustering algorithms - I found a solution so simple it makes you want to cry!

I hope you are using Linux;

pdftotext -layout NAME_OF_PDF.pdf

AMAZING!!

Now you have a nice text file with all the information lined up in nice columns, now it is trivial to format into a csv etc..

It is for times like this that I love Linux, these guys came up with AMAZING solutions to everything, and put it there for FREE!

Trinary

I was able to get pdftotext on Windows 10. Just download the [XPDFTools][1] for Windows. [1]: xpdfreader.com/download.html – Schrecklichkeit

@Schrecklichkeit just got back to it today... basically I'm taking the text file generated by pdftotext, splitting it into a list of strings (one for each line), creating a set of field names using csv.DictWriter(), and then looping through each line, slicing it into the fields I want and then feeding those back to DictWriter. HTH. gist.github.com/memilanuk/c6e0bb9f98076a172d4f39d044ed6ecf – Berton

This library is written for Python 2.x version and doesn't work's with Python 3.x. – Chipper

Also possible on Mac superuser.com/questions/56272/… – Marital

Answer 3

I'd just like to add to the very helpful answer from Kurt Pfeifle - there is now a Python wrapper for Tabula, and this seems to work very well so far: https://github.com/chezou/tabula-py

This will convert your PDF table to a Pandas data frame. You can also set the area in x,y co-ordinates which is obviously very handy for irregular data.

Servility