Existing API for NLP in C++?

Question

Is/are there existing C++ NLP API(s) out there? The closest thing I have found is CLucene, a port of Lucene. However, it seems a bit obsolete and the documentation is far from complete.

Ideally, this/these API(s) would permit tokenization, stemming and PoS tagging.

Nabila

possible duplicate of c/c++ NLP library – Rosarosabel

@Rosarosabel this question is four years old and doesn't have an adequate answer, since it focuses on PoS taggers. – Nabila

I think the OP meant "Part of Speech" for PoS – Sneaker

Answer 1

Freeling is written in C++ too, although most people just use their binaries to run the tools: http://devel.cpl.upc.edu/freeling/downloads?order=time&desc=1

Try something like DyNet, it's a generic neural net framework but most of its processes are focusing on NLP because the maintainers are creators of the NLP community.

Or perhaps Marian-NMT, it was designed for sequence-to-sequence model machine translation but potentially many NLP tasks can be structured as a sequence-to-sequence task.

---

Outdated

Maybe you can try Ellogon http://www.ellogon.org/ , they have GUI support and also C/C++ API for NLP too.

Rosarosabel

Both seem interesting, I'll give them a look. – Nabila

Both seem to be dead project. Ellogon for sure is. – Fawnia

Answer 2

if you remove the restriction on c++ , you get the perfect NLTK (python)

the remaining effort is then interfacing between python and c++.

Restivo

nah, we know NLTK a cython / c/c++ port would exponentially increase the processing time for realistic big data. – Rosarosabel

NLTK is a toy and education system (and it was designed as one) not a practical solution. – Fawnia

Answer 3

Apache Lucy would get you part of the way there. It is under active development.

Change

It's a good start, but since it is a search engine, there is a lot of features I don't need. Also, NLP capability is a bit limited. I'll keep observing it, though. – Nabila

Answer 4

Maybe you can use Weka-C++. It's the very popular Weka library for machine learning and data mining (including NLP) ported from Java to C++.

Weka supports tokenization and stemming, you'll probably need to train a classifier for PoS tagging.

I only used Weka with Java though, so I'm afraid can't give you more details on this version.

Eulogist

While Weka itself is quite good, weka C++ seems to be a dead project (last commit en 2007) – Nabila

Answer 5

There is TurboParser by André Martins at CMU, also has a Python wrapper. There is is an online demo for it.

Oldfangled

Answer 6

This project provides free (even for commercial use) state-of-the-art information extraction tools. The current release includes tools for performing named entity extraction and binary relation detection as well as tools for training custom extractors and relation detectors.

MITIE is built on top of dlib, a high-performance machine-learning library, MITIE makes use of several state-of-the-art techniques including the use of distributional word embeddings and Structural Support Vector Machines[3]. MITIE offers several pre-trained models providing varying levels of support for both English and Spanish, trained using a variety of linguistic resources (e.g., CoNLL 2003, ACE, Wikipedia, Freebase, and Gigaword). The core MITIE software is written in C++, but bindings for several other software languages including Python, R, Java, C, and MATLAB allow a user to quickly integrate MITIE into his/her own applications.

https://github.com/mit-nlp/MITIE

Airedale