NLTK vs Stanford NLP

Question

I have recently started to use NLTK toolkit for creating few solutions using Python.

I hear a lot of community activity regarding using Stanford NLP. Can anyone tell me the difference between NLTK and Stanford NLP? Are they two different libraries? I know that NLTK has an interface to Stanford NLP but can anyone throw some light on few basic differences or even more in detail.

Can Stanford NLP be used using Python?

Snake

Well it depends. I chose Stanford NLP for it's entity recognition. Maybe you can decide on selecting the library based on running some sample tests against your data and see what you are most comfortable with. – Congeries

My experience is limited. A cursory study showed that Stanford's pattern is better and faster at POS tagging than NLTK. I did this work about 2 years ago. – Dearth

pattern (clips.ua.ac.be/pattern) don't belong to stanford. it's from CLIPS from University of Antwerpen... – Philippa

Answer 1

Can anyone tell me what is the difference between NLTK and Stanford NLP? Are they 2 different libraries ? I know that NLTK has an interface to Stanford NLP but can anyone throw some light on few basic differences or even more in detail.

(I'm assuming you mean "Stanford CoreNLP".)

They are two different libraries.

• Stanford CoreNLP is written in Java
• NLTK is a Python library

The main functional difference is that NLTK has multiple versions or interfaces to other versions of NLP tools, while Stanford CoreNLP only has their version. NLTK also supports installing third-party Java projects, and even includes instructions for installing some Stanford NLP packages on the wiki.

Both have good support for English, but if you are dealing with other languages:

• Stanford CoreNLP comes with models for English, Chinese, French, German, Spanish, and Arabic.
• NLTK comes with corpora in additional languages like Portugese, Russian, and Polish. Individual tools may support even more languages (e.g. no Danish corpora, but has a DanishStemmer).

That said, which one is "best" will depend on your specific application and required performance (what features you are using, language, vocabulary, desired speed, etc.).

Can Stanford NLP be used using Python?

Yes, there are a number of interfaces and packages for using Stanford CoreNLP in Python (independent of NLTK).

Theocritus

Thanks for the information , so does nlkt library in python uses StanfordNLP core ? Is Nltk more like an interface to StanfordNLP ? – Snake

NLTK is its own NLP package, which just so happens to provide an interface to Stanford NLP packages, among others. It's not "based on" Stanford CoreNLP or anything like that - unless NLTK specifically says a function / module / etc. is an interface to Stanford NLP, it's not. – Theocritus

Among NLTK and Stanford CoreNLP, It depends on which programming language you're more comfortable working with if you're only dealing with English Text. However, If you're dealing with languages other than English, Choice should be based on your availability. You could also check out variety of parsers available today, at the time of writing (December 2020). Kudos! – Chiropodist

Answer 2

The choice will depend upon your use case. NLTK is great for pre-processing and tokenizing text. It also includes a good POS tagger. Standford Core NLP for only tokenizing/POS tagging is a bit of overkill, because Standford NLP requires more resources.
But one fundamental difference is, you can't parse syntactic dependencies out of the box with NLTK. You need to specify a Grammar for that which can be very tedious if the text domain is not restricted. Whereas Standford NLP provides a probabilistic parser for general text as a down-loadable model, which is quite accurate. It also has built in NER (Named Entity Recognition) and more. Also I will recomend to take a look at Spacy, which is written in python, easy to use and much faster than CoreNLP.

Cottonade

+1 on the Spacy suggestion. Currently its probably THE NLP library for python. It's available in many languages, but anyone using it should be careful regardings to weak models (portuguese, for example, has very poor POS tagging). – Egidio

Answer 3

It appears that you are new to NLP.

I have recently started to use NLTK toolkit

If indeed you are new to NLP, then the best thing would be to start simple. So ideally you would start off with nltk. I am relatively new to natural language processing (a few months old). I can confirm that for beginners, nltk is better, since it has a great and free online book which helps the beginner learn quickly.

Once you are comfortable and actually have a problem to solve, look at Stanford Core NLP to see if it will be better at solving your problem.

If you want to stick to NLTK, you can also access the Stanford CoreNLP API in NLTK.

Now for the similarities and differences:

Can anyone tell me what is the difference between NLTK and Stanford NLP ? Are they 2 different libraries?

Both offer natural language processing. Some of the most useful parts of Stanford Core NLP include the part-of-speech tagger, the named entity recognizer, sentiment analysis, and pattern learning.

The named entity recognizer is better in the Stanford Core NLP. Stanford Core NLP is better at grammatical functions for instance picking up subject, object, predictae (that is partially why I switched from nltk to Stanford Core NLP). As @user812786 said, NLTK has multiple interfaces to other versions of NLP tools. NLTK is also better for learning NLP. If you need to use multiple corpora, use NLTK, as you can easily access a wide multitude of text corpora and lexical resources. Both have POS tagging and sentiment analysis.

Can stanford NLP be used using Python ?

Yes absolutely. You can use StanfordNLP which is a Python natural language analysis package that is able to call the CoreNLP Java package. There are also multiple Python packages using the Stanford CoreNLP server

Vallievalliere

Answer 4

I would add to this answer that if you are looking to parse date/time events StanfordCoreNLP contains SuTime which is the best datetime parser available. The support for arbitrary texts like 'Next Monday afternoon' is not present in any other package.

Refinery

Answer 5

NLTK can be used for the learning phase to and perform natural language process from scratch and basic level. Standford NLP gives you high-level flexibility to done task very fast and easiest way.

If you want fast and production use, can go for Standford NLP.

Interpretative

Answer 6

In 2020, Stanford released STANZA, Python library based on Stanford NLP. You can find it here https://stanfordnlp.github.io/stanza/

If you familiar with Spacy NLP, it quite similar :

>>> import stanza
>>> stanza.download('en') # download English model
>>> nlp = stanza.Pipeline('en') # initialize English neural pipeline
>>> doc = nlp("Barack Obama was born in Hawaii.") # run annotation over a sentence

Minelayer

Hmm any chance you would know what is the difference between the new stanza and the original stanfordnlp libraries? – Mouton

actually i can't answer that precisely since I never use java stanfordnlp in my project because I use Python. But in their website stanfordnlp.github.io/stanza/corenlp_client.html they claim that Stanza actually access their native java toolkit library via server in background process. So, my assumption is it just the same, only different programming language interface, CMIIW :) – Minelayer

Thanks because I'm currently using the Python stanfornlp interface and not stanza hence I wasn't sure what the difference is – Mouton

Stanza is a new version of stanfordnlp – it adds NER and sentiment analysis and increases the range of languages – but it's the same basic package. We renamed it because we decided that calling it "stanfordnlp" was too confusing and a bad idea (confusing our group name and one piece of software). – Nostril

Answer 7

Those 2 are different libraries.

They are written in different languages like Standford CoreNLP is written in Java and NLTK is written in python, you can check the documentation in the main website, in my point of view NLTK is much more useful to be used for tokenizing and Data PRE-PROCESSING.

Aston

Answer 8

Scope and Purpose:

NLTK: NLTK is a comprehensive library for natural language processing in Python. It provides tools for working with human language data (text). NLTK covers a wide range of tasks, including tokenization, stemming, tagging, parsing, and more. It is designed for educational purposes and research in NLP. Stanford NLP: Stanford NLP refers to a suite of natural language processing tools developed by the Stanford Natural Language Processing Group. It includes various tools and models for tasks such as part-of-speech tagging, named entity recognition, sentiment analysis, and more. Language Support:

NLTK: NLTK primarily focuses on the English language, although it does provide some support for other languages. Stanford NLP: Stanford NLP tools support multiple languages, making them more versatile for multilingual applications. Ease of Use:

NLTK: NLTK is known for being user-friendly and is often used in educational settings. It's suitable for beginners and researchers. Stanford NLP: While powerful, the Stanford NLP tools may have a steeper learning curve compared to NLTK. They are often chosen for more advanced or production-level NLP tasks. Models and Algorithms:

NLTK: NLTK provides various algorithms and models for NLP tasks. Users can choose different approaches based on their specific needs. Stanford NLP: Stanford NLP includes pre-trained models for various NLP tasks. These models are based on machine learning algorithms and are often considered state-of-the-art for certain tasks. Dependencies:

NLTK: NLTK is a standalone library that can be easily integrated into Python projects. Stanford NLP: Stanford NLP tools may have dependencies on Java libraries, and setting them up might involve additional steps compared to NLTK.

Pliocene