NLTK was unable to find stanford-postagger.jar! Set the CLASSPATH environment variable

Question

I am working on a project that requires me to tag tokens using nltk and python. So I wanted to use this. But came up with a few problems. I went through a lot of other already asked questions and other forums but I was still unable to get a soultion to this problem. The problem is when I try to execute the following:

---

from nltk.tag import StanfordPOSTagger st = StanfordPOSTagger('english-bidirectional-distsim.tagger')

---

I get the following:

---

    Traceback (most recent call last):

    `File "<pyshell#13>", line 1, in <module>
        st = StanfordPOSTagger('english-bidirectional-distsim.tagger')`

    `File "C:\Users\MY3\AppData\Local\Programs\Python\Python35-32\lib\site-packages\nltk-3.1-py3.5.egg\nltk\tag\stanford.py", line 131, in __init__
        super(StanfordPOSTagger, self).__init__(*args, **kwargs)`

    `File "C:\Users\MY3\AppData\Local\Programs\Python\Python35-32\lib\site-packages\nltk-3.1-py3.5.egg\nltk\tag\stanford.py", line 53, in __init__
        verbose=verbose)`

     `File "C:\Users\MY3\AppData\Local\Programs\Python\Python35-32\lib\site-packages\nltk-3.1-py3.5.egg\nltk\internals.py", line 652, in find_jar
        searchpath, url, verbose, is_regex))`

     `File "C:\Users\MY3\AppData\Local\Programs\Python\Python35-32\lib\site-packages\nltk-3.1-py3.5.egg\nltk\internals.py", line 647, in find_jar_iter
        raise LookupError('\n\n%s\n%s\n%s' % (div, msg, div))`

    LookupError: 

    ===========================================================================
      NLTK was unable to find stanford-postagger.jar! Set the CLASSPATH
      environment variable.

    ===========================================================================

I already set the CLASSPATH - C:\Users\MY3\Desktop\nltk\stanford\stanford-postagger.jar I tried it as C:\Users\MY3\Desktop\nltk\stanford as well..

STANFORD_MODELS - C:\Users\MY3\Desktop\nltk\stanford\models\

I tried doing this as well..in vain File "C:\Python27\lib\site-packages\nltk\tag\stanford.py", line 45, in __init__ env_vars=('STANFORD_MODELS',), verbose=verbose) but it doesn't solve the problem either. Please Help me in solving this issue.

I use Windows 8, python 3.5 and nltk 3.1

Makkah

Have you tried looking in the jar file and models directory for english-bidirectional-distsim.tagger? – Iodic

Please try solutions from https://mcmap.net/q/215937/-how-to-use-stanford-parser-in-nltk-using-python/… or https://mcmap.net/q/215937/-how-to-use-stanford-parser-in-nltk-using-python or gist.github.com/alvations/e1df0ba227e542955a8a – Publus

Also this: #3702146 – Publus

I smell windows: gist.github.com/alvations/0ed8641d7d2e1941b9f9 "Please please please understand the solution don't just copy and paste!!! We're not monkeys typing Shakespeare ;P" – Publus

Answer 1

Update

The original answer was written for Stanford POS Tagger Version 3.6.0, Date 2015-12-09

There is a new Version (3.7.0, released 2016-10-31). Here's the code for the newer version:

from nltk.tag import StanfordPOSTagger
from nltk import word_tokenize

# Add the jar and model via their path (instead of setting environment variables):
jar = 'your_path/stanford-postagger-full-2016-10-31/stanford-postagger.jar'
model = 'your_path/stanford-postagger-full-2016-10-31/models/english-left3words-distsim.tagger'

pos_tagger = StanfordPOSTagger(model, jar, encoding='utf8')

text = pos_tagger.tag(word_tokenize("What's the airspeed of an unladen swallow ?"))
print(text)

---

Original answer

I had the same problem (but using OS X and PyCharm), finally got it to work. Here's what I've pieced together from the StanfordPOSTagger Documentation and alvas' work on the issue (big thanks!):

from nltk.internals import find_jars_within_path
from nltk.tag import StanfordPOSTagger
from nltk import word_tokenize

# Alternatively to setting the CLASSPATH add the jar and model via their path:
jar = '/Users/nischi/PycharmProjects/stanford-postagger-full-2015-12-09/stanford-postagger.jar'
model = '/Users/nischi/PycharmProjects/stanford-postagger-full-2015-12-09/models/english-left3words-distsim.tagger'

pos_tagger = StanfordPOSTagger(model, jar)

# Add other jars from Stanford directory
stanford_dir = pos_tagger._stanford_jar.rpartition('/')[0]
stanford_jars = find_jars_within_path(stanford_dir)
pos_tagger._stanford_jar = ':'.join(stanford_jars)

text = pos_tagger.tag(word_tokenize("What's the airspeed of an unladen swallow ?"))
print(text)

Hope this helps.

Orthopteran

@rima in PyCharm you can set the environment variables via Run -> Edit Configurations... -> Environment: Environment Variables -> [...] -> [+] – Orthopteran

Good job understanding the solution =) – Publus

https://mcmap.net/q/215937/-how-to-use-stanford-parser-in-nltk-using-python/… – Rhodium

Answer 2

I use Jupyter Notebook with Pycharm. I tried the Run Configuration in Pycharm to set env variable, but not work. So I use os.environ to set it in the code:

import os

os.environ["CLASSPATH"] = "/yourPath/stanford-parser-full-2018-10-17:yourPath/stanford-postagger-full-2018-10-16:yourPath/stanford-ner-2018-10-16"
os.environ["STANFORD_MODELS"] = "yourPath/stanford-postagger-full-2018-10-16/models:yourPath/stanford-ner-2018-10-16/classifiers"

stanford_tagger = StanfordPOSTagger('english-bidirectional-distsim.tagger')

Hope it will help!

Saddletree