How to obtain better results using NLTK pos tag

Question

I am just learning nltk using Python. I tried doing pos_tag on various sentences. But the results obtained are not accurate. How can I improvise the results ?

broke = NN
flimsy = NN
crap = NN

Also I am getting lot of extra words being categorized as NN. How can I filter these out to get better results.?

Shifra

see #30821688 – Conjugated

Answer 1

Give the context, there you obtained these results. Just as example, I'm obtaining other results with pos_tag on the context phrase "They broke climsy crap":

import nltk
text=nltk.word_tokenize("They broke flimsy crap")
nltk.pos_tag(text)

[('They', 'PRP'), ('broke', 'VBP'), ('flimsy', 'JJ'), ('crap', 'NN')]

Anyway, if you see that in your opinion a lot of word are falsely cathegorized as 'NN', you can apply some other technique specially on those which are marked a s 'NN'. For instance, you can take some appropriate tagged corpora and classify it with trigram tagger. (actually in the same way the authors do it with bigrams on http://nltk.googlecode.com/svn/trunk/doc/book/ch05.html).

Something like this:

pos_tag_results=nltk.pos_tag(your_text) #tagged sentences with pos_tag
trigram_tagger=nltk.TrigramTagger(tagged_corpora) #build trigram tagger based on your tagged_corpora
trigram_tag_results=trigram_tagger(your_text) #tagged sentences with trigram tagger
for i in range(0,len(pos_tag_results)):
    if pos_tag_results[i][1]=='NN':
        pos_tag_results[i][1]=trigram_tag_results[i][1]#for 'NN' take trigram_tagger instead

Let me know if it improves your results.

Paulettapaulette