Good dataset for sentiment analysis? [closed]

Question

I am working on sentiment analysis and I am using dataset given in this link: http://www.cs.jhu.edu/~mdredze/datasets/sentiment/index2.html and I have divided my dataset into 50:50 ratio. 50% are used as test samples and 50% are used as train samples and the features extracted from train samples and perform classification using Weka classifier, but my predication accuracy is about 70-75%.

Can anybody suggest some other datasets which can help me to increase the result - I have used unigram, bigram and POStags as my features.

Prim

You may try this kaggle.com/data/36745#latest-205286 – Noyade

Answer 1

There are many sources to get sentiment analysis dataset:

• huge ngrams dataset from google storage.googleapis.com/books/ngrams/books/datasetsv2.html
• http://www.sananalytics.com/lab/twitter-sentiment/
• http://inclass.kaggle.com/c/si650winter11/data
• http://nlp.stanford.edu/sentiment/treebank.html
• or you can look into this global ML dataset repository: https://archive.ics.uci.edu/ml

Anyway, it does not mean it will help you to get a better accuracy for your current dataset because the corpus might be very different from your dataset. Apart from reducing the testing percentage vs training, you could: test other classifiers or fine tune all hyperparameters using semi-automated wrapper like CVParameterSelection or GridSearch, or even auto-weka if it fits.

It is quite rare to use 50/50, 80/20 is quite a commonly occurring ratio. A better practice is to use: 60% for training, 20% for cross validation, 20% for testing.

Latour

as you said if i reduce the train %.it will affect the learning process.that means learning from lesser samples will be hard.also if i increase the train % it will cause overfitting...thats why i took 50:5) ratio.. – Prim

It is quite rare to use 50/50, 80/20 is quite a commonly occurring ratio. A better practice is to use: 60% for training, 20% for cross validation, 20% for testing. PS: I just remember of this huge ngram dataset from google storage.googleapis.com/books/ngrams/books/datasetsv2.html – Latour

The following contains more than 1,578,627 classified datasets thinknook.com/wp-content/uploads/2012/09/… or ai.stanford.edu/~amaas/data/sentiment – Achaean

kaggle.com/bittlingmayer/amazonreviews – Samphire

Answer 2

I started to gather sentiment analysis tools/datasets/lexicons in one place, it could be useful for you too: https://github.com/laugustyniak/awesome-sentiment-analysis

Start PR if you want to add something more or just write to me. I worked a lot with Amazon data [millions of reviews].

Heber

Answer 3

Here is a list of datasets that give the sentiments for individual words.. http://positivewordsresearch.com/sentiment-analysis-resources/

Musician

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Hanshansard

I will try to put the links here if I get a chance – Musician