Bag of Words (BOW) vs N-gram (sklearn CountVectorizer) - text documents classification

Question

As far as I know, in Bag Of Words method, features are a set of words and their frequency counts in a document. In another hand, N-grams, for example unigrams does exactly the same, but it does not take into consideration the frequency of occurance of a word.

I want to use sklearn and CountVectorizer to implement both BOW and n-gram methods.

For BOW my code looks like this:

CountVectorizer(ngram_range=(1, 1), max_features=3000)

Is is enought to set 'binary' parameter to True to perform n-gram feature selection?

CountVectorizer(ngram_range=(1, 1), max_features=3000, binary=True)

What are the advantages of n-gram over the BOW method?

Lesko

"N-grams ... does not take into consideration the frequency of occurrence of a word" No, it takes into account the frequency of occurrence of the n-grams. And no, the 'binary' parameter has nothing to do with n-grams. If you want to use n-grams, you need to provide an n-gram order where like (1,2) (which includes "one"-grams and bigrams) or (2,2) which would include only bigrams. – Chlodwig

Correctymy if I am wrong, but if I will do that it will count the occurance of words/phrases (unigrams or bigrams, depending on configuration). This is a BOW method. Shouldn't n-gram take care about words occurance frequency? – Lesko

What? Look, consider n-gram models as a type of BOW. You generally take the frequency of the ngrams. You don't have to, which is what the binary parameter is for. You can explore different approaches and their results, but typically, counts are used, not just a binary in the doc or not. – Chlodwig

Ahh, ok. I thought that n-gram and BOW are completely different methods... Now everything became understable. Thanks! – Lesko

And really, you don't generally use the raw counts, but some sort of weighting factor like tf–idf. – Chlodwig

I am using tf-idf as well. My task is to compare different approaches. – Lesko

Answer 1

As answered by @daniel-kurniadi you need to adapt the values of the ngram_range parameter to use the n-gram. For instance by using (1, 2), the vectorizer will take into account unigrams and bigrams.

The main advantages of ngrams over BOW i to take into account the sequence of words. For instance, in the sentences:

1. "I love vanilla but I hate chocolate"
2. "I love chocolate but I hate vanilla"

The meaning is clearly different but a basic BOW representation will be the same in both cases. With n-grams (with n>=2), it will capture the order of the terms and thus the representations will be different.

Obediah

Such a clever example! Impossible to forget. – Forecast

Answer 2

If you set the ngram_range params to (m, n), then it will become an N-gram implementation.

Mailable