Apache solr search part of the word

Question

I'm using apache solr search engine for indexing my website database..

I'm using django+http://haystacksearch.org/

So let's say I have document that have word "Chicken"

When I search for "chicken" - solr can find this document

But When I search "chick" - it does not find anything..

Is there a way to fix this ?

Calli

Answer 1

Note: The following solution is Solr 1.4 (and above) specific!

For more flexibility, I would recommend indexing your data with the NGramTokenizerFactory to do complete front and back wildcard searches. If you just want to search for substrings at the beginning or end of the string, consider using the EdgeNGramTokenizerFactory.

Here's a drop in replacement of the text field type which would accomodate your need:

<fieldType name="text" class="solr.TextField" >
<analyzer type="index">
    <tokenizer class="solr.NGramTokenizerFactory" minGramSize="3" maxGramSize="15" />
    <filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
    <tokenizer class="solr.WhitespaceTokenizerFactory" />
    <filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>

Mocha

solr 1.5 - is this development version ? (not released ?) – Calli

is there solution like this for 1.4 ? – Calli

That's really useful... Personally i think i'd create a new fieldtype like the above.. but called ntext or something.. just so you don't mess with the original text fieldtype. – Homozygous

The NGramTokenizerFactory works great for me (even on 1.3 version) – Calli

Answer 2

If you want to find all words that start with chick, search for chick*.

Cookout

I am not able to use *chick*? is it possible? – Own

@soundar, That's a bad chick ;) You can't have * infront of the search keyword, Lucene does not support that. – Suppress

Answer 3

When I've used

<tokenizer class="solr.NGramTokenizerFactory" minGramSize="3" maxGramSize="15" />

for making wildcard search from Brian's answer, Solr indexing time dramaticly increased. In more than 20 times! The other decision of wildcard searching problem I found here:

http://www.lucidimagination.com/blog/2009/09/08/auto-suggest-from-popular-queries-using-edgengrams/

You need just add filter

<filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="25" />

(default tokenizer - solr.WhitespaceTokenizerFactory in index block of FieldType). For me result was the same with less system costs.

Sabo

Answer 4

A different approach, if you are having trouble with a small set of words, would be to use the solr.SynonymFilterFactory

http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory

You just have to maintain a simple text file that contains synonyms:

chick peep chicken
dawg hound dog
moggie puss kitten cat

Plurals should take care of themselves with other filters.

Mig

Answer 5

I haven't changed any configuration. I am just using star in front and in the back of my searchString: *chicke * (without white space at the end -> it's because of SO formatting word as italic if you use * at the beginning and at the end)

Gryphon