View raw Solr tokens for a single field on a single document

Question

I'm debugging my Solr schema and I'd like to see the results of tokenizing a specific field.

For a simplified example, if I have:

<fieldType name="text" class="solr.TextField" omitNorms="false">
    <analyzer>
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.StandardFilterFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.PorterStemFilterFactory"/>
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15" side="front"/>
    </analyzer>
</fieldType>

and I indexed a field with the value "Hello, worlds!", I want to see something along the lines of:

hello world he el ll lo hel ell llo hell ello hello wo or rl ld wor orl rld worl orld

to ensure that everything is being tokenized as I envisage it is.

Is this in any way possible?

Morril

Answer 1

Yes, Admin > Analysis is exactly what you want.

But there's another great tool that allows you to read index and see how exactly a field or document was indexed.

It's called Luke and it's invaluable when troubleshooting and tweaking your schema.

Release

Answer 2

yes, use the Analysis page in the Solr Admin section: here It has exactly that purpose

Ensnare