Full Text Searching with Rails

Question

I've been looking into searching plugins/gems for Rails. Most of the articles compare Ferret (Lucene) to Ultrasphinx or possibly Thinking Sphinx, but none that talk about SearchLogic. Does anyone have any clues as to how that one compares? What do you use, and how does it perform?

Ruphina

Answer 1

• thinking_sphinx and sphinx work beautifully, no indexing, query, install problems ever (5 or 6 install, including production slicehost )

• why doesn't everybody use sphinx, like, say craigslist? read here about its limitations (year and a half old articles. The sphinx developer, Aksyonoff, is working on these and he's putting in features and reliability and stamping out bugs at an amazing pace)

http://codemonkey.ravelry.com/2008/01/09/sphinx-for-search/

http://www.ibm.com/developerworks/opensource/library/os-php-apachesolr/

Comparison of full text search engine - Lucene, Sphinx, Postgresql, MySQL?

• ferret: easy install, doesn't stem properly, very slow indexing (one mysql db: sphinx: 3 seconds, ferret: 50 minutes). Well documented problems (index corruption) in drb servers in production under load. Having said that, i have use it in develometn since acts-as_ferret came out 3 years ago, and it has served me well. Not adhering to porter stemming is an advantage in some contexts.

• Lucene and Solr is the gorilla/mack truck / heavyweight champ of open source search. The teams have been doing an impressive number of new features in solr 14 release:

• acts-as-solr: works well, once the tomcat or jetty is in place, but those sometimes are a pain. The A-A-S fork by mattmatt is the main fork, but the project is relatively unmaintained.

• re the tomcat install: SOLR/lucene has unquestionably the best knowledge base/ support search engine of any software package i've seen ( i guess i'm not that surprised), the search box here:

http://www.lucidimagination.com/

• Sunspot the new ruby wrapper, build on solr-ruby. Looks promising, but I couldn't get it to install on OSX. Indexes all ruby objects, not just databases through AR

• one thing that's really instructive is to install 2 search plugins, e.g. sphinx and SOLR, sphinx and ferret, and see what different results they return. It's as easy as @sphinx_results - @ferret_results

---

just saw this post and responses

http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/

http://www.jroller.com/otis/entry/open_source_search_engine_benchmark

http://www.flax.co.uk/blog/2009/07/07/xapian-compared/

Plagiarize

Answer 2

First off, my obvious bias: I created and maintain Thinking Sphinx.

As it so happens, I actually saw Ben Johnson (creator of SearchLogic) present at the NYC ruby meet about it last night. SearchLogic is SQL-only - so if you're not dealing with massive tables, and relevance rankings aren't needed, then it could be exactly what you're looking for. The syntax is pretty clean, too.

However, if you want all the query intelligence handled by code that is not your own, then Sphinx or Solr (which is Lucene under the hood, I think) is probably going to work out better.

Pedlar

Answer 3

SearchLogic is a good plugin, but is really meant to make your search code more readable, it doesn't provide the automatic indexing that Sphinx does. I haven't used Ferret, but Sphinx is incredibly powerful.

http://railscasts.com/episodes/120-thinking-sphinx

Great introduction to see how flexible it is.

Cordierite

Answer 4

I have not used SearchLogic but I can tell you that Lucene is a very mature project, that has implementation in many languages. It is fast and flexible and the API is fun to work with. It's a good bet.

Proprioceptor

Answer 5

Given this question is still highly ranked at google for full text search, I'd really like to say that Sunspot is even stronger today if you're interested in adding full text search capabilities to your Rails application (and would like to have Solr behind you for that). You can check a full tutorial on this here.

And while we're at it, another contender that has arrived in the field is ElasticSearch, that aims to be a real time full text search engine built on top of Lucene (but doing things differently when compared to Solr). ElasticSearch includes out-of-the-box sharding and replication to multiple nodes, faster real time search, "percolators" to allow you to receive notifications when something that matches your criteria becomes available and it's moving really fast with many more other features. It's easy to build something on top of it, since the API is dead simple and completely based on REST using JSON as a format. One could say you don't even need a plugin to use it.

Murray

Answer 6

Personally, I don't bother with database agnostics for web applications and am quite happy using the full text search in pg83. The benefit is, if and when you change your framework/language, that you will still have full text search.

Brouwer

Answer 7

Full Text Indexing and MATCH() AGAINST().

If you're just looking to do a fast search against a few text columns in your table, you can simply use a full text index of those columns and use MATCH() AGAINST() in your queries.

1. Create the full text index in a migration file:

   add_index :table, :column, type: :fulltext
   

2. Query using that index:

   where( "MATCH( column ) AGAINST( ? )", term )
   

ElasticSearch and Searchkick

If you're looking for a full blown search indexing solution that allows you to search for any column in any of your records while still being lightning quick, take a look at ElasticSearch and Searchkick.

ElasticSearch is the indexing and search engine.

Searchkick is the integration library with Rails that makes it very easy to index your records and search them.

Searchkick's README does a fantastic job at explaining how to get up and running and to fine tune your setup, but here is a little snippet:

1. Install and start ElasticSearch.

   brew install elasticsearch
   brew services start elasticsearch
   

2. Add searchkick gem to your bundle:

   bundle add searchkick --strict
   

   The --strict option just tells Bundler to use an exact version in your Gemfile, which I highly recommend.

3. Add searchkick to a model you want to index:

   class MyModel < ApplicationRecord
     searchkick
   end
   

4. Index your records.

   MyModel.reindex
   

5. Search your index.

   matching_records = MyModel.search( "term" )
   

Radiator

Good solution. However, if you look to search into several columns you should add both column name into your migration. > add_index :table, [:column1, :column2], type: :fulltext > where( "MATCH( column1, column2 ) AGAINST( ? )", term ) – Cheeky

Answer 8

For anyone looking for a simple search gem without any dependencies, check out acts_as_indexed

Thionic