How to test ElasticSearch in a Rails application (Rspec)

Question

I was wondering how you were testing the search in your application when using ElasticSearch and Tire.

• How do you setup a new ElasticSearch test instance? Is there a way to mock it?

• Any gems you know of that might help with that?

---

Some stuff I found helpful:

I found a great article answering pretty much all my questions :)

http://bitsandbit.es/post/11295134047/unit-testing-with-tire-and-elastic-search#disqus_thread

Plus, there is an answer from Karmi, Tire author.

This is useful as well: https://github.com/karmi/tire/wiki/Integration-Testing-Rails-Models-with-Tire

I can't believe I did not find these before asking...

Chanel

Answer 1

Prefixing your index-names for the current environment

You could set a different index-name for each environment (in your case: the test environment).

For example, you could create an initializer in

config/initializers/tire.rb

with the following line:

Tire::Model::Search.index_prefix "#{Rails.application.class.parent_name.downcase}_#{Rails.env.to_s.downcase}"

A conceivable approach for deleting the indexes

Assuming that you have models named Customer, Order and Product, put the following code somewhere at your test-startup/before-block/each-run-block.

# iterate over the model types
# there are also ways to fetch all model classes of the rails app automaticly, e.g.:
#   https://mcmap.net/q/116618/-is-there-a-way-to-get-a-collection-of-all-the-models-in-your-rails-app
[Customer, Order, Product].each do |klass|

  # make sure that the current model is using tire
  if klass.respond_to? :tire
    # delete the index for the current model
    klass.tire.index.delete

    # the mapping definition must get executed again. for that, we reload the model class.
    load File.expand_path("../../app/models/#{klass.name.downcase}.rb", __FILE__)

  end
end

Alternative

An alternative could be to set up a different ElasticSearch instance for testing on another port, let's say 1234. In your enviornment/test.rb you could then set

Tire::Configuration.url "http://localhost:1234"

And at a suitable location (e.g. your testing startup) you can then delete all indexes on the ElasticSearch testing-instance with:

Tire::Configuration.client.delete(Tire::Configuration.url)

Maybe you must still make sure that your Tire-Mapping definitions for you model classes are still getting called.

Payable

That's a good idea, I'll try that, thanks. Do you have a particular setup to clean the index for every test? – Chanel

Thanks for the update. I wonder how slow the tests will get... I guess I'll have to try it. – Chanel

Please let us know if this will be a huge performance problem (I think you could be right :-). – Payable

After I do any of those, I get no search results anymore. It would be nice to be able to delete the documents themselves instead of the whole indexes. That way, you wouldn't need to reload models or whatever. – Tasteless

BTW to run another ES instance, use something like elasticsearch -f -Des.http.port=9400-9500 -Des.cluster.name=testing. If you don't specify a cluster name, it will automatically join the original server's cluster and you'll still be writing to development-environment indexes. – Metalworking

Answer 2

I ran into a quirky bug when deleting my elasticsearch index via tire in my rspec suite. In my Rspec configuration, similar to the Bits and Bytes blog, I have an after_each call which cleans the database and wipes out the index.

I found I needed to call Tire's create_elasticsearch_index method which is responsible for reading the mapping in the ActiveRecord class to set up the appropriate analyzers, etc. The issue I was seeing was I had some :not_analyzed fields in my model which were actually getting analyzed (this broke how I wanted faceting to work).

Everything was fine on dev, but the test suite was failing as facets were being broken down by individual words and not the entire multi word string. It seems that the mapping configuration was not being created appropriately in rspec after the index was deleted. Adding the create_elasticsearch_index call fixed the problem:

config.after(:each) do
  DatabaseCleaner.clean
  Media.tire.index.delete
  Media.tire.create_elasticsearch_index
end

Media is my model class.

Woollen

Answer 3

I ran into similar issues and here's how I solved it. Bare in mind that my solution builds on top of @spaudanjo solution. Since I'm using spork, I add this inside the spec_helper.rb's Spork.each_run block, but you may add this into any other each/before block.

# Define random prefix to prevent indexes from clashing
Tire::Model::Search.index_prefix "#{Rails.application.class.parent_name.downcase}_#{Rails.env.to_s.downcase}_#{rand(1000000)}"

# In order to know what all of the models are, we need to load all of them
Dir["#{Rails.root}/app/models/**/*.rb"].each do |model|
  load model
end

# Refresh Elastic Search indexes
# NOTE: relies on all app/models/**/*.rb to be loaded
models = ActiveRecord::Base.subclasses.collect { |type| type.name }.sort
models.each do |klass|
  # make sure that the current model is using tire
  if klass.respond_to? :tire
    # delete the index for the current model
    klass.tire.index.delete

    # the mapping definition must get executed again. for that, we reload the model class.
    load File.expand_path("../../app/models/#{klass.name.downcase}.rb", __FILE__)
  end
end

It basically defines it's own unique prefix for every test case so that there are no in indexes. The other solutions all suffered from a problem where even after deleting the index, Elastic Search wouldn't refresh the indexes (even after running Model.index.refresh) which is why the randomized prefix is there.

It also loads every model and checks if it responds to tire so that we no longer need to maintain a list of all of the models that respond to tire both in spec_helper.rb and in other areas.

As this method doesn't "delete" the indexes after using it, you will have to manually delete it on a regular basis. Though I don't imagine this to be a huge issue, you can delete with the following command:

curl -XDELETE 'http://localhost:9200/YOURRAILSNAMEHERE_test_*/'

To find what YOURRAILSNAMEHERE is, run rails console and run Rails.application.class.parent_name.downcase. The output will be your project's name.

Vesiculate