Rails counter_cache for Model.count without any association, in order to make SELECT COUNT (*) faster

Question

I'm totaling the rows in one of my models using Model.count and am a bit concerned about performance, as eventually, this model will get very large, and, therefore, SELECT COUNT (*) very slow.

Is there a way to use counter_cache without the :belongs_to relationship? Or another performance-friendly way of counting the rows? I thought about making another model, just one where I store calculations like this but not sure that's the best way.

Corkwood

Answer 1

Even more trivial than making a Cache model is to just use Rails.cache.

Rails.cache.read("elephant_count") #=> nil
Rails.cache.write("elephant_count", 1) #=> true
Rails.cache.read("elephant_count") #=> 1

Rails uses a file store by default (tmp/cache).

Then you could just place a Rails.cache.write increment and decrement into your model's after_create and after_destroy hooks, and override Model.size with a call to Rails.cache.read.

You could initialize the cache whenever Rails first initializes by placing a file named something like initialize_cache.rb in config/initializers containing:

Rails.cache.write('elephant_count', 0) if Rails.cache.read('elephant_count').nil?

Junia

Answer 2

If you want to have a maintained counter at all, whether using counter_cache or doing it manually, Rails will maintain your counters using callbacks, which will increase/decrease the counter when a new descendant is created/destroyed.

I am not aware of a means to store a counter_cache without using the belongs_to relationship, because only the parent can store the count of the children.

Weighing Performance

If your table is going to get 'large', populate your test database with a large number of rows then start running some SQL queries using EXPLAIN to get the performance of your database queries. See if the performance hit in doing record creation/destruction with counter_cache is offset by how often you need to access these counters in the first place.

If the counter does not need to be 100% accurate at all times, you can instead update the caches periodically using a cron job or background worker.

In summary:

1. You should only use counter_cache if you need those counters enough to offset the slightly longer time taken to create/destroy a record.
2. Using counter_cache vs a manual alternative that uses callbacks is, as far as I am aware, unlikely to result in much of a detriment to performance.
3. If the cache does not need to be accurate, take advantage of that and perform the calculations less often.

Supporting

Answer 3

Take a look at http://guides.rubyonrails.org/caching_with_rails.html Specifically, you'll want to take a look at the section regarding cache stores. Using cache stores, you can store values into cache for arbitrary things.

For example, you could have a method called on the Model called get_count which would be filled initially by the count but incremented by 1 with an after_create callback. If it's not necessary to keep it up to date, you can update this every x minutes so that you're mostly accurate.

I personally use memcache as a store for things like this. Just make sure you keep the cache up to date according to your needs.

Carib

Answer 4

How about defining a CachedCount concern like this?

module CachedCount
  extend ActiveSupport::Concern

  included do
    after_create :increment_cached_count
    after_destroy :decrement_cached_count
  end

  class_methods do
    def count
      return cached_count if cached_count
      Rails.cache.write(cached_count_key, super)
      Rails.cache.read(cached_count_key) || super # fallback because in some Rails env. Rails.cache may not be available
    end

    def cached_count_key
      "#{model_name.collection}_count"
    end

    def cached_count
      Rails.cache.read(cached_count_key)
    end
  end

  def increment_cached_count
    return self.class.count unless self.class.cached_count
    Rails.cache.write(self.class.cached_count_key, self.class.cached_count + 1)
  end

  def decrement_cached_count
    return self.class.count unless self.class.cached_count
    Rails.cache.write(self.class.cached_count_key, self.class.cached_count - 1)
  end
end

Then you include it in your numerous model:

class MyNumerousModel
include CachedCount

# [...]

end

Now, every time you call MyNumerousModel.count you're actually calling the class method in the concern. And when you create or destroy one instance of MyNumerousModel, the after_create and after_destroy callbacks take care of updating MyNumerousModel.cached_count.

Selhorst