NoMethodError undefined method `fields' for nil:NilClass

Question

Using: Rails 3.1.1

I am trying to create a search engine in my application that browses a large database (apprx 100 000 items) for string matches.

I am using the following:

fp = FeededProduct.where("name LIKE '%blue%'  OR description LIKE '%blue%'  OR category LIKE '%blue%'")

for the search query for "blue".

When I run the equivalent search phrase in MySQL it works fine but when I try to run this in rails it shows:

NoMethodError: undefined method `fields' for nil:NilClass: SELECT `feeded_products`.* FROM `feeded_products`  WHERE (name LIKE '%blue%'  OR description LIKE '%blue%'  OR category LIKE '%blue%')

Clues and troubleshooting:

This happens only for large search results, I have not been able to distinguish a number but it crashes when it should have returned 920 results but it does NOT crash when returning 6 results!

My conclusion of the above is either that it cannot keep all the 920 results in the memory OR that there is some type of row that makes it crash and the more results, the more likely it is that it will contain that type of row. I am leaning more towards the first conclusion.

I cannot troubleshoot it very well because it crashes (with the same error code) when I try to run:

raise fp.inspect

It also crashes for:

fp.each do |prod| 
begin 
puts 'Do nothing'
rescue
puts 'Something crashed'
end

but it does NOT crash for:

raise fp.count.inspect 

So, am I having a memory type of problem? What can I do to trouble shoot this further and/or solve the problem?

Using: Mac OS X 10.7.2. Lion
Database: InnoDB
Adapter: Mysql2 (don't know which version)

Stack:

ActiveRecord::StatementInvalid (NoMethodError: undefined method fields' for nil:NilClass: SELECT feeded_products`.* FROM feeded_products WHERE (name LIKE '%blue%' OR description LIKE '%blue%' OR category LIKE '%blue%')): app/controllers/search_results_controller.rb:190:in `show'

Edit 2012-03-06 Additional trouble shooting:

• I tried with

  fp2 = FeededProduct.limit(60000)

to create a really big array of hits and it worked fine. So I guess that rules out my guess that the fp variable cannot hold a certain amount of items.

The core of the problem seems to be that if I use the:

fp = FeededProduct.where("name LIKE '%blue%'  OR description LIKE '%blue%'  OR category LIKE '%blue%'")

I cannot use the fp-variable for anything afterwards without the application crashing.

Phiona

Are you in development mode? What does the stack trace tell you? – Castrate

Yes, I am in development mode. Actually, I don't really know what the stack trace is (googling it right now). If you refer to the webserver terminal output it says: ActiveRecord::StatementInvalid (NoMethodError: undefined method fields' for nil:NilClass: SELECT feeded_products.* FROM feeded_products` WHERE (name LIKE '%blue%' OR description LIKE '%blue%' OR category LIKE '%blue%')): app/controllers/search_results_controller.rb:190:in `show' – Phiona

...it should perhaps be mentioned that this is my 2nd large Rails project and that I am still learning rails. – Phiona

which version of mysql gem are you using? which operating system are you using? and which storage engine are you using for large database search? like MyISAM or InnoDB?? Think for mysql2 (note the "2") gem - (github.com/brianmario/mysql2) – Iver

The stacktrace in the log should be many many lines long. The line you gave "StatementInvalid...." is just the first line of the trace. – Castrate

I tried to gather the info that might be needed to help me troubleshoot the problem. My gut feeling is that there is a problem related to the database, adapter or memory. – Phiona

Christoffer, can you post the extra information that's been requested: the full stack trace and the relevant lines from your search_results_controller.rb – Finisterre

Related: github.com/rails/rails/issues/2943 – Slink

Answer 1

I changed to mysql instead of mysql2 adapter and it solved the problem. Thanks everyone for trying! I learned a lot trouble shooting your suggestions.

Phiona

Answer 2

Can you post the code at line 190 in search_results_controller.rb, and any others that possibly refers to the "fields" attribute in your show method? Also relevant parts of your model FeededProduct from app/models/feeded_product.rb

app/controllers/search_results_controller.rb:190:in `show'

It's not clear what fields refers to from the info you posted. It could be a typo, bad code or a migration that needs to be run.

Also note that

fp.each.do |prod|

is not syntactically correct. It should be:

fp.each do |prod|

Appose

edk750: Row #190 is just the respond_to do |format| format.html # show.html.erb format.json { render :json => @search_result } end – Phiona

Regarding the fp.each.do it was just a typo (I wrote it manually here on SO, it's ok in the code). – Phiona

The big thing is that I do not use any method, attribute or anything called "fields". I have not named anything "fields" and there is nothing in the db that is called fields. Basically, the string "fields" cannot be found anywhere in the code throughout the whole application. – Phiona

I think the standing point is that if I make a search that would make the "fp"-variable get some 5-10 results, it works fine. If I make a search that returns more hits, it will crash with the error message given. It seems to me that there is something wrong with the adapter, mysql or similar that cannot handle too many results. Is there a limit here? Does the fact that I am using "LIKE" affect anything? – Phiona

There's no inherent limitation in the stack that I've come across that resembles anything you're describing. I strongly suspect it's something specific to your code or some gem you're using. Given that a FeededProduct is derived from ActiveRecord::Base, and you're sending a .where to the class, you should receive a ActiveRecord::Relation object. This allows you to iterate over the results. The key here is "NoMethodError: undefined method fields' for nil:NilClass" who is calling fields on a nil? This must be in your code somewhere, either directly or indirectly. We need more info. – Appose

Thanks for the response. There is definetely no calling for "fields" in MY code but it could be something related to gems. I will check that possibility out, that's a good tip. – Phiona

Answer 3

What if you moved that fp logic into your model instead?

class Item < ActiveRecord::Base
  ...
  def self.feeded_products(query)
    self.where("name LIKE '%#{query}%'  OR description LIKE '%#{query}%'  OR category LIKE '%#{query}%'")
  end
  ...
end

...and then in your controller, you can just call:

Item.feeded_products(query)

Hope this helps.

Kleeman

Answer 4

For these situations in which you have large sets of objects you can consider the use of find_each and find_in_batches methods or some kind of pagination.

Anatol

I guess that would have solved the problem, if it would have been a memory problem. I guess it isn't after all...the solution didn't work unfortunately. Thanks anyway! – Phiona