Login/Register
Solr with Rails - rake sunspot:reindex is not working
Asked Answered
Solved ruby-on-railssolrsunspot-solrengineyard
D

1

11

I am stuck into a strange issue where I am looking for your inputs.

My problem is:

After deploying my application on Production using Capistrano, when I am doing solr re-indexing, it is giving me below error:

$ bundle exec rake sunspot:reindex --trace
** Invoke sunspot:reindex (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute sunspot:reindex
Skipping progress bar: for progress reporting, add gem 'progress_bar' to your Gemfile
rake aborted!
RSolr::Error::Http - 404 Not Found
Error:     Not Found

Request Data: "<?xml version=\"1.0\" encoding=\"UTF-8\"?><delete>query>type:OccupationData</query></delete>"
Backtrace: /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:268:in `adapt_response'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:175:in `execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:161:in `send_and_receive'
(eval):2:in `post'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:67:in `update'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:131:in `delete_by_query'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/indexer.rb:55:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `block in remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:268:in `adapt_response'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:175:in `execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:161:in `send_and_receive'
(eval):2:in `post'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:67:in `update'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.9/lib/rsolr/client.rb:131:in `delete_by_query'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/indexer.rb:55:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `block in remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/retry_5xx_session_proxy.rb:17:in `method_missing'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot.rb:464:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/searchable.rb:178:in `solr_remove_all_from_index'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/searchable.rb:197:in `solr_reindex'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/tasks.rb:58:in `block (3 levels) in <top (required)>'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/class_set.rb:16:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/class_set.rb:16:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/tasks.rb:57:in `block (2 levels) in <top (required)>'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `call'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `block in execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:184:in `block in invoke_with_call_chain'
/usr/lib64/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:177:in `invoke_with_call_chain'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:170:in `invoke'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:143:in `invoke_task'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block (2 levels) in top_level'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block in top_level'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:110:in `run_with_threads'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:95:in `top_level'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:73:in `block in run'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:160:in `standard_exception_handling'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:70:in `run'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/bin/rake:33:in `<top (required)>'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bin/rake:19:in `load'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bin/rake:19:in `<main>'
Tasks: TOP => sunspot:reindex

To investigate it more, when I looked at sunspot production log, I saw:

INFO: Creating SolrCore 'collection1' using instanceDir: /data/APP_NAME/releases/20130325112326/solr/collection1
Apr 1, 2013 2:38:16 AM org.apache.solr.core.SolrResourceLoader <init>
INFO: new SolrResourceLoader for directory: '/data/APP_NAME/releases/20130325112326/solr/collection1/'
Apr 1, 2013 2:38:16 AM org.apache.solr.core.CoreContainer recordAndThrow
SEVERE: Unable to create core: collection1
org.apache.solr.common.SolrException: Could not load config for solrconfig.xml
    at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:991)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051)
    at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634)
    at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or '/data/APP_NAME/releases/20130325112326/solr/collection1/conf/', cwd=/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr
    at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:318)
    at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:283)
    at org.apache.solr.core.Config.<init>(Config.java:103)
    at org.apache.solr.core.Config.<init>(Config.java:73)
    at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:117)
    at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:989)
    ... 11 more
Apr 1, 2013 2:38:16 AM org.apache.solr.common.SolrException log
SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: collection1
    at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1672)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1057)
    at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:634)
    at org.apache.solr.core.CoreContainer$3.call(CoreContainer.java:629)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.common.SolrException: Could not load config for solrconfig.xml
    at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:991)
    at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1051)
    ... 10 more
Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or '/data/APP_NAME/releases/20130325112326/solr/collection1/conf/', cwd=/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr
    at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:318)
    at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:283)
    at org.apache.solr.core.Config.<init>(Config.java:103)
    at org.apache.solr.core.Config.<init>(Config.java:73)
    at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:117)
    at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:989)
    ... 11 more
Apr 1, 2013 2:38:16 AM org.apache.solr.servlet.SolrDispatchFilter init
INFO: user.dir=/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr
Apr 1, 2013 2:38:16 AM org.apache.solr.servlet.SolrDispatchFilter init
INFO: SolrDispatchFilter.init() done

From the logs, it is clear that re-indexing is failing because of 2 reasons:

A. Unable to create core: collection1
B. Can't find resource 'solrconfig.xml' in classpath

Based on the errors above, below are the tricks I tried:

A. Checked the folder permission for solr
B. Manually created collection1 inside solr and copied solrconfig.xml inside /solr/collection1/conf.
C. copied the lib and conf folders from /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr and pasted it in APP_NAME/solr/sunspot_rails folder.

But none of the solutions worked. :-(

Kindly suggest.

[UPDATED]: My Gemfile looks like this:

gem "sunspot", "~> 1.3.3"
gem 'sunspot_rails', '~> 1.3.0'
gem 'sunspot_cell', :git => 'git://github.com/zheileman/sunspot_cell.git'
gem 'sunspot_cell_jars', :git => 'https://github.com/mrcsparker/sunspot_cell_jars.git'
gem 'sunspot_solr', :git => "git://github.com/sunspot/sunspot.git"

[UPDATED]: As of 4/4/2013: I tried with the steps below, but still no luck: Please check the details, maybe this will help you to understand if I am heading towards the right direction.

1. Stopped solr <br />
${PATH_TO_APP}: bundle exec rake sunspot:solr:stop

2. remove solr folder
${PATH_TO_APP}: rm -rf solr

3. Start solr
${PATH_TO_APP}: bundle exec rake sunspot:solr:start
--backtrace
java version "1.6.0_30"
Java(TM) SE Runtime Environment (build 1.6.0_30-b12)
Java HotSpot(TM) 64-Bit Server VM (build 20.5-b03, mixed mode)
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/solrconfig.xml => /data/APP_NAME/releases/20130402102051/solr/conf/solrconfig.xml
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/spellings.txt => /data/APP_NAME/releases/20130402102051/solr/conf/spellings.txt
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/synonyms.txt => /data/APP_NAME/releases/20130402102051/solr/conf/synonyms.txt
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/elevate.xml => /data/APP_NAME/releases/20130402102051/solr/conf/elevate.xml
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/admin-extra.html => /data/APP_NAME/releases/20130402102051/solr/conf/admin-extra.html
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/schema.xml => /data/APP_NAME/releases/20130402102051/solr/conf/schema.xml
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/protwords.txt => /data/APP_NAME/releases/20130402102051/solr/conf/protwords.txt
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/scripts.conf => /data/APP_NAME/releases/20130402102051/solr/conf/scripts.conf
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/stopwords.txt => /data/APP_NAME/releases/20130402102051/solr/conf/stopwords.txt
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/conf/mapping-ISOLatin1Accent.txt => /data/APP_NAME/releases/20130402102051/solr/conf/mapping-ISOLatin1Accent.txt
Copying /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bundler/gems/sunspot-87e29212527f/sunspot_solr/solr/solr/solr.xml => /data/APP_NAME/releases/20130402102051/solr
Successfully started Solr ...

4. Ensure config file is correct (solr/conf/schema.xml)
I used this link for schema.xml: https://mcmap.net/q/1158470/-400-bad-request-unknown-field-39-type-39 and overwrite schema.xml file under {PATH_TO_APP}/solr/conf

5. Run Sunspot re-index
${PATH_TO_APP}: bundle exec rake sunspot:reindex

--backtrace
rake aborted!
RSolr::Error::Http - 404 Not Found
Error:     Not Found

Request Data: "<?xml version=\"1.0\" encoding=\"UTF-8\"?><delete><query>type:JobSeeker</query></delete>"

One thing I noticed is, when I executed bundle exec rake sunspot:solr:start command and checked the log files, I found this:

SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: development
Caused by: org.apache.solr.common.SolrException: Cannot create directory: /data/APP_NAME/current/solr/data/index

But when I visited this directory index folder was there.

Interesting to see that solr creates solr.xml file in Production mode, whereas it's not there in my staging environment. Please see solr.xml (after changing it):

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
<cores adminPath="/admin/cores" host="${host:}" hostPort="${jetty.port:}">
    <core name="default"     instanceDir="/data/APP_NAME/current/solr" dataDir="/data/APP_NAME/current/solr/data"/>
    <core name="development" instanceDir="/data/APP_NAME/current/solr" dataDir="/data/APP_NAME/current/solr/data"/>
    <core name="test"        instanceDir="/data/APP_NAME/current/solr" dataDir="/data/APP_NAME/current/solr/data"/>
</cores>
</solr>

Any idea about this?

[UPDATED]: As on 5th-April-2013 I deployed a fresh build on production instance and after that, when I do: 

$ RAILS_ENV=production bundle exec rake sunspot:reindex --trace

I got this:

** Invoke sunspot:reindex (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute sunspot:reindex
Skipping progress bar: for progress reporting, add gem 'progress_bar' to your Gemfile

rake aborted!
RSolr::Error::Http - 404 Not Found
Error:     Not Found

Request Data: "<?xml version=\"1.0\" encoding=\"UTF-8\"?><delete>query>type:OccupationData</query></delete>"
Backtrace: /data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:230:in `adapt_response'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:167:in `execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:161:in `send_and_receive'
(eval):2:in `post'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:67:in `update'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:131:in `delete_by_query'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/indexer.rb:55:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `block in remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:230:in `adapt_response'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:167:in `execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:161:in `send_and_receive'
(eval):2:in `post'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:67:in `update'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rsolr-1.0.8/lib/rsolr/client.rb:131:in `delete_by_query'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/indexer.rb:55:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `block in remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session.rb:181:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/retry_5xx_session_proxy.rb:17:in `method_missing'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot.rb:464:in `remove_all'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/searchable.rb:178:in `solr_remove_all_from_index'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/searchable.rb:197:in `solr_reindex'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/tasks.rb:58:in `block (3levels) in <top (required)>'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/class_set.rb:16:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot-1.3.3/lib/sunspot/class_set.rb:16:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/sunspot_rails-1.3.3/lib/sunspot/rails/tasks.rb:57:in `block (2levels) in <top (required)>'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `call'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:246:in `block in execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:241:in `execute'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:184:in `block in invoke_with_call_chain'
/usr/lib64/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:177:in `invoke_with_call_chain'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/task.rb:170:in `invoke'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:143:in `invoke_task'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block (2 levels) in top_level'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `each'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:101:in `block in top_level'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:110:in `run_with_threads'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:95:in `top_level'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:73:in `block in run'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:160:in `standard_exception_handling'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/lib/rake/application.rb:70:in `run'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/gems/rake-10.0.4/bin/rake:33:in `<top (required)>'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bin/rake:19:in `load'
/data/APP_NAME/shared/bundled_gems/ruby/1.9.1/bin/rake:19:in `<main>'
Tasks: TOP => sunspot:reindex

Below is my model:

class OccupationData < ActiveRecord::Base
    attr_accessible :onetsoc_code, :title, :description

    searchable :auto_index => false do
        text :title
        text :description
    end
end

Tried to run all the sunspot rake tasks by specifying RAILS_ENV=production and cross checked if SOLR is running with ROOT or deploy, and got this:

/data/APP_NAME/current $ ps -ef | grep solr
deploy    2342     1  0 Apr04 ?        00:00:30 /usr/lib/jvm/sun-jdk-1.6/bin/java -Djetty.port=8983 -Dsolr.data.dir=/data/APP_NAME/releases/20130404095808/solr/data/production -Dsolr.solr.home=/data/APP_NAME/releases/20130404095808/solr -Djava.util.logging.config.file=/tmp/logging.properties20130404-2342-17k2tsp -jar start.jar
deploy    9770  9461  0 06:42 pts/0    00:00:00 grep --colour=auto solr

It tells me that SOLR is running on deploy. I am not even in a situation to provide SOLR logs, because failing sunspot:reindex is not generating any log level information as well. :-(

9th-April-2013 - [UPDATE] [SOLUTION]

After struggling a lot with re-indexing, I finally found a solution of this. The error was occurring due to wrong gem mentioned in Gemfile.

Here's the old copy of my Gemfile:

gem "sunspot", "~> 1.3.3"
gem 'sunspot_rails', '~> 1.3.0'
gem 'sunspot_cell', :git => 'git://github.com/zheileman/sunspot_cell.git'
gem 'sunspot_cell_jars', :git => 'https://github.com/mrcsparker/sunspot_cell_jars.git'
gem 'sunspot_solr', :git => "git://github.com/sunspot/sunspot.git"

When I browsed this link: https://github.com/mrcsparker/sunspot_cell_jars, I came to know that I have to use the updated sunspot_solr gem in order to keep it work.

so I am using:

gem 'sunspot_solr', :git => "https://github.com/mrcsparker/sunspot.git"

and then I created JAR files again with "rails g command" and when ran sunspot:reindex everything worked. Hope this will help someone.

Demur answered 1/4, 2013 at 10:57 Comment(17)
Is regular indexing of objects working? – Gush
no it is not working as well. However, I can start-stop solr but when it comes to indexing, it breaks. – Demur
Have you set up a solr server, or are you trying to use the solr-rails bundled version? If it's the latter, it won't work in production mode. – Gush
Have updated the post and added the Gemfile. However, I am using solr-rails bundled version. But how it will not work in Production? I have a staging server as well, where solr is working fine. – Demur
I am still not sure how to proceed on this. But I am executing below steps: - Stopping the engineyard instance. - Delete the environment. - re-creating the environment again - deploying the application – Demur
I'd suggest deploying a standalone solr instance as recommended here: github.com/sunspot/sunspot/wiki/… (which links to wiki.apache.org/solr/SolrTomcat ) – Gush
Tried with creating a fresh instance and deploying the application there. But still facing the same issue. Will see today how deeper I can dig into this problem. – Demur
One thing I've noticed so far is the way solr creates folder and files are very different. Because I've a staging environment, when I look at the solr folder there and compare it with the one in Production, I saw differences. Production has solr.xml file inside solr/ folder whereas staging doesn't has. – Demur
I did some changes in solr.xml file as mentioned here: pastie.org/7299586. After doing that when I run reindex command, I get: "SEVERE: Unable to create core". I will double check the folder permissions now. Do you suggest something different? – Demur
https://mcmap.net/q/1158470/-400-bad-request-unknown-field-39-type-39/684934 might be relevant. – Gush
Thanks a lot @bdares for your helping hand, but this is really annoying that after executing all the steps and steps I found on other sites, I have still no luck with solr. I am going to give this a try today as well, but if you get a chance, please see the updated post: [UPDATED]: As of 4/4/2013 and provide your valuable inputs on the same. I really appreciate your help on this. – Demur
You shouldn't be starting solr with a rake command. It'll be sitting in its own Tomcat container if you set it up according to wiki.apache.org/solr/SolrTomcat – Gush
@bdares: I think the steps you're suggesting when using Solr with Tomcat. But here I am not using this approach. I am using sunspot-rails bundled version and jetty. Please see: [UPDATED]: As on 5th-April-2013 – Demur
You're not hearing me. Don't use the sunspot_solr gem packaged solr server for your production environment. It's not supported. – Gush
Thanks @bdares for letting me know and yes after reading the WIKI pages of sunspot, I came to know that re-index won't work on Production environment. But the only concern I am having is, if re-index is not working then start and stop should also not work. Secondly, apart from configuring Tomcat, don't I have other option to configure and run solr for Production environment? – Demur
Hello guys, Do you found any solutions on this issue. I have a same problem with Rails 4rc1 application? Here is the my issue on github: github.com/sunspot/sunspot/issues/403 – Thunderpeal
@Zeck: Double check sunspot_solr gem path. You must have to use: gem 'sunspot_solr', :git => "github.com/mrcsparker/sunspot.git" – Demur
C
7

Had the same problem slightly different on error logging. In my case it was like:

2013-07-18 10:26:47.750:INFO:oejw.StandardDescriptorProcessor:NO JSP Support for /solr, did not find org.apache.jasper.servlet.JspServlet
Null identity service, trying login service: null
Finding identity service: null

What I'm using is Solr 4, which has a better Admin interface, so into the Gemfile:

gem 'sunspot_solr', github: 'sunspot/sunspot', branch: 'master'
gem 'sunspot_rails', github: 'sunspot/sunspot', branch: 'master'

And that's it, no more. Just notice the github: part. I did something similar to what you did: deleted the solr directory, then,

bundle install

then, 

rails generate sunspot_rails:install

and then,

bundle exec rake sunspot:solr:start

and eventually it started to work.

Next, run

bundle exec rake sunspot:reindex

One thing I have to admit, is that before the above procedure, since I have Solr 4, I replaced my xml files with these from here. Not sure whether this is critical.

Hope this helps someone.

Cardigan answered 19/7, 2013 at 7:30 Comment(2)
I think running it with bundle exec rake instead of just rake did it for me – Sedate
Thank you! This worked for my issue of getting solr working with JRuby. – Brozak

© 2022 - 2024 — McMap. All rights reserved.