rails 7 zietwerk constants not found

Question

I'm upgrading our app from rails 6 to 7, I was using zeitwerk instead of the classic autoloader with rails 6 without any issues, now without any changes to the folder/file naming convention or structure it fails to find any classes/modules inside lib folder. I've been debugging the zeitwerk gem and the autoloads hash has file paths as keys and [Namespace, Constant Name] as the value, with rails 6 it had correct name space, e.g. for a file in \user\project\rails6_test\lib\folder\config.rb it'd have Folder as the namespace and Config defined inside config.rb would be found correctly in Folder namespace but after the migration with rails 7 it always has Object as the namespace, not just for top level folders but for all nested files and obviously it can't find those constants in the Object.

I don't see anything that I missed from the upgrade guide, what could be missing here ?

Update

I tried running bin/rails r 'pp ActiveSupport::Dependencies.autoload_paths' and I get the error below,

/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader/callbacks.rb:25:in `on_file_autoloaded': expected file /home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/restful_resource.rb to define constant RestfulResource, but didn't (Zeitwerk::NameError)

  raise Zeitwerk::NameError.new("expected file #{file} to define constant #{cpath}, but didn't", cref.last)
  ^^^^^
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/kernel.rb:28:in `require'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader/helpers.rb:95:in `const_get'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader/helpers.rb:95:in `cget'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader.rb:238:in `block (2 levels) in eager_load'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader/helpers.rb:26:in `block in ls'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader/helpers.rb:18:in `each_child'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader/helpers.rb:18:in `ls'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader.rb:233:in `block in eager_load'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader.rb:218:in `synchronize'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader.rb:218:in `eager_load'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader.rb:318:in `each'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/loader.rb:318:in `eager_load_all'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/application/finisher.rb:78:in `block in <module:Finisher>'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/initializable.rb:32:in `instance_exec'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/initializable.rb:32:in `run'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/initializable.rb:61:in `block in run_initializers'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:228:in `block in tsort_each'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:431:in `each_strongly_connected_component_from'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:349:in `block in each_strongly_connected_component'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:347:in `each'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:347:in `call'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:347:in `each_strongly_connected_component'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:226:in `tsort_each'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/3.1.0/tsort.rb:205:in `tsort_each'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/initializable.rb:60:in `run_initializers'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/application.rb:372:in `initialize!'
    from /home/leonidas/projects/ruby_on_rails/rainman/config/environment.rb:5:in `<top (required)>'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/kernel.rb:35:in `require'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/zeitwerk-2.5.4/lib/zeitwerk/kernel.rb:35:in `require'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/application.rb:348:in `require_environment!'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/command/actions.rb:28:in `require_environment!'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/command/actions.rb:15:in `require_application_and_environment!'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/commands/runner/runner_command.rb:33:in `perform'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/command/base.rb:87:in `perform'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/command.rb:48:in `invoke'
    from /home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/railties-7.0.2.2/lib/rails/commands.rb:18:in `<top (required)>'
    from bin/rails:4:in `require'
    from bin/rails:4:in `<main>'

I have config.autoload_paths << "#{config.root}/lib" in my application.rb and to test eager loading with zeitwerk I add config.eager_load_paths += Dir["#{config.root}/lib/**/"], let me know if you need any more info.

Update 2

After getting rid of the wildcard with just /lib running bin/rails r 'pp ActiveSupport::Dependencies.eager_load_paths' gives me the following output

["/home/leonidas/projects/ruby_on_rails/project_name/lib",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/microservices/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/microservices/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/dir/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/dir/dir/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/dir/dir/dir/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/dir/",
 "/home/leonidas/projects/ruby_on_rails/project_name/lib/tasks/",
 "/home/leonidas/projects/ruby_on_rails/project_name/app/controllers",
 "/home/leonidas/projects/ruby_on_rails/project_name/app/mailers",
 "/home/leonidas/projects/ruby_on_rails/project_name/app/models",
 "/home/leonidas/projects/ruby_on_rails/project_name/app/serializers",
 "/home/leonidas/projects/ruby_on_rails/project_name/app/workers",
 "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actionmailbox-7.0.2.2/app/controllers",
 "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actionmailbox-7.0.2.2/app/jobs",
 "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actionmailbox-7.0.2.2/app/models",
 "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/controllers",
 "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/controllers/concerns",
 "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/jobs",
 "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/models",
 "/home/leonidas/projects/ruby_on_rails/project_name/spec/mailers/previews"]

and running bin/rails r 'pp ActiveSupport::Dependencies._eager_load_paths' outputs the following

{"/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/controllers",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/controllers/concerns",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/jobs",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/activestorage-7.0.2.2/app/models",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actionmailbox-7.0.2.2/app/controllers",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actionmailbox-7.0.2.2/app/jobs",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actionmailbox-7.0.2.2/app/models",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actiontext-7.0.2.2/app/helpers",
  "/home/leonidas/.rbenv/versions/3.1.1/lib/ruby/gems/3.1.0/gems/actiontext-7.0.2.2/app/models",
  "/home/leonidas/projects/ruby_on_rails/project_name/app/controllers",
  "/home/leonidas/projects/ruby_on_rails/project_name/app/mailers",
  "/home/leonidas/projects/ruby_on_rails/project_name/app/models",
  "/home/leonidas/projects/ruby_on_rails/project_name/app/serializers",
  "/home/leonidas/projects/ruby_on_rails/project_name/app/workers",
  "/home/leonidas/projects/ruby_on_rails/project_name/lib"}

I was looking into the zeitwerk.rake task in rails and it seems it doesn't check any existing and non empty directories that exist in autoload paths but not in the eager load paths, I don't understand what should be happening here, is autoload_paths not supposed to have the expanded folder structure and only list root folders ? because in my case autoload having nested dirs too and eager_load only having root dirs is printing that warning about not checking the folders inside lib.

Makeyevka

Does /home/leonidas/projects/ruby_on_rails/project_name/lib/project_name/restful_resource.rb define RestfulResource? – Choroid

yeah, it defines module ProjectName; class RestfulResource; end; end – Makeyevka

Answer 1

Thanks for the updates. The problem is this glob:

config.eager_load_paths += Dir["#{config.root}/lib/**/"]

Adding to the eager load paths, adds also to the autoload paths (I find this a bit confusing, but it is how it works). Therefore, that glob is adding, for example, lib/project_name to the autoload paths. The key observation to understand what is happening is that when there are nested root paths, the most nested one wins as far as the files and directories below that one is concerned.

So, since lib is in the autoload paths, anything below lib is assumed to be defined in Object. But, since lib/project_name is in the autoload paths too, that subtree also represents Object. It is the same situation like the one you have in app/models and app/models/concerns.

These docs explain it, but they were easy to overlook in this case because it was not obvious your configuration is affected by that point.

The solution is to add only lib:

config.autoload_paths << "#{config.root}/lib"
config.eager_load_paths << "#{config.root}/lib"

Just like that, without wildcards. Technically, the first line is redundant given the second one, but in my view this is subtle and not obvious, I personally like to make both explicit, but as you like here. (There won't be duplicated entries, the final collection passes through uniq.)

Since we are on it, let me also comment something extra. Maybe your application already does this, but just in case, let me say that lib has often files and directories that are not meant to be autoloaded. For example, lib/tasks. It is clean to be deliberate about them and tell the autoloader:

Rails.autoloaders.main.ignore("#{config.root}/lib/tasks")

Otherwise, the autoloader will believe lib/tasks is a namespace, will define an autoload for Tasks in Object, will eager load (== actually define) the module if eager loading is enabled, etc. Conceptually, lib/tasks does not have code to be autoloaded/eager loaded, and the configuration has to reflect that. Same with other similar directories.

Jessabell

Hi @XavierNoria, yeah I started with config.eager_load_paths << "#{config.root}/lib" first because that's how I tested and set it up with rails 6 but now with rails 7 having that in application.rb and running the check gives me a warning saying the directories inside lib/project_name will only be checked if I configure them to be eager loaded and it lists all the nested directories inside the lib folder – Makeyevka

That is strange. I just checked and cannot reproduce. That would happen if the configuration pushes lib to the autoload paths and not to the eager load paths. And it would say so only for lib, not for nested directories. If the app does what my answer above says, zeitwerk:check reports a missing directory in the eager load paths? – Jessabell

I've put some more info with the difference in autoload and eager load paths after doing some debugging in the zeitwerk rake task as Update 2, can you see something wrong there ? – Makeyevka

Definitely, the autoload paths are suspicious. All those inner directories shouldn't be there, just lib. I cannot imagine a way of having them there other than a manual configuration that we are overlooking in the application (assuming no weird external things like side-effects of Spring or Docker containers). Could you do a search for "*" within config? – Jessabell

found it, turns out I had accidentally left the auto load paths with wildcards in development.rb, which I had put in while trying to figure out the initial issue of everything inheriting from Object, anyway thanks for all you help in debugging the issue, +1 to Zeitwerk. – Makeyevka

Awesome, we found it 🎉. – Jessabell

Answer 2

I had a similar error with Zeitwerk and the solution was to revise my class name, as I had something like "SomeImportantClassNameWithID"

And the mistake was that I had "ID" at the end of it, but it should have been "Id" - to make a proper CamelCase that matches the snake_case name of the file, to be loaded correctly by Zeitwerk.

i.e. SomeImportantClassNameWithID was wrong and SomeImportantClassNameWithId fixed it

Conclusion: Zeitwerk is case sensitive so double check your class names.

Langham