`FrozenError: can't modify frozen Array` in Rails 7 Engine when running `rspec`

Question

I just upgraded an Engine from Rails 5 to Rails 7. This error started appearing at Rails 6.1.7.6, but I thought perhaps it might've been fixed in Rails 7.

Here's the error I get when I run rspec

An error occurred while loading ./spec/awesome_engine/services/awesome_engine/pdf_exporter/termination_spec.rb.
Failure/Error: Rails.application.initialize!

FrozenError:
  can't modify frozen Array: ["/Users/bobbert/.gem/ruby/2.7.6/gems/actiontext-7.0.7.2/app/helpers", "/Users/bobbert/.gem/ruby/2.7.6/gems/actiontext-7.0.7.2/app/models"]
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/engine.rb:575:in `unshift'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/engine.rb:575:in `block in <class:Engine>'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:32:in `instance_exec'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:32:in `run'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:61:in `block in run_initializers'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:50:in `each'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:50:in `tsort_each_child'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:50:in `each'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:50:in `tsort_each_child'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/initializable.rb:60:in `run_initializers'
# /Users/bobbert/.gem/ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/application.rb:372:in `initialize!'
# ./spec/dummy/config/environment.rb:5:in `<top (required)>'
# /Users/bobbert/.gem/ruby/2.7.6/gems/zeitwerk-2.6.11/lib/zeitwerk/kernel.rb:38:in `require'
# /Users/bobbert/.gem/ruby/2.7.6/gems/zeitwerk-2.6.11/lib/zeitwerk/kernel.rb:38:in `require'
# ./spec/spec_helper.rb:5:in `<top (required)>'
# /Users/bobbert/.gem/ruby/2.7.6/gems/zeitwerk-2.6.11/lib/zeitwerk/kernel.rb:38:in `require'
# /Users/bobbert/.gem/ruby/2.7.6/gems/zeitwerk-2.6.11/lib/zeitwerk/kernel.rb:38:in `require'
# ./spec/awesome_engine/services/awesome_engine/pdf_exporter/termination_spec.rb:1:in `<top (required)>'
...
Finished in 0.00005 seconds (files took 10.79 seconds to load)
0 examples, 0 failures, 133 errors occurred outside of examples

It occurs multiple times when trying different specs, and it always stems from the first line in the spec, then spec_helper.rb:5, and finally environment.rb:5.

First line of every spec is:

require 'spec_helper'

spec_helper.rb, Line 5

require File.expand_path("../dummy/config/environment.rb",  __FILE__)

environment.rb, Line 5

Rails.application.initialize!

And this is the Rails code that's throwing the error (ruby/2.7.6/gems/railties-7.0.7.2/lib/rails/engine.rb:575):

573    initializer :set_autoload_paths, before: :bootstrap_hook do
574      ActiveSupport::Dependencies.autoload_paths.unshift(*_all_autoload_paths)
575      ActiveSupport::Dependencies.autoload_once_paths.unshift(*_all_autoload_once_paths)
576
577      config.autoload_paths.freeze
578      config.autoload_once_paths.freeze
579    end

I've been following various Rails guides on the changes to autoloading, including:

• https://guides.rubyonrails.org/upgrading_ruby_on_rails.html#autoloading-during-initialization
• https://rubyonrails.org/2021/9/3/autoloading-in-rails-7-get-ready
• https://guides.rubyonrails.org/v7.0/autoloading_and_reloading_constants.html#use-case-1-during-boot-load-reloadable-code

I've also tried the suggestions in other Stackoverflow questions:

• Rails : RuntimeError - can't modify frozen Array when running rspec in rails
• can't modify frozen array (TypeError) - config/application.rb:42:in `<<':

I've been at this now for several hours and am not progressing. Does anyone have any clue what's happening here and/or how to fix this issue?

---

UPDATE: 2023 AUGUST 28

So, I've decided to debug and step through the code.

There are 573 initializers in total. I placed a breakpoint on initializer :set_autoload_paths in engine.rb Line 574 to keep track of how many times this initializer is called. Here's what I found:

Rails::Application.initialize! L372
    Rails::Initializable::Initalizer.run_initializers L61
        initializer(name: :set_autoload_paths) #initializer 103 of 573
        initializer(name: :set_autoload_paths) #initializer 119 of 573
        initializer(name: :set_autoload_paths) #initializer 140 of 573
        initializer(name: :set_autoload_paths) #initializer 158 of 573
        initializer(name: :set_autoload_paths) #initializer 171 of 573
        ...

At this point, it's obvious that it's being called multiple times. So I decided analyze the initializers array to see just how many times it's being called and who is calling it. Here's what I found:

:set_autoload_paths=>{:count=>34, :contexts=>[#<ActionView::Railtie>, #<ActiveStorage::Engine>, #<ActionCable::Engine>, #<ActionMailbox::Engine>, #<ActionText::Engine>, #<StateMachine::RailsEngine>, #<Select2::Rails::Engine>, #<Doccex::Engine>, #<SmartListing::Engine>, #<Kaminari::Engine>, #<Devise::Engine>, #<DeviseInvitable::Engine>, #<Bootstrap::Rails::Engine>, #<Bootstrap::Switch::Rails::Engine>, #<Cocoon::Engine>, #<FontAwesome::Rails::Engine>, #<Remotipart::Rails::Engine>, #<I18n::JS::Engine>, #<Jquery::Rails::Engine>, #<Jquery::Ui::Rails::Engine>, #<JsRoutes::Engine>, #<DropzonejsRails::Engine>, #<TinyMCE::Rails::Engine>, #<BootstrapDatepickerRails::Rails::Engine>, #<Bootstrap3Datetimepicker::Rails::Engine>, #<Momentjs::Rails::Engine>, #<Uri::Js::Rails::Engine>, #<Sidekiq::Rails>, #<ActsAsTaggableOn::Engine>, #<Jscolor::Rails::Engine>, #<Tribute::Engine>, #<Doorkeeper::Engine>, #<AwesomeEngine::Engine>, #<Dummy::Application>]}

It's being called 34 times by various engines, several of which are from the Rails framework but the majority from 3rd party libraries that are included in the engine's gemspec.

Twinscrew

You linked to section 2.7, I assume you saw the notes in 2.8? - especially relating to engines? – Telpher

@Telpher You mean this? Similarly, engines can configure that collection in the class body of the engine class or in the configuration for environments.Are you suggesting that I need to add the initializers to the autoload_once_paths collection? – Twinscrew

Well that and the last paragraph in that section too. I'm not suggesting anything because I can't see your code, just pointing you to something to check in your own code (ie. to make sure that any addition to autoload_once_paths (and autoload_paths) is done in the body of the engine's application config (not afterwards). – Telpher

@Telpher thank you, that's a good suggestion. I took a look and they are only set in engine.rb (equivalent to application.rb in an app). The docs also say you can set them in the environment-specific configs (e.g., config/environments/test.rb). I have nothing being set in any of the environment-specific configs. – Twinscrew

That is happening inside Rails, for some reason ActiveSupport::Dependencies.autoload_once_paths is frozen, but it should not be. If this is related to you app, it is hard to see from the description, but I'd expect it to be somehow, because only Rails freezes that collection (later). Blind shot: if you are using Spring, could you stop it and try to reproduce? – Niel

@XavierNoria, thanks for the response and the guess. Unfortunately, we're not using Spring. I guess I'll just have to step through the code! Horray! :( – Twinscrew

I've added an update of what I've found by debugging (see above). The initializer is called 34 times by various engines, several of which are from the Rails framework but the majority from 3rd party libraries that are included in the engine's gemspec. – Twinscrew

Yes, that initializer is called by all engines, in addition to the application (which is an engine itself, you know). In that initializer, each engine pushes their own autoload and eager load paths into the global collections stored in AS::Dependencies that you see. When the application has finished booting, Rails freezes that collection, because further edits would have no effect. It does so in the so-called "finisher", here github.com/rails/rails/blob/main/railties/lib/rails/application/…. – Niel

Since those initializers are run and coordinated by Rails itself, the error you are seeing could indicate that there is a thread trying to boot an application that was already booted. I don't know how could that happen, or if it is really what is happening, just helping build a mental model of the situation. – Niel

@XavierNoria Thanks for that insight - gives me something to look into. I wonder how these Engines are able to pre-empt the initializer :set_autoload_paths from completing its execution. If you look at it engine.rb, starting at Line 573, there is code there that also freezes the autoload_paths and autoload_once_paths here github.com/rails/rails/blob/main/railties/lib/rails/… – Twinscrew

Yeah, let me explain. config.* are the blessed way to configure your autoload and eager load paths. They can be set in config/application.rb and config/environments/*.rb, and are processed after those files are executed. However, the global collections are left unfrozen at that point, because people mutate them out there. That is why autoloaders are configured with whatever is there when the finisher runs, and only frozen at that very last point. – Niel

So, in the initializer you pointed to, config.*_paths are used to populate the AS::Dependencies global arrays, and get frozen. But the global arrays are not frozen yet, because people mutate them in application initializers and what not (this was a compromise I had to face when integrating Zeitwerk). So, we allow those direct mutations for backwards compat, consider them when configuring the autoloader in the finisher, and only freeze then. – Niel

If you look at the backtrace, the error happens because a global array happens to be frozen (which is unrelated to config.* being frozen). Only the finisher freezes that one, hence my hypothesis. – Niel

@xaviernoria I just realized you wrote that article on rubyonrails.org. Thank you! So I've had a little more progress. Re-reading the autoloading documentation and being a little more meticulous with applying the concepts there, my test suite is actually running past Rails.application.initialize! I haven't resolved the frozen hash error just yet, but it's coming from different specs now rather than from engine.rb on initialize. I was wondering if you'd be open to starting a chat so we can dialogue off of comments? Let me know! – Twinscrew

@Twinscrew absolutely! Yes, I am the author of the autoloading guides, also of Zeitwerk, and its integration in Rails. Please feel free to write me to [email protected]. – Niel

@Twinscrew did you resolve this issue? If so can you post your findings? Thanks! – Banderilla

Try booting the app from the engine's directory with just rails c. This should reveal the underlying error. For me, it was the Current class couldn't be auto-loaded. Explicitly requiring the file fixed the issue, and I was able to run the tests – Sawfish