Questions about Reloading and Engines in rails with Zeitwerk autoloader

Question

Background:

I am getting some reloading errors in development when switching to Zeitwerk for a rails 6.0 application that uses an engine (thredded). I'm also a developer on thredded, so want to understand this fully before committing any apparent fixes:

I've already read:

• https://guides.rubyonrails.org/autoloading_and_reloading_constants
• https://guides.rubyonrails.org/engines.html#overriding-models-and-controllers (and implemented these fixes in the main app)
• https://github.com/fxn/zeitwerk/blob/main/README.md (amazingly informative)
• https://github.com/fxn/zeitwerk/issues/143

First Question (separation of autoloading and engines): Are engines autoloaded in the same zeitwerk instance/loader as the main app, or are they somehow loaded separately? That is to say does wrapping some code with Rails.application.reloader.to_prepare do ensure that code is run before both the main app and the engine are reloaded.

Second Question (engine code reloading): Are engine's constants reloaded when the main app reloads? (my understanding is yes).

Third Question (configuring engines): Currently Thredded's docs suggest that the configuration of Thredded happens in an initializer -- e.g. Thredded.some_configuration_option = value - but I think that would get wiped away with autoloading? So therefore probably needs to be wrapped with (I think) Rails.application.reloader.to_prepare do (but this isn't what e.g. Devise recommends (see https://github.com/heartcombo/devise/blob/main/lib/generators/templates/devise.rb) and seems to conflict with https://github.com/fxn/zeitwerk/issues/143). What have I misunderstood here?

Fourth Question (all these to_prepares): Can someone explain or point me to docs that clarify the lifecycle difference between:

• Rails.application.reloader.to_prepare do (and is the block run at least once even when it is eager loaded in production with reloading on)
• Rails.application.config.to_prepare do
• SomeEngine::Engine.config.to_prepare do

Any answers would be great. Kind of a long Q with multiple parts. Happy to split this into multiple StackOverflow Qs if appropriate but they seem quite linked.

Hagride

Quick reply: (1) Yes, the main autoloader manages both the application and all engines. (2) Your understanding is correct. (3) The Thredded constant is not reloadable, it is defined in lib/thredded.rb, which is a top-level file required. Only what is in the engines app directory is autoloaded and reloaded (by default, don't see anything special configured). (4) The existing config.to_prepare seems good to me and is what you want to use. What problem do you have exactly? – Fivespot

Thanks @XavierNoria I'm getting problems with some aspect of Thredded being broken when reloading (firstly "undefined method reset_instance!' for Thredded::AllViewHooks:Class", then (if I refresh the page again) "undefined method thredded_can_moderate_messageboards' for #<User:0x00007f89b28bf5e0>"). Maybe something to do with the way that Thredded extends the main app's User class with Thredded::UserExtender maybe? -- github.com/thredded/thredded/blob/…) – Hagride

Also - slightly mindblowing that the top level constant Thredded (hadn't spotted that it was defined in lib) can be loaded quite separately from its nested (namespaced) classes. I don't know why that surprises me, but it does. – Hagride

Actually no, I failed to notice/mention I'm also using the extension to Thredded called Thredded::Workgroup - I can spot that this is overriding thredded in a pre-zeitwerk way (e.g.github.com/thredded/thredded-workgroup/blob/…). I'll update that and see if it fixes my problem. Thanks for your clarifications – Hagride

Hey! Saw you opened an issue in the repo, I'll followup there because the interface is less constrainted. – Fivespot

Discussion continued here github.com/fxn/zeitwerk/issues/240#issuecomment-1327914390 (will come back here to summarize later) – Hagride

Answer 1

Answering my question for the future based on the discussion in comments above and the discussion in https://github.com/fxn/zeitwerk/issues/240.

(1) Yes, the main autoloader manages both the application and all engines.

(2) Yes, engine's constants are reloaded when the main app reloads.

(3) By default only what is in the engine's app directory is autoloaded (unless config.autoload_paths is changed in for example the engines engine.rb) and the engine's top-level constant (in this case the Thredded module) is typically defined outside of app, in lib (e.g. lib/thredded.rb).

(4) approximately definitive answers (any updates appreciated)

• Rails.application.reloader.to_prepare is run when the app boots (even when eager loaded) and each time it is reloaded (in development) -- see https://guides.rubyonrails.org/v6.1.0/autoloading_and_reloading_constants.html#autoloading-when-the-application-boots

• Rails.application.config.to_prepare and SomeEngine::Engine.config.to_prepare are run on boot, and after each reload see https://guides.rubyonrails.org/autoloading_and_reloading_constants.html#use-case-1-during-boot-load-reloadable-code (and I assume after reloader.to_prepare - the relative order of application.config.to_prepare and ...Engine.config.to_prepare will depend on their relative load orders (see https://api.rubyonrails.org/classes/Rails/Engine.html#class-Rails::Engine-label-Loading+priority HT https://mcmap.net/q/1777300/-how-do-i-configure-the-load-order-of-rails-engines)

• Additionally I discovered you can define initializers that your engine needs (and will be run on boot time only before autoloading) with

  initializer "some explanation of what is for" do
     # code that occurs on initialization 
  end
  

  (See https://guides.rubyonrails.org/engines.html#separate-assets-and-precompiling for an example of this initializer method)

  I found this is particularly useful for defining "overrides" when one engine wants to monkey-patch another engine:

  initializer "let the main autoloader ignore this engine's overrides" do
     overrides = root.join("app/overrides")
     Rails.autoloaders.main.ignore(overrides)
  end
  

  This is parallel to what you do in the main app when monkey-patching autoloaded constants in an engine (https://guides.rubyonrails.org/engines.html#overriding-models-and-controllers).

Hagride