Autoload paths and nested services classes crash in Ruby

Question

I've multiple issues to load / require classes under my app/services folder in a Rails 5 project and I'm starting to give up on this issue.

First of all and to be clear, services/ are simple PORO classes I use throughout my project to abstract most of the business logic from the controllers, models, etc.

The tree looks like this

app/
 services/
  my_service/
    base.rb
    funny_name.rb
  my_service.rb  
models/
 funny_name.rb

Failure #1

First, when I tried to use MyService.const_get('FunnyName') it got FunnyName from my models directory. It does not seem to have the same behavior when I do MyService::FunnyName directly though, in most of my tests and changes this was working fine, it's odd.

I realised Rails config.autoload_paths does not load things recursively ; it would makes sense that the first FunnyName to be catch is the models/funny_name.rb because it's definitely loaded but not the other.

That's ok, let's find a workaround. I added this to my application.rb :

config.autoload_paths += Dir[Rails.root.join('app', 'services', '**/')]

Which will add all the subdirectories of services into config.autoload_paths. Apparently it's not recommended to write things like that since Rails 5 ; but the idea does look right to me.

Failure #2

Now, when I start my application it crashes and output something like this

Unable to autoload constant Base, expected /.../backend/app/services/my_service/base.rb to define it (LoadError)

Names were changed but it's the matching path from the tree I wrote previously

The thing is, base.rb is defined in the exact file the error leads me, which contains something like

class MyService
  class Base
  end
end

Poor solution

So I try other workaround, lots of them, nothing ever works. So I end up totally removing the autoload_paths and add this directly in the application.rb

Dir[Rails.root.join('app', 'services', '**', '*.rb')].each { |file| require file }

Now the base.rb is correctly loaded, the MyService.const_get('FunnyName') will actually return the correct class and everything works, but it's a disgusting workaround. Also, it has yet not been tested in production but it might create problems depending the environment.

Requiring the whole tree from the application.rb sounds like a bad idea and I don't think it can be kept this way.

What's the cleanest way to add custom services/ directory in Rails ? It contains multiple subdirectories and classes with simple names which are also present in other parts of the app (models, base.rb, etc.)

How do you avoid confusing the autoload_paths ? Is there something else I don't know which could do the trick ? Why did base.rb even crash here ?

Predictory

Please add one of your service definitions (first line should do it) to your question. – Allfired

what do you mean ? should i write what's a service to me ? – Predictory

I mean, does app/services/my_class/base.rb start like class MyClass::Base? – Allfired

It's written below, sorry I had to edit it at first because i wasn't highlighted as code – Predictory

Oh, I see. That's incorrect. class MyClass should be module MyClass. – Allfired

this is an example ... there would be a my_class.rb at the base as well to be more correct, i'll add it – Predictory

Answer 1

Working solution

After deeper investigation and attempts, I realised that I had to eager_load the services to avoid getting wrong constants when calling meta functionalities such as const_get('MyClassWithModelName').

But here's is the thing : the classic eager_load_paths won't work because for some reason those classes will apparently be loaded before the entire core of Rails is initialized, and simple class names such as Base will actually be mixed up with the core, therefore make everything crash.

Some could say "then rename Base into something else" but should I change a class name wrapped into a namespace because Rails tell me to ? I don't think so. Class names should be kept simple, and what I do inside a custom namespace is no concern of Rails.

I had to think it through and write down my own hook of Rails configuration. We load the core and all its functionalities and then service/ recursively.

On a side note, it won't add any weight to the production environment, and it's very convenient for development.

Code to add

Place this in config/environment/development.rb and all other environment you want to eager load without Rails class conflicts (such as test.rb in my case)

# we eager load all services and subdirectories after Rails itself has been initializer
# why not use `eager_load_paths` or `autoload_paths` ? it makes conflict with the Rails core classes
# here we do eager them the same way but afterwards so it never crashes or has conflicts.
# see `initializers/after_eager_load_paths.rb` for more details
config.after_eager_load_paths = Dir[Rails.root.join('app', 'services', '**/')]

Then create a new file initializers/after_eager_load_paths.rb containing this

# this is a customized eager load system
# after Rails has been initialized and if the `after_eager_load_paths` contains something
# we will go through the directories recursively and eager load all ruby files
# this is to avoid constant mismatch on startup with `autoload_paths` or `eager_load_paths`
# it also prevent any autoload failure dû to deep recursive folders with subclasses
# which have similar name to top level constants.
Rails.application.configure do
  if config.respond_to?(:after_eager_load_paths) && config.after_eager_load_paths.instance_of?(Array)
    config.after_initialize do
      config.after_eager_load_paths.each do |path|
        Dir["#{path}/*.rb"].each { |file| require file }
      end
    end
  end
end

Works like a charm. You can also change require by load if you need it.

Predictory

Glad you got this working. As to your question: "should I change a class name wrapped into a namespace because Rails tell me to?". One of the Rails mantras is "convention over configuration". So, if you buy into the mantra, then the answer would likely be "yes". But, you've done things the way that works for you. So, nice work! – Allfired

but aren't namespaces supposed to be used for that purpose ? split things so you can work on them without worrying about what's outside ? – Predictory

Answer 2

When I do this (which is in all of my projects), it looks something like this:

app
 |- services
 |   |- sub_service
 |   |   |- service_base.rb
 |   |   |- useful_service.rb     
 |   |- service_base.rb

I put all common method definitions in app/services/service_base.rb:

app/services/service_base.rb

class ServiceBase

  attr_accessor *%w(
    args
  ).freeze

  class < self 

    def call(args={})
      new(args).call
    end

  end

    def initialize(args)
      @args = args
    end

end

I put any methods common to the sub_services in app/services/sub_service/service_base.rb:

app/services/sub_service/service_base.rb

class SubService::ServiceBase < ServiceBase

    def call

    end

  private

    def a_subservice_method
    end

end

And then any unique methods in useful_service:

app/services/sub_service/useful_service.rb

class SubService::UsefulService < SubService::ServiceBase

    def call
      a_subservice_method
      a_useful_service_method
    end

  private

    def a_useful_service_method
    end

end

Then, I can do something like:

SubService::UsefulService.call(some: :args)

Allfired

Alright, but the issue isn't about those, it's more about rails autoloading and customised directories ... the biggest error being base.rb apparently can't be defined twice in the same project and a service with the same name of a model - which can occur - has some kind of mismatch .. – Predictory

Welp, I guess the point is that if you do it like I showed (which is pretty much the same as the other answer, IMO), then autoloading and customised directories will work flawlessly. But, yes, you are correct. If you begin doing things in unconventional ways, you will run into problems with autoloading and customised directories. – Allfired

Well it's not about being unconventional so much, if a service starts to be really complex (i'm saying service but it could be any design pattern) you need to abstract into multiple PORO down the class so it's easily testable, etc. but why would you use "_service" to each one of those internally used subclasses to avoid collision ? Namespaces are supposed to deal with this exact issue. I hope you see what I mean, I don't get why Rails do that, and how to solve it properly .. Still investigating ... – Predictory

Defining class Base inside of class MyClass in the file 'app/services/my_class/base.rb` is unconventional, I believe. That file structure, conventionally, expects that MyClass is a module, not a class. Which is what both answers recommend. But, hey, what do I know? BTW, in my projects, I have dozens and dozens of POROs (services, decorators, presenters, managers, etc.) that call each other in endless permutations. For things I use over and over, I extract them into gems. All testable. All quite lovely. – Allfired

... Right a module .. Except for my_class.rb on the side of the tree I wrote which could be anything, right ? But, hey, what do I know ? Now if you read carefully and rename MyClass by MyClassService I match your convention, I highlighted the internal aspect for a reason, did we solve my problem by changing the name ? No we didn't. The issue is a loading problem which could be applied to any subfolder not a "this is a service it should be named like this" one. You all replied out of topic and all read "service" as it was the problem, not even quoting the const_get problem. – Predictory

I already solved the issue myself, but the solution isn't ideal and I wanted expert opinions to preload those constants in the right place, in the right way. Yes turning things into gem is a good idea but overdone for this example. Also, we talk big corporate project here. – Predictory

It seems neither of the answers provide you with a satisfactory solution for autoloading and customized directory structures. I'll look forward to better answers. Best of luck. – Allfired

Thanks for your help btw, I think I found a working solution today and will post it there at some point if no one has better than that :) – Predictory

Answer 3

With your tree,

app/
 services/
  my_class/
    base.rb
    funny_name.rb
  my_class.rb  
models/
 funny_name.rb

services/my_class/base.rb should look similar to:

module MyClass
  class Base

services/my_class/funny_name.rb should look similar to:

module MyClass
  class FunnyName

services/my_class.rb should look similar to:

class MyClass

models/funny_name.rb should look similar to:

class FunnyName

I say "should look similar to" because class/module are interchangable; Rails is merely looking for these constants to be defined in these locations.

You don't need to add anything to your autoload path. Rails automatically picks up everything in app

Anecdotal: With your services directory, it's fairly common to treat their naming convention (both name of file and underlying constant) to be "_service.rb" or "ThingService" — just like how controllers look. Models don't get this suffix because they're treated as first-class objects.

GitLab has some great file structure that is very worth a look at. https://gitlab.com/gitlab-org/gitlab-ce

Cordie

Thanks, i know all this, actually all my services classes have "_service.rb" added at the end, but those services can become very complex so i usually create "my_service/" directory and then inside ... that's where it goes wrong. Due to the numbers of classes it can contain, and the fact those classes are called internally to the service, I keep the names simple, like base.rb but I started to get weird results when the names match some models, or other classes ... Now i'm wondering how to find a good workaround for this – Predictory

The real issue is, if it's namespaced, why is there a problem with name crash ? – Predictory

Let me see if I can reproduce it real quick, one sec. – Cordie

I wasn't able to reproduce it with a really naive example. If you want to reproduce this in an example Rails app and push it to Github I can give it a look. – Cordie

This is the solution, people. Just name your files correctly, and have the correct module and class names in your code, and they will autoload, even in subdirectories. – Xeroderma