How to compose Thor tasks in separate classes/modules/files?

Question

I'm having some trouble getting Thor to do this, so hopefully someone can point out what I'm doing wrong.

I have a main class class MyApp < Thor that I want to break out into separate files for multiple namespaces, like thor create:app_type and thor update:app_type. I can't find any examples that show how one should break apart a Thor app into pieces, and what I've tried doesn't seem to work.

Take for instance, this class I'm trying to break out from the main Thor class:

module Things
  module Grouping

    desc "something", "Do something cool in this group"
    def something
      ....
    end
  end
end

When I try to include or require this in my main class:

class App < Thor
  ....
  require 'grouping_file'
  include Things::Grouping
  ....
end

I get an exception: '<module:Grouping>': undefined method 'desc' for Things::Grouping:Module (NoMethodError)

Is it possible to have multiple namespaces for Thor tasks, and if so, how does one break it out so that you don't have one monolithic class that takes several hundred lines?

Lilllie

Answer 1

Why doesn't it work: when you use desc inside a Thor class, you are actually calling a class method Thor.desc. When you do that in the module, it calls YourModule.desc which obviously doesn't exist.

There are two ways I can suggest to fix this.

Fix one: using Module.included

Did you want to have those tasks reused in multiple Thor classes?

When a module is used as an include in Ruby, the included class method is called. http://www.ruby-doc.org/core/classes/Module.html#M000458

module MyModule
  def self.included(thor)
    thor.class_eval do

      desc "Something", "Something cool"
      def something
        # ...
      end

    end
  end
end

Fix two: separating your Thor classes into multiple files

Did you merely want to separately define tasks in another file?

If so, just reopen your App class in another file. Your Thorfile would look something like:

# Thorfile
Dir['./lib/thor/**/*.rb'].sort.each { |f| load f }

Then your lib/thor/app.rb would contain some tasks for App, while another file lib/thor/app-grouping.rb would contain some more tasks for the same App class.

Cacophony

Answer 2

Use an over-arching module, let's say Foo, inside of which you will define all sub-modules and sub-classes.

Start the definition of this module in a single foo.thor file, which is in the directory from which you will run all Thor tasks. At the top of the Foo module in this foo.thor, define this method:

# Load all our thor files
module Foo
  def self.load_thorfiles(dir)
    Dir.chdir(dir) do
      thor_files = Dir.glob('**/*.thor').delete_if { |x| not File.file?(x) }
      thor_files.each do |f|
        Thor::Util.load_thorfile(f)
      end
    end
  end
end

Then at the bottom of your main foo.thor file, add:

Foo.load_thorfiles('directory_a')
Foo.load_thorfiles('directory_b')

That will recursively include all the *.thor files in those directories. Nest modules within your main Foo module to namespace your tasks. Doesn't matter where the files live or what they're called at that point, as long as you include all your Thor-related directories via the method described above.

Minium

I want to thank you for this. Thor's documentation is horrible and I'm no ruby expert. :) – Graze

Answer 3

I had this same problem, and had almost given up but then I had an idea:

If you write your tasks into Thorfiles rather than as ruby classes, then you can simply require in Ruby files that contain Thor subclasses and they will appear in the list of available tasks when you run thor -T.

This is all managed by the Thor::Runner class. If you look through this you'll see a #thorfiles method which is responsible for looking for files named Thorfile under the current working directory.

All I had to do to a) break my Thor tasks into multiple files whilst b) not having to have a single Thorfile was to create a local subclass of Thor::Runner, overwrite its #thorfile method with one that returned my app specific list of Thor task files and then call its #start method and it all worked:

class MyApp::Runner < ::Thor::Runner
  private
  def thorfiles(*args)
    Dir['thortasks/**/*.rb']
  end
end

MyApp::Runner.start

So I can have any number of Ruby classes defining Thor tasks under thortasks e.g.

class MyApp::MyThorNamespace < ::Thor
  namespace :mynamespace

  # Unless you include the namespace in the task name the -T task list
  # will list everything under the top-level namespace
  # (which I think is a bug in Thor)
  desc "#{namespace}:task", "Does something"
  def task
    # do something
  end
end

I'd almost given up on Thor until I figured this out but there aren't many libraries that deal with creating generators as well as building namespaced tasks, so I'm glad I found a solution.

Vagabond

Answer 4

Thor Documentation really needs to be improved. The following is gleaned from hours of reading code, specs, issues, and google-fu. I can't say that this is the way it's supposed to work, but it certainly will work when setup this way.

When a class inherits from Thor, it gains a few important Class methods.

1. register. This allows you to register a new subcommand as a task
2. class_options. This gives you a hash of all the class options.
3. tasks. This gives you a hash of all the defined tasks.

We can use those to include tasks from many classes into a single runner.

I included a few extra files so that you could see an entire working thor app. Grantesd it doesn't do much...

#############################################################
#my_app/bin/my_app                                          #
#                                                           #
#This file is the executable that requires the MyApp module,#
#then starts the runner.                                    #
#############################################################
#!/usr/bin/env ruby
$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../lib') unless $LOAD_PATH.include(File.dirname(__FILE__) + '/../lib')

require "rubygems" # ruby1.9 doesn't "require" it though
require "my_app"
MyApp::Runner.start

########################################################
#my_app/lib/my_app.rb                                  #
#                                                      #
#This is the main module, used to control the requires #
#the my_app requires should be done last to make sure  #
#everything else is defined first.                     #
########################################################
require 'thor'
require 'thor/group'

module MyApp
  #include other helper apps here

  require 'my_app/runner' #first so all subcommands can register
  require 'my_app/more'
  require 'my_app/config'
end

###################################################################
#my_app/lib/my_app/runner.rb                                      #
#                                                                 #
#This is the main runner class.                                   #
#ALL class_methods should be defined here except for Thor::Groups #
###################################################################
class MyApp::Runner < ::Thor
  class_option :config, :type => :string,
         :desc => "configuration file.  accepts ENV $MYAPP_CONFIG_FILE",
         :default => ENV["MYAPP_CONFIG_FILE"] || "~/.my_apprc" 

  method_option :rf, :type => :numeric,
         :desc => "repeat greeting X times",
         :default => 3
  desc "foo","prints foo"
  def foo
    puts "foo" * options.rf
  end
end

#######################################################################
#my_app/lib/my_app/more.rb                                            #
#                                                                     #
#A Thor Group example.                                                #
#Class_options defined for a Thor Group become method_options when    #
#used as a subcommand.                                                #
#Since MyApp::Runner is already defined when this class is evaluated  #
#It can automatcially register itself as a subcommand for the runner, #
#######################################################################
class Revamp::Init < ::Thor::Group

  class_option :repeat, :type => :numeric,
         :desc => "repeat greeting X times",
         :default => 3

  desc "prints woot"
  def woot
    puts "woot! " * options.repeat
  end

  desc "prints toow"
  def toow
    puts "!toow" * options.repeat
  end

  #This line registers this group as a sub command of the runner
  MyApp::Runner.register MyApp::More, :more, "more", "print more stuff"
  #This line copies the class_options for this class to the method_options of the :more task 
  MyApp::Runner.tasks["more"].options = MyApp::More.class_options
end

#####################################################################
#my_app/lib/my_app/config.rb                                        #
#                                                                   #
#For normal Thor classes, each task must be registered individually #
#####################################################################
class MyApp::Config < Thor

  method_option :dr, :type => :numeric,
         :desc => "repeat greeting X times",
         :default => 3
  desc "show_default", "show the default config"
  def show_default
    puts "default " * options.dr
  end
  MyApp::Runner.register MyApp::Config, :show_default, "show_default", "print default"
  MyApp::Runner.tasks["show_default"].options = MyApp::Config.tasks["show_default"].options

  method_option :cr, :type => :numeric,
         :desc => "repeat greeting X times",
         :default => 3
  desc "show_config", "show the config"
  def show_config
    puts "config " * options.cr
  end
  MyApp::Runner.register MyApp::Config, :show_config, "show_config", "print config"
  MyApp::Runner.tasks["show_config"].options = MyApp::Config.tasks["show_config"].options

end

Variance

Answer 5

You may find this helpful: https://github.com/lastobelus/cleanthor

I wanted to have a thor-based executable for a gem, with namespaced subcommands, but organize the task files according to the normal ruby gem lib/mygem/*/.rb structure.

I also wanted to have a root level Thorfile so that running thor normally in the project directory during development also showed all the gem tasks.

The solution involved the following steps:

• subclassing Thor::Runner in Mygem::Thor::Runner and overriding its private thorfiles and method_missing methods. In method_missing I also stripped the gem name from a command if it appeared.
• the gem executable calls Mygem::Thor::Runner.start
• subclassing Thor::Task in Mygem::Thor::Task and
  • overriding its private namespace class method. The custom namespace method strips out the Mygem::Thor::Tasks part of the tasks' module hierarchies.
  • overriding its private thorfiles method to return Dir[File.join(File.dirname(__FILE__), 'tasks/**/*.rb')]
• now tasks can be organized in lib/mygem/thor/tasks/**/*.rb. They should all inherit from Mygem::Thor::Task
• the Thorfile at the root of the project also loads all the tasks in lib/mygem/thor/tasks/**/*.rb

Hixson

Answer 6

desc is a class method, you need to use extend instead of include. Look here for an explanation.

Juvenescent

This solution works, not sure why it was voted down. It is the simplest IMO. At the top of your template relative_require ..\myhelpers\mystuff.rb then extend MyStuff and voila the functions defined in module MyStuff may be called in your template and those functions can use thor commands as needed. I'm on Ruby 3.0.1 and Rails 6.1.4 under Ubuntu 20.04.02 LTS. – Blowzy