Tests not run automatically

Asked 5/12, 2014 at 20:55 Answered 10/1, 2018 at 1:49

I'm trying to setup Guard in my Exercism folder, so that all tests are automatically run (using guard-minitest).

The folder structure looks like this:

.
├── exercism
└── ruby
    ├── bob
    │   ├── bob.rb
    │   ├── bob_test.rb
    ├── etl
    │   ├── etl.rb
    │   ├── etl_test.rb
...
    └── binary_search_tree
        ├── bst.rb
        ├── binary_search_tree.rb
        └── README.md

Due to the unusual folder structure, I made the following configuration in the Guardfile:

# tell Guard to watch all folders under ruby/
all_dirs = Dir.glob('ruby/*')

directories all_dirs

options = {
  test_folders: all_dirs, 
  test_file_patterns: '*_test.rb',
  pride: true
}

guard :minitest, options do   
  watch(%r{^ruby/([^/]+)/([^/]+).rb$})
end

With this setup I expect that all my tests will be run after any edit to a .rb file.

I run guard in debug mode.

All tests run on startup, as expected.

However, when I edit a .rb file the tests are not run again, although Guard outputs this:

20:43:26 - DEBUG - Interactor was stopped or killed    
20:43:26 - DEBUG - Hook :run_on_additions_begin executed for Guard::Minitest
20:43:26 - DEBUG - Hook :run_on_additions_end executed for Guard::Minitest
20:43:26 - DEBUG - Start interactor

I tried many variations in the guard configuration, such as:

watch(%r{^ruby/([^/]+)/([^/]+).rb$}) do |m|
  Dir.glob("ruby/#{m[1]}/*_test.rb").first  
end

(I expect this to only run the test from the folder in which a file was modified.)

Nothing seems to work. I have to go to guard's interactive console and press ENTER to get tests to run.

What am I doing wrong?

EDIT

After reading Cezary's answer below, I tried some more things, including trying this on other environments.

It turns out it wasn't guard's fault, but probably the environment I was using.

I initially ran everything in a Codio box. I moved the project to other two machines, a Win8 and an Ubuntu 14.04, and it works fine on both using this Guardfile:

all_dirs = Dir.glob('ruby/*')

directories all_dirs

clearing :on

options = {
  test_folders: all_dirs, 
  test_file_patterns: '*_test.rb',
  pride: true
}

guard :minitest, options do
  watch(%r{^ruby/([^/]+)/([^/]+).rb$}) do |m|
    file = Dir.glob("ruby/#{m[1]}/*_test.rb").first
    puts "  Should test #{file}"
    file
  end
end

The output of guard -d is as follows:

On the Codio box (where it doesn't work):

<!-- language: lang-none -->

08:55:09 - DEBUG - Interactor was stopped or killed
  Should test ruby/anagram/anagram_test.rb
08:55:09 - DEBUG - Hook :run_on_additions_begin executed for Guard::Minitest
08:55:09 - DEBUG - Hook :run_on_additions_end executed for Guard::Minitest
08:55:09 - DEBUG - Start interactor
[1] guard(main)>

On Win/Ubuntu (where it works fine):

<!-- language: lang-none -->

11:02:10 - DEBUG - Interactor was stopped or killed
  Should test ruby/anagram/anagram_test.rb
11:02:10 - DEBUG - Hook :run_on_modifications_begin executed for Guard::Minitest
11:02:10 - INFO - Running: ruby/anagram/anagram_test.rb
Run options: --seed 52507

# Running tests:

..........

Finished tests in 0.001249s, 8006.0205 tests/s, 8006.0205 assertions/s.

10 tests, 10 assertions, 0 failures, 0 errors, 0 skips

11:02:10 - DEBUG - Hook :run_on_modifications_end executed for Guard::Minitest
11:02:10 - DEBUG - Start interactor
[1] guard(main)>

I don't know why on Codio I get run_on_additions and on the other two I get run_on_modifications. Anyway, I guess that's a Codio issue. I tried with and without Manual Save and it's the same.

Porosity answered 5/12, 2014 at 20:55 Comment(0)

Hi I'm one of the guys "reponsible" for Guard, and based on this I wrote a walkthrough: https://github.com/guard/guard/wiki/Understanding-Guard

I highly recommend going through it (feedback appreciated).

Here's a list of things to consider:

structure: if you can, put tests in a separate top folder, ideally 'ruby/test' in your example, and the files in 'ruby/lib'. This is closer to the convention and then things like Guard::Minitest would work out of the box. (Some editors - like Vim and Emacs - allow you to switch between "alternative" files and they will automatically know where to look for tests or implementation files).
Guard watches directories recursively (and this can't even be turned off right now). For small projects, you can just watch everything by not specifying a directories option. But, if you have lots of disk activity in your folder and you have large projects, you'll want to select directories. E.g. in your case, it would be: directories ruby. So `Dir.glob('ruby/*') doesn't make much sense here.
Guard::Minitest options - if you use the structure I described, this shouldn't be necessary. If not, I think having test_folder: %w(ruby) should be enough. Also, test_file_patterns shouldn't be necessary, since your files seem to following the defaults/convention (bob_test.rb and etl_test.rb) - unless you have something really strange as having test suites inside the bob.rb and etl.rb files.
Your watch expression has no block, so it returns the files changed. Unfortunately, when you change an implementation file (like bob.rb), that file is passed to Guard::Minitest, which ignore non-test files (probably using the test_file_patterns option, so since bob.rb won't match bob_test.rb, Guard::Minitest will quietly do ... nothing.
Ideally, rename your Guardfile, update your gems (Guard::Minitest basically) and run bundle exec guard init minitest to so what the "current" recommended template is for Guard::Minitest and try to tweak that instead. You'll see that the default has:
```
watch(%r{^lib/(.*/)?([^/]+)\.rb$})  { |m| "test/#{m[1]}test_#{m[2]}.rb" }
```

which shows how to translate changed implementations files into test files (which are the only files Guard::Minitest cares about).

In your case you may want:

watch(%r{^ruby/(.*/)?([^/]+)\.rb$}) { |m| "ruby/#{m[1]}/#{m[2]}_test.rb" }

I'd say the regexen are horrible - and I'm planning to implement glob pattern support - but this will take a while (lots of critical bug fixes in the pipeline for me).

If this doesn't help - definitely go through the Wiki document above, since there are about 20-50 problems you can have with Guard/Listen - and most are completely outside Guard's/Listen's control. (I wish I could change this to make it simpler, so any suggestions would be helpful).
Since bugs are a priority for me, like most maintainers - we rely on issues reported int GitHub. So we don't usually watch stuff in Stack Overflow (we expect tough issues to be reported in Github).

Ok, I've pointed out so many things wrong, time for stuff you did very, very well:

Running Guard in debug mode - you're AWESOME for doing that
Showing output regarding the changes - you're AWESOME for doing that as well (since this shows the Guard::Minitest is getting the changes, but it's ignoring them - likely since you're passing implementation files and not test files).
Submitting such a detailed issue - that's so helpful. Ideally, you shouldn't have gone through all the trouble (or even had the issue in the first place), but by reporting this you're showing us what is wrong with Guard on so many levels. (Often, as developers/maintainers we "get" the internals, so we "know" what's wrong immediately - even though the docs or messages or debug output says nothing to others - so issues like this help us help others very much).

Thanks again for this - if you ever see other people with issues you can resolve - please do so. If you see obvious bugs or issues in Guard - please open an issue in Guard. If it's in another plugin repository (like Guard::Minitest), just mention me so that important issues aren't ignored (mention me as @e2 on GitHub).

I do hope to make Guard better - currently I'm working hard to make it simpler, more intuitive, more flexible and more reliable. The more I can count on people reporting broken stuff, the faster I can move ahead with cool stuff.

Thanks again and have a great day!

Himelman answered 17/12, 2014 at 0:27 Comment(5)

Thank you for the detailed response and the walkthrough. You're AWESOME. The part where I'm not awesome is that I forgot to mention the environment where I was running guard; turns out it was not guard's fault at all. I posted details in an edit to my question. – Porosity 17/12, 2014 at 9:11

Check out the LISTEN_GEM_DEBUGGING=2 option in Listen - if you're getting different events on Codio, that would explain everything. If you upload the output to a Gist and/or file an issue in Listen, I could maybe cook up a fix/workaround in Listen. – Himelman 17/12, 2014 at 9:16

Here's the LISTEN DEBUG output on codio and on a working machine: gist.github.com/clupascu/705022748d6bc28cb9dd – Porosity 17/12, 2014 at 9:33

What are you using the edit the file? I don't have the :moved_from event in that output (should have the same cookie as the :moved_to). – Himelman 17/12, 2014 at 11:43

I created a GitHub issue here: github.com/guard/listen/issues/287 - let's move this discussion there (so it's easier for me to keep track, link with pull requests, etc). – Himelman 17/12, 2014 at 11:47

I had a similar issue when wanting to use Guard with any new Ruby Exercism exercise I'd pull down: running the full test suite would work, but tests would not run automatically when files changed. What I did in order to get this working was the following:

Create a common Guardfile and place it at ~/exercism/ruby/Guardfile with the following content:

# frozen_string_literal: true

group :red_green_refactor, halt_on_fail: true do
  guard :minitest,
        all_on_start: false,
        test_folders: ["."] do
    # Re-test test files when they're edited.
    watch(%r{\A.+_test\.rb\z}) { |m| "./#{m[1]}" }
    # Run the test file of the (non-test) file that was edited.
    watch(%r{\A(.+)(?<!_test)\.rb\z}) { |m| "./#{m[1]}_test.rb" }
  end

  guard :rubocop,
        all_on_start: false,
        cli: ["--display-cop-names"] do
    # Only run Rubocop over implementation files only
    # as test files are not written by me.
    watch(%r{\A(.+)(?<!_test)\.rb\z})
    watch(%r{(?:.+/)?\.rubocop\.yml\z}) { |m| File.dirname(m[0]) }
  end
end

When running Guard within an exercise folder, always specifically reference this Guardfile ie guard --guardfile ~/exercism/ruby/Guardfile.

Some specific gotchas here about the Guardfile configuration:

The test_folders option, the locations where guard-minitest will go looking for test files, defaults to %w[test spec]. This is fine for any standard Ruby project, but not for Exercism, where the test file and the implementation file are in the same directory. So, it needed to be changed to ["."] to specify that.
In the blocks for the Minitest watch methods, the string returned needs to look like a path, otherwise the tests will just not run when a file is changed (with no error output or indication that something went wrong). For example:

watch(%r{\A.+_test\.rb\z}) { |m| "#{m[1]}" } will not work. It has to be watch(%r{\A.+_test\.rb\z}) { |m| "./#{m[1]}" }.

Paillette answered 10/1, 2018 at 1:49 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

EDIT

Recommended topics

Hot tags