`go test` succeed for individual tests but fails when testing package

Question

When I run single test with go test -run TestNewProbeServiceIsSingleton it passes as expected.

Problem occurs whenever I am trying to test the whole package/app using go test ./... from my project's root directory:

madjlzz@MadSfeirLab $ go test ./...              
?       github.com/madjlzz/madprobe     [no test files]
ok      github.com/madjlzz/madprobe/controller  (cached) [no tests to run]
?       github.com/madjlzz/madprobe/internal/alerter    [no test files]
?       github.com/madjlzz/madprobe/internal/mock       [no test files]
?       github.com/madjlzz/madprobe/internal/persistence        [no test files]
--- FAIL: TestInsertReturnErrorOnGetFailure (0.00s)
panic: Fail in goroutine after TestNewProbeServiceIsSingleton has completed [recovered]
        panic: Fail in goroutine after TestNewProbeServiceIsSingleton has completed [recovered]
        panic: Fail in goroutine after TestNewProbeServiceIsSingleton has completed

goroutine 23 [running]:
testing.tRunner.func1(0xc0000f6400)
        /usr/local/Cellar/go/1.13.6/libexec/src/testing/testing.go:874 +0x3a3
panic(0x12ccca0, 0xc000095190)
        /usr/local/Cellar/go/1.13.6/libexec/src/runtime/panic.go:679 +0x1b2
github.com/golang/mock/gomock.(*Controller).Finish(0xc000098ff0)
        /Users/madjlzz/Documents/Projects/Go/pkg/mod/github.com/golang/[email protected]/gomock/controller.go:246 +0x2b2
panic(0x12ccca0, 0xc000095190)
        /usr/local/Cellar/go/1.13.6/libexec/src/runtime/panic.go:679 +0x1b2
testing.(*common).Fail(0xc0000f6200)
        /usr/local/Cellar/go/1.13.6/libexec/src/testing/testing.go:609 +0x151
testing.(*common).FailNow(0xc0000f6200)
        /usr/local/Cellar/go/1.13.6/libexec/src/testing/testing.go:631 +0x2b
testing.(*common).Fatalf(0xc0000f6200, 0x1351581, 0x2e, 0xc0000c4140, 0x5, 0x5)
        /usr/local/Cellar/go/1.13.6/libexec/src/testing/testing.go:716 +0x90
github.com/golang/mock/gomock.(*Controller).Call.func1(0xc000098de0, 0x1301ca0, 0xc000094f90, 0x1343807, 0x3, 0xc000095110, 0x1, 0x1, 0x0, 0x0, ...)
        /Users/madjlzz/Documents/Projects/Go/pkg/mod/github.com/golang/[email protected]/gomock/controller.go:201 +0x486
github.com/golang/mock/gomock.(*Controller).Call(0xc000098de0, 0x1301ca0, 0xc000094f90, 0x1343807, 0x3, 0xc000095110, 0x1, 0x1, 0x5, 0xc00008a660, ...)
        /Users/madjlzz/Documents/Projects/Go/pkg/mod/github.com/golang/[email protected]/gomock/controller.go:217 +0xb4
github.com/madjlzz/madprobe/internal/mock.(*MockPersister).Get(0xc000094f90, 0x134426b, 0x7, 0x16, 0x0, 0x0)
        /Users/madjlzz/Documents/Projects/Go/src/github.com/madjlzz/madprobe/internal/mock/entity.go:53 +0xe5
github.com/madjlzz/madprobe/internal/prober.(*service).Insert(0xc000098e70, 0x134426b, 0x7, 0x1348f2d, 0x16, 0x0, 0x0, 0x5, 0xc00008a660, 0x156df40, ...)
        /Users/madjlzz/Documents/Projects/Go/src/github.com/madjlzz/madprobe/internal/prober/service.go:63 +0x19b
github.com/madjlzz/madprobe/internal/prober.TestInsertReturnErrorOnGetFailure(0xc0000f6400)
        /Users/madjlzz/Documents/Projects/Go/src/github.com/madjlzz/madprobe/internal/prober/service_test.go:59 +0x4fb
testing.tRunner(0xc0000f6400, 0x135be98)
        /usr/local/Cellar/go/1.13.6/libexec/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
        /usr/local/Cellar/go/1.13.6/libexec/src/testing/testing.go:960 +0x350
FAIL    github.com/madjlzz/madprobe/internal/prober     0.287s
?       github.com/madjlzz/madprobe/util        [no test files]
FAIL

App builds without any errors:

madjlzz@MadSfeirLab $ go build .   
madjlzz@MadSfeirLab $ 

My Golang version is:

madjlzz@MadSfeirLab $ go version
go version go1.13.6 darwin/amd64

I am using also mockgen to mock my interfaces in version 1.4.3

I am new to Golang but it feels I kinda miss something on how to run tests...

You can also run the tests by yourself by cloning the project

Marta

This is hard to debug without code, but the fact alone that you are working with a singleton is a red flag. Another test might be mutating the singleton's value, leading to the crash. – Rugby

You have the code available in GitHub here: github.com/MadJlzz/madprobe/tree/bug/testing. So it means that golang is sharing the memory between multiple tests ? If this is the case, that might be a problem since I am using sync.Once and one of my mocked function is called inside Do() of sync.Once... – Marta

Is there any real compelling reason to use a singleton? The only place where NewProbeService is used is in your main function, so maybe a better fix might be to just not use the singleton pattern and spare yourself the pain that comes with it :) – Trejo

"So it means that golang is sharing the memory between multiple tests?" - Yes, it is. It all runs in one process. – Rugby

@deR_Ed, well there is one that might break the app if someone (me), is creating multiple ProbeService. But you guys are right, I am gonna remove that pattern to stay simple. I'll cover the breaking case in another way :) @Zyl, yup so now it's all clear, I have to be careful next time. Good job pinpointing the problem! – Marta

@Marta what fix did you do? Facing the same problem. – Zoltai

Answer 1

What's probably happening (I don't have the source code available to verify) is that your code is spinning up a goroutine, and then the test is exiting, but the goroutine is still running and attempting to do stuff. For example:

package panicatthegocode

import (
    "testing"
    "time"
)

func TestPanicMaybe(t *testing.T) {
    go func() {
        time.Sleep(1 * time.Millisecond)
        t.Skip("Skipping")
    }()
}

yields

$ go test . -count=100
panic: Log in goroutine after TestPanicMaybe has completed: Skipping


goroutine 32 [running]:
testing.(*common).logDepth(0xc0000db860, {0xc000116020, 0x9}, 0x3)  
        C:/Program Files/Go/src/testing/testing.go:1028 +0x4c5      
testing.(*common).log(...)
        C:/Program Files/Go/src/testing/testing.go:1010
testing.(*common).Skip(0xc0000db860, {0xc0000fbfc0?, 0x0?, 0x0?})   
        C:/Program Files/Go/src/testing/testing.go:1095 +0x4a       
leetcode.TestPanicMaybe.func1()
        C:/Users/mparn/Desktop/panic_test.go:11 +0x5c
created by leetcode.TestPanicMaybe in goroutine 31
        C:/Users/mparn/Desktop/panic_test.go:9 +0x4f
panic: Log in goroutine after TestPanicMaybe has completed: Skipping

Vair

Answer 2

I assume ur code looks like (I had such problem):

for _, subtest := range subtests{
   t.Run(subtest.name,
        func(t *testing.T) {
           do something with subtest...
        }
}

But it must look like:

for _, s := range subtests{
   subtest:=s

   t.Run(subtest.name,
        func(t *testing.T) {
           do something with subtest...
        }
}

Attention on subtest:=s

This is one of the possible solutions in this situation.

Sanitary

i have the same problem, few tests that are part of same suite are breaking when run together, but running fine individually. Your fix also does not work for me. – Zoltai

This doesn't address OP's problem. What you suggested is for parallel tests – Vair