Go: how to run tests for multiple packages?

Question

I have multiple packages under a subdirectory under src/, running the tests for each package with go test is working fine.

When trying to run all tests with go test ./... the tests are running but it fails..

the tests are running against local database servers, each test file has global variables with db pointers.

I tried to run the tests with -parallel 1 to prevent contention in the db, but the tests still fail.

what can be the issue here?

EDIT: some tests are failing on missing DB entries, I completely clear the DB before and after each test. the only reason I can think of why this is happening is because of some contention between tests.

EDIT 2:

each one of my test files has 2 global variables (using mgo):

var session *mgo.Session
var db *mgo.Database

also it has the following setup and teardown functions:

func setUp() {
   s, err := cfg.GetDBSession()
   if err != nil {
       panic(err)
   }

   session = s

   db = cfg.GetDB(session)

   db.DropDatabase()
}

func tearDown() {
   db.DropDatabase()

   session.Close()
}

each tests startup with setUp() and defer tearDown()

also cfg is:

package cfg

import (
    "labix.org/v2/mgo"
)

func GetDBSession() (*mgo.Session, error) {
    session, err := mgo.Dial("localhost")

    return session, err
}

func GetDB(session *mgo.Session) *mgo.Database {
    return session.DB("test_db")
}

EDIT 3:

I changed cfg to use a random database, the tests passed. it seems that the tests from multiple packages are running somewhat in parallel.

is it possible to force go test to run everything sequentially across packages ?

Stanley

What's the error message? Do you use any files? Relative paths might be wrong when go test is launched from a different directory. – Incontestable

What exactly fails? The more information you provide, the more we can help you. – Eckstein

To be clear: the answer to the stated question is go test ./.... I do it every day. You have some other question related to your specific situation, and you need to more clearly ask that question. – Probate

some tests seem to fail because of missing DB entries, I completely drop the DB before and after each test, so the only reason I can think of why this is happening when running go test ./... and everything passes when running a single test file is because of some contention between tests... – Stanley

What do you mean by "before and after each test?" How are you implementing that? A simplified example of your tests would go a long way here. – Probate

I added some more info to the question – Stanley

Answer 1

apparently running go test -p 1 runs everything sequentially (including build), I haven't see this argument in go help test or go help testflag

Stanley

It's in there: golang.org/cmd/go/#hdr-Compile_packages_and_dependencies : "-p n the number of programs, such as build commands or test binaries, that can be run in parallel. The default is the number of CPUs available, except on darwin/arm which defaults to 1." – Wayzgoose

Answer 2

Update: As pointed out by @Gal Ben-Haim, adding the (undocumented) go test -p 1 flag builds and tests all packages in serial. As put by the testflag usage message in the Go source code:

-p=n: build and test up to n packages in parallel

Old answer:

When running go test ./..., the tests of the different packages are in fact run in parallel, even if you set parallel=1 (only tests within a specific package are guaranteed to be run one at a time). If it is important that the packages be tested in sequence, like when there is database setup/teardown involved, it seems like the only way right now is to use the shell to emulate the behavior of go test ./..., and forcing the packages to be tested one by one.

Something like this, for example, works in Bash:

find . -name '*.go' -printf '%h\n' | sort -u | xargs -n1 -P1 go test

The command first lists all the subdirectories containing *.go files. Then it uses sort -u to list each subdirectory only once (removing duplicates). Finally all the subdirectories containing go files get fed to go test via xargs. The -P1 indicates that at most one command is to be run at a time.

Unfortunately, this is a lot uglier than just running go test ./..., but it might be acceptable if it is put into a shell script or aliased into a function that's more memorable:

function gotest(){   find $1 -name '*.go' -printf '%h\n' | sort -u | xargs -n1 -P1 go test; }

Now all tests can be run in the current directory by calling:

gotest .

Catchall

Oh wow, you're right - I actually checked the Go source code before posting this answer and yet somehow still didn't notice that option. – Catchall

Answer 3

I am assuming that because the packages individually pass that in this situation you are also dropping the DB before that test as well.

Therefore it sounds like the state of the DB for each package test is expected to be empty.
So between each set of the package tests the DB must be emptied. There are two ways around this, not knowing your entire situation I will briefly explain both options:

Option 1. Test Setup

Add an init() function to the start of each package _test file which you then put processing to remove the DB. This will be run before the init() method of the actual package:

func init() {
    fmt.Println("INIT TEST")
    // My test state initialization
    // Remove database contents
}

Assuming that the package also had a similar print line you would see in the output (note the stdout output is only displayed when the a test fails or you supply the -v option)

INIT TEST
INIT PACKAGE

Option 2. Mock the database

Create a mock for the database (unless that is specifically what you are testing). The mock db can always act like the DB is blank for the starting state of each test.

Acetanilide

I added some more info to the question, isn't what I'm doing before and after each test is enough ? – Stanley

Based on what you have added in your second edit, as far as my knowledge goes its seems enough. Maybe its a db server restriction getting in the way? – Acetanilide

its just a locally installed Mongodb on Ubuntu.. I have no problems running similar tests against it with Python or Node.js. – Stanley

is it possible that the tests are somehow run in parallel ? – Stanley

Answer 4

I used a cheap way to pick-up tests in multiple packages.

go test -v ./... -run "TestUnit?"

Just name your tests so they get caught:

func TestUnitCountriesStructs(t *testing.T)

You mentioned about needing some sort of set up and teardown for your tests. I use testify. In short:

import (
    "testing"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/suite"
)

type ExampleTestSuite struct {
    suite.Suite
    VariableThatShouldStartAtFive int
}

// before each test
func (suite *ExampleTestSuite) SetupTest() {
    suite.VariableThatShouldStartAtFive = 5
}

func (suite *ExampleTestSuite) TestExample() {
    assert.Equal(suite.T(), 5, suite.VariableThatShouldStartAtFive)
    suite.Equal(5, suite.VariableThatShouldStartAtFive)
}

func TestUnitExampleTestSuite(t *testing.T) {
    suite.Run(t, new(ExampleTestSuite))
}

Beforehand

Answer 5

Please try out the following github repository.

https://github.com/appleboy/golang-testing

Copy coverage.sh to /usr/local/bin/coverage and change permission.

$ curl -fsSL https://raw.githubusercontent.com/appleboy/golang-testing/master/coverage.sh /usr/local/bin/coverage
$ chmod +x /usr/local/bin/coverage

Hekate