Unit testing for functions that use gorilla/mux URL parameters

Question

TLDR: gorilla/mux used to not offer the possibility to set URL Vars. Now it does, that's why the second-most upvoted answer was the right answer for a long time.

Original question to follow:

---

Here's what I'm trying to do :

main.go

package main

import (
    "fmt"
    "net/http"
    
    "github.com/gorilla/mux"
)
    
func main() {
    mainRouter := mux.NewRouter().StrictSlash(true)
    mainRouter.HandleFunc("/test/{mystring}", GetRequest).Name("/test/{mystring}").Methods("GET")
    http.Handle("/", mainRouter)
    
    err := http.ListenAndServe(":8080", mainRouter)
    if err != nil {
        fmt.Println("Something is wrong : " + err.Error())
    }
}

func GetRequest(w http.ResponseWriter, r *http.Request) {
    vars := mux.Vars(r)
    myString := vars["mystring"]
    
    w.WriteHeader(http.StatusOK)
    w.Header().Set("Content-Type", "text/plain")
    w.Write([]byte(myString))
}

This creates a basic http server listening on port 8080 that echoes the URL parameter given in the path. So for http://localhost:8080/test/abcd it will write back a response containing abcd in the response body.

The unit test for the GetRequest() function is in main_test.go :

package main

import (
    "net/http"
    "net/http/httptest"
    "testing"

    "github.com/gorilla/context"
    "github.com/stretchr/testify/assert"
)

func TestGetRequest(t *testing.T) {
    t.Parallel()
    
    r, _ := http.NewRequest("GET", "/test/abcd", nil)
    w := httptest.NewRecorder()

    //Hack to try to fake gorilla/mux vars
    vars := map[string]string{
        "mystring": "abcd",
    }
    context.Set(r, 0, vars)
    
    GetRequest(w, r)

    assert.Equal(t, http.StatusOK, w.Code)
    assert.Equal(t, []byte("abcd"), w.Body.Bytes())
}

The test result is :

--- FAIL: TestGetRequest (0.00s)
    assertions.go:203: 
                        
    Error Trace:    main_test.go:27
        
    Error:      Not equal: []byte{0x61, 0x62, 0x63, 0x64} (expected)
                    != []byte(nil) (actual)
            
            Diff:
            --- Expected
            +++ Actual
            @@ -1,4 +1,2 @@
            -([]uint8) (len=4 cap=8) {
            - 00000000  61 62 63 64                                       |abcd|
            -}
            +([]uint8) <nil>
             
        
FAIL
FAIL    command-line-arguments  0.045s

The question is how do I fake the mux.Vars(r) for the unit tests? I've found some discussions here but the proposed solution no longer works. The proposed solution was :

func buildRequest(method string, url string, doctype uint32, docid uint32) *http.Request {
    req, _ := http.NewRequest(method, url, nil)
    req.ParseForm()
    var vars = map[string]string{
        "doctype": strconv.FormatUint(uint64(doctype), 10),
        "docid":   strconv.FormatUint(uint64(docid), 10),
    }
    context.DefaultContext.Set(req, mux.ContextKey(0), vars) // mux.ContextKey exported
    return req
}

This solution doesn't work since context.DefaultContext and mux.ContextKey no longer exist.

Another proposed solution would be to alter your code so that the request functions also accept a map[string]string as a third parameter. Other solutions include actually starting a server and building the request and sending it directly to the server. In my opinion this would defeat the purpose of unit testing, turning them essentially into functional tests.

Considering the fact the the linked thread is from 2013. Are there any other options?

EDIT

So I've read the gorilla/mux source code, and according to mux.go the function mux.Vars() is defined here like this :

// Vars returns the route variables for the current request, if any.
func Vars(r *http.Request) map[string]string {
    if rv := context.Get(r, varsKey); rv != nil {
        return rv.(map[string]string)
    }
    return nil
}

The value of varsKey is defined as iota here. So essentially, the key value is 0. I've written a small test app to check this : main.go

package main

import (
    "fmt"
    "net/http"
    
    "github.com/gorilla/mux"
    "github.com/gorilla/context"
)
    
func main() {
    r, _ := http.NewRequest("GET", "/test/abcd", nil)
    vars := map[string]string{
        "mystring": "abcd",
    }
    context.Set(r, 0, vars)
    what := Vars(r)
        
    for key, value := range what {
        fmt.Println("Key:", key, "Value:", value)
    }

    what2 := mux.Vars(r)
    fmt.Println(what2)
    
    for key, value := range what2 {
        fmt.Println("Key:", key, "Value:", value)
    }

}

func Vars(r *http.Request) map[string]string {
    if rv := context.Get(r, 0); rv != nil {
        return rv.(map[string]string)
    }
    return nil
}

Which when run, outputs :

Key: mystring Value: abcd
map[]
 

Which makes me wonder why the test doesn't work and why the direct call to mux.Vars doesn't work.

Jellaba

Answer 1

gorilla/mux provides the SetURLVars function for testing purposes, which you can use to inject your mock vars.

func TestGetRequest(t *testing.T) {
    r, _ := http.NewRequest("GET", "/test/abcd", nil)
    w := httptest.NewRecorder()

    //Hack to try to fake gorilla/mux vars
    vars := map[string]string{
        "mystring": "abcd",
    }

    // CHANGE THIS LINE!!!
    r = mux.SetURLVars(r, vars)
    
    GetRequest(w, r)

    assert.Equal(t, http.StatusOK, w.Code)
    assert.Equal(t, []byte("abcd"), w.Body.Bytes())
}

Bloodroot

This should be the accepted answer. Saved my behind! Thanks! – Comber

Yeah, this works really well and does not bog down the package with useless functions – Tender

What a great answer! – Ricer

The test specifies the variable contents twice -- one of which is a hack/fake. For most purposes, this seems clearly worse than just including the actual Gorilla Router in the test by configuring and calling ServeHTTP(). – Sands

Answer 2

Trouble is, even when you use 0 as value to set context values, it is not same value that mux.Vars() reads. mux.Vars() is using varsKey (as you already saw) which is of type contextKey and not int.

Sure, contextKey is defined as:

type contextKey int

which means that it has int as underlying object, but type plays part when comparing values in go, so int(0) != contextKey(0).

I do not see how you could trick gorilla mux or context into returning your values.

---

That being said, couple of ways to test this comes to mind (note that code below is untested, I have typed it directly here, so there might be some stupid errors):

1. As somebody suggested, run a server and send HTTP requests to it.

2. Instead of running server, just use gorilla mux Router in your tests. In this scenario, you would have one router that you pass to ListenAndServe, but you could also use that same router instance in tests and call ServeHTTP on it. Router would take care of setting context values and they would be available in your handlers.

   func Router() *mux.Router {
       r := mux.Router()
       r.HandleFunc("/employees/{1}", GetRequest)
       (...)
       return r 
   }
   

   somewhere in main function you would do something like this:

   http.Handle("/", Router())
   

   and in your tests you can do:

   func TestGetRequest(t *testing.T) {
       r := http.NewRequest("GET", "employees/1", nil)
       w := httptest.NewRecorder()
   
       Router().ServeHTTP(w, r)
       // assertions
   }
   

3. Wrap your handlers so that they accept URL parameters as third argument and wrapper should call mux.Vars() and pass URL parameters to handler.

   With this solution, your handlers would have signature:

   type VarsHandler func (w http.ResponseWriter, r *http.Request, vars map[string]string)
   

   and you would have to adapt calls to it to conform to http.Handler interface:

   func (vh VarsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
       vars := mux.Vars(r)
       vh(w, r, vars)
   }
   

   To register handler you would use:

   func GetRequest(w http.ResponseWriter, r *http.Request, vars map[string]string) {
       // process request using vars
   }
   
   mainRouter := mux.NewRouter().StrictSlash(true)
   mainRouter.HandleFunc("/test/{mystring}", VarsHandler(GetRequest)).Name("/test/{mystring}").Methods("GET")
   

Which one you use is matter of personal preference. Personally, I would probably go with option 2 or 3, with slight preference towards 3.

Terrijo

Thank you. I think I'll use option 3. However gorilla/mux should fix this to make unit test possible. – Heater

I got around this by inspecting the way gorilla/mux did testing in their own package here, and put my own assertion inside the handler where they were checking the value of vars: github.com/gorilla/mux/blob/master/context_native_test.go#L22 – Burner

Option 3 gives this error (type VarsHandler) as type func(http.ResponseWriter, *http.Request) in argument to router.HandleFunc – Kashmiri

I am using mux.SetURLVars() – Kashmiri

Answer 3

In golang, I have slightly different approach to testing.

I slightly rewrite your lib code:

package main

import (
        "fmt"
        "net/http"

        "github.com/gorilla/mux"
)

func main() {
        startServer()
}

func startServer() {
        mainRouter := mux.NewRouter().StrictSlash(true)
        mainRouter.HandleFunc("/test/{mystring}", GetRequest).Name("/test/{mystring}").Methods("GET")
        http.Handle("/", mainRouter)

        err := http.ListenAndServe(":8080", mainRouter)
        if err != nil {
                fmt.Println("Something is wrong : " + err.Error())
        }
}

func GetRequest(w http.ResponseWriter, r *http.Request) {
        vars := mux.Vars(r)
        myString := vars["mystring"]

        w.WriteHeader(http.StatusOK)
        w.Header().Set("Content-Type", "text/plain")
        w.Write([]byte(myString))
}

And here is test for it:

package main

import (
        "io/ioutil"
        "net/http"
        "testing"
        "time"

        "github.com/stretchr/testify/assert"
)

func TestGetRequest(t *testing.T) {
        go startServer()
        client := &http.Client{
                Timeout: 1 * time.Second,
        }

        r, _ := http.NewRequest("GET", "http://localhost:8080/test/abcd", nil)

        resp, err := client.Do(r)
        if err != nil {
                panic(err)
        }
        assert.Equal(t, http.StatusOK, resp.StatusCode)
        body, err := ioutil.ReadAll(resp.Body)
        if err != nil {
                panic(err)
        }
        assert.Equal(t, []byte("abcd"), body)
}

I think this is a better approach - you are really testing what you wrote since its very easy to start/stop listeners in go!

Softener

Isn't this a functional test though? I was trying to do a unit test, basically just call the function as you would any function with some values you create and check the result. – Heater

It is. But in my opinion, to test a network server, functional test is enough. In go, I do unit testing mostly for complicated logic. – Gargantuan

Ok. Thank you for the tip but that doesn't really answer the question. If I were to NOT use URL parameters with gorilla/mux the unit tests would work and essentially I could conducts tests in parallel for each endpoint handler function without having to go through an actual http server. If I were to only use functional/acceptance tests I would avoid the actual problem that gorilla/mux has and my tests would take longer. I am planning to use both functional and unit tests in my project. – Heater

Let me be clear, I can use a JSON in a POST message to achieve the same thing and the unit tests will work. But that would avoid the problem, not solve it. – Heater

Answer 4

I use the following helper function to invoke handlers from unit tests:

func InvokeHandler(handler http.Handler, routePath string,
    w http.ResponseWriter, r *http.Request) {

    // Add a new sub-path for each invocation since
    // we cannot (easily) remove old handler
    invokeCount++
    router := mux.NewRouter()
    http.Handle(fmt.Sprintf("/%d", invokeCount), router)

    router.Path(routePath).Handler(handler)

    // Modify the request to add "/%d" to the request-URL
    r.URL.RawPath = fmt.Sprintf("/%d%s", invokeCount, r.URL.RawPath)
    router.ServeHTTP(w, r)
}

Because there is no (easy) way to deregister HTTP handlers and multiple calls to http.Handle for the same route will fail. Therefore the function adds a new route (e.g. /1 or /2) to ensure the path is unique. This magic is necessary to use the function in multiple unit test in the same process.

To test your GetRequest-function:

func TestGetRequest(t *testing.T) {
    t.Parallel()

    r, _ := http.NewRequest("GET", "/test/abcd", nil)
    w := httptest.NewRecorder()

    InvokeHandler(http.HandlerFunc(GetRequest), "/test/{mystring}", w, r)

    assert.Equal(t, http.StatusOK, w.Code)
    assert.Equal(t, []byte("abcd"), w.Body.Bytes())
}

Stallion

Answer 5

The issue is you can't set vars.

var r *http.Request
var key, value string

// runtime panic, map not initialized
mux.Vars(r)[key] = value

The solution is to create a new router on each test.

// api/route.go

package api

import (
    "net/http"
    "github.com/gorilla/mux"
)

type Route struct {
    http.Handler
    Method string
    Path string
}

func (route *Route) Test(w http.ResponseWriter, r *http.Request) {
    m := mux.NewRouter()
    m.Handle(route.Path, route).Methods(route.Method)
    m.ServeHTTP(w, r)
}

In your handler file.

// api/employees/show.go

package employees

import (
    "github.com/gorilla/mux"
)

func Show(db *sql.DB) *api.Route {
    h := func(w http.ResponseWriter, r http.Request) {
        username := mux.Vars(r)["username"]
        // .. etc ..
    }
    return &api.Route{
        Method: "GET",
        Path: "/employees/{username}",

        // Maybe apply middleware too, who knows.
        Handler: http.HandlerFunc(h),
    }
}

In your tests.

// api/employees/show_test.go

package employees

import (
    "testing"
)

func TestShow(t *testing.T) {
    w := httptest.NewRecorder()
    r, err := http.NewRequest("GET", "/employees/ajcodez", nil)
    Show(db).Test(w, r)
}

You can use *api.Route wherever an http.Handler is needed.

Andri