How can I receive an uploaded file using a Golang net/http server?

Question

I'm playing around with Mux and net/http. Lately, I'm trying to get a simple server with one endpoint to accept a file upload.

Here's the code I've got so far:

server.go

package main

import (
    "fmt"
    "github.com/gorilla/mux"
    "log"
    "net/http"
)

func main() {
    router := mux.NewRouter()
    router.
        Path("/upload").
        Methods("POST").
        HandlerFunc(UploadCsv)
    fmt.Println("Starting")
    log.Fatal(http.ListenAndServe(":8080", router))
}

endpoint.go

package main

import (
    "fmt"
    "net/http"
)

func UploadFile(w http.ResponseWriter, r *http.Request) {
    err := r.ParseMultipartForm(5 * 1024 * 1024)
    if err != nil {
        panic(err)
    }

    fmt.Println(r.FormValue("fileupload"))
}

I think I've narrowed the issue down to actually retrieving the body from the request inside UploadFile. When I run this cURL command:

curl http://localhost:8080/upload -F "[email protected]" -vvv

I get an empty response (as expected; I'm not printing to the ResponseWriter), but I just get a new (empty) line printed at the prompt where I'm running the server, instead of the request body.

I'm sending the file as multipart (AFAIK, implied by using -F rather than -d in cURL), and cURL's verbose output is showing 502 bytes sent:

$ curl http://localhost:8080/upload -F "[email protected]" -vvv
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> POST /upload HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.51.0
> Accept: */*
> Content-Length: 520
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------b578878d86779dc5
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Date: Fri, 18 Nov 2016 19:01:50 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
< 
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

What's the proper way to receive files uploaded as multipart form data using a net/http server in Go?

Drisko

maybe you are looking for golang.org/pkg/net/http/#Request.FormFile ? – Muriah

Also this is what I have used to parse multiple files: golang.org/pkg/mime/multipart/#Form – Crooked

@mh-cbon that's it! I get a []byte, which prints properly, which means I've got something to work with. Thanks! – Drisko

I have an example here github.com/yanpozka/go-httprouter-upfiles-token/blob/master/… – Prebendary

@mh-cbon please add your answer as an answer so it can be marked as accepted and this question can be filtered out when people are looking for questions to answer. – Hindrance

can this be done with a file and a struct in the same request? – Demb

Answer 1

Here's a quick example

func ReceiveFile(w http.ResponseWriter, r *http.Request) {
    r.ParseMultipartForm(32 << 20) // limit your max input length!
    var buf bytes.Buffer
    // in your case file would be fileupload
    file, header, err := r.FormFile("file")
    if err != nil {
        panic(err)
    }
    defer file.Close()
    name := strings.Split(header.Filename, ".")
    fmt.Printf("File name %s\n", name[0])
    // Copy the file data to my buffer
    io.Copy(&buf, file)
    // do something with the contents...
    // I normally have a struct defined and unmarshal into a struct, but this will
    // work as an example
    contents := buf.String()
    fmt.Println(contents)
    // I reset the buffer in case I want to use it again
    // reduces memory allocations in more intense projects
    buf.Reset()
    // do something else
    // etc write header
    return
}

Peel

Note that to avoid memory consumption you might write ondisk directly at io.Copy(&Buf, file) where Buf is to replace with a File handler. – Muriah

True that, large file no es bueno. – Peel

i also put +1 on below answer from keith wachira for pointing it out. – Muriah

I checked the error that's return on defer file.Close() and it seems the file is unable to be closed -- is that actually needed here? – Uppermost

Sorry I'm unfamiliar with bitwise operators, but does 32 << 20 have a special meaning? Is there a reason you use it in this example? – Fedora

@Fedora in this use case it's shorthand to expand and convert 32 into 32MB as the method takes a read limit in bytes. see this article demonstrating it further. medium.com/@owlwalks/… – Peel

Answer 2

You should use FormFile instead of FormValue:

file, fileHeader, err := r.FormFile("fileupload")
defer file.Close()

// copy example
f, err := os.OpenFile("./downloaded", os.O_WRONLY|os.O_CREATE, 0666)
defer f.Close()
io.Copy(f, file)

Bristol

what if revieving a serialized abject and a file in the same request? – Demb

@pltvs if i pass the content-length explicitly through file upload in postman. handler is throwing an error. with the above code. looking for suggestions. – Vignette

do you mean header instead of handler? – Minorite

"./test/"+handler.Filename – in general it's a very bad idea to write to files with paths controllable by external input. Don't do this. – Damp

How about uploading multiple files at single upload? – Planet

Answer 3

Here a function i wrote to help me in uploading my files.You can check the full version here . How to upload files in golang

package helpers

import (
    "io"
    "net/http"
    "os"
)

// This function returns the filename(to save in database) of the saved file
// or an error if it occurs
func FileUpload(r *http.Request) (string, error) {
    // ParseMultipartForm parses a request body as multipart/form-data
    r.ParseMultipartForm(32 << 20)

    file, handler, err := r.FormFile("file") // Retrieve the file from form data

    if err != nil {
        return "", err
    }
    defer file.Close()                       // Close the file when we finish

    // This is path which we want to store the file
    f, err := os.OpenFile("/pathToStoreFile/"+handler.Filename, os.O_WRONLY|os.O_CREATE, 0666)

    if err != nil {
        return "", err
    }

    // Copy the file to the destination path
    io.Copy(f, file)

    return handler.Filename, nil
}

Morningglory

The indentation for your attached code is inconsistent, please fix it. – Sinuosity

How about uploading multiple files at single upload? – Planet