How do I upload multiple files using the Flask test_client?

Question

How can I use the Flask test_client to upload multiple files to one API endpoint?

I'm trying to use the Flask test_client to upload multiple files to a web service that accepts multiple files to combine them into one large file.

My controller looks like this:

@app.route("/combine/file", methods=["POST"])
@flask_login.login_required
def combine_files():

  user = flask_login.current_user

  combined_file_name = request.form.get("file_name")

  # Store file locally
  file_infos = []
  for file_data in request.files.getlist('file[]'):

    # Get the content of the file
    file_temp_path="/tmp/{}-request.csv".format(file_id)
    file_data.save(file_temp_path)

    # Create a namedtuple with information about the file
    FileInfo = namedtuple("FileInfo", ["id", "name", "path"])
    file_infos.append(
      FileInfo(
        id=file_id,
        name=file_data.filename,
        path=file_temp_path
      )
    )
    ...

My test code looks like this:

def test_combine_file(get_project_files):

project = get_project_files["project"]

r = web_client.post(
    "/combine/file",
    content_type='multipart/form-data',
    buffered=True,
    follow_redirects=True,
    data={
        "project_id": project.project_id,
        "file_name": "API Test Combined File",
        "file": [
            (open("data/CC-Th0-MolsPerCell.csv", "rb"), "CC-Th0-MolsPerCell.csv"),
            (open("data/CC-Th1-MolsPerCell.csv", "rb"), "CC-Th1-MolsPerCell.csv")
]})
response_data = json.loads(r.data)

assert "status" in response_data
assert response_data["status"] == "OK"

However, I can't get the test_client to actually upload both files. With more than one file specified, the file_data is empty when the API code loops. I have tried my own ImmutableDict with two "file" entries, a list of file tuples, a tuple of file tuples, anything I could think of.

What is the API to specify multiple files for upload in the Flask test_client? I can't find this anywhere on the web! :(

Monas

I'm reading the source for the framework Flask is on top of, and I still can't figure out what the hell it expects. – Monas

Answer 1

The test client takes a list of file objects (as returned by open()), so this is the testing utility I use:

def multi_file_upload(test_client, src_file_paths, dest_folder):
    files = []
    try:
        files = [open(fpath, 'rb') for fpath in src_file_paths]
        return test_client.post('/api/upload/', data={
            'files': files,
            'dest': dest_folder
        })
    finally:
        for fp in files:
            fp.close()

I think if you lose your tuples (but keeping the open()s) then your code might work.

Billow

This is the correct solution, just a little note on how to retrieve those files on the server side. The thing is, Flask wraps them in werkzeug.ImmutableMultiDict class, which is a multidict, so it can store multiple entries for each key. In this case, retrieving files would be done via request.files.getlist('files'). Simply trying to access request.files['files'] will give you only the first file, which really confused me. – Tomkins

@SergeMosin Above I used for file_data in request.files.getlist('file[]'): is that right? – Monas

@Monas not sure about .getlist('file[]') part, but If it worked, means it's right. – Tomkins

Answer 2

You should just send data object with your files named as you want:

test_client.post('/api/upload', 
                 data={'title': 'upload sample', 
                       'file1': (io.BytesIO(b'get something'), 'file1'), 
                       'file2': (io.BytesIO(b'forthright'), 'file2')},  
                 content_type='multipart/form-data')

Lillith

Note that the Flask test_client takes file tuples with the format (file, filename, …) whereas requests takes file tuples with the format (filename, file, …). – Characterization

Answer 3

Another way of doing this- if you want to explicitly name your file uploads here (my use case was for two CSVs, but could be anything) with test_client is like this:

   resp = test_client.post(
                           '/data_upload_api', # flask route
                           file_upload_one=[open(FILE_PATH, 'rb')],
                           file_upload_two=[open(FILE_PATH_2, 'rb')]
                           )

Using this syntax, these files would be accessible as:

request.files['file_upload_one'] # etc.

Saponaceous

I didn't get this to work, I got an error TypeError: __init__() got an unexpected keyword argument 'file_upload_one' – Derinna

Hey- the code above assumes that the endpoint had two kwargs called 'file_upload_one' and 'file_upload_two' - change these to be whatever you called the incoming files in your route – Saponaceous