web2py upload with original filename

Question

I want to upload file with SQL.factory() I would just like to maintain the original filename my code currently is

form = SQLFORM.factory(
    Field('file_name', requires=IS_NOT_EMPTY()),
    Field('file', 'upload',uploadfolder=upload_folder))
if form.accepts(request.vars, session):  #.process().accepted:
    response.flash = u'File uploaded'
    session.your_name = form.vars.file_name
    session.filename = request.vars.file 
elif form.errors:
    response.flash = 'form has errors'
return dict(form=form)

I guess session.filename = request.vars.file is where you set file name. Why do i get the autogenerated file name no_data.smth.23u8o8274823zu4i2.smth

Thank you

Cypriot

Answer 1

First, request.vars.file is a Python cgi.FieldStorage object, so session.filename = request.vars.file should result in an error. request.vars.file.file is the actual file object, and request.vars.file.filename is the original name of the uploaded file.

When you upload a file via an upload field, web2py automatically generates a new name of the form 'table_name.field_name.random_id.b16encoded_original_filename.extension'. This is done to prevent directory traversal attacks and to enable the download mechanism (which needs to know the table and field name). In the case of SQLFORM.factory, there is no database table name, so it defaults to a table name of 'no_table'.

The code you have shown should not actually generate a filename like 'no_data.smth.23u8o8274823zu4i2.smth'. That filename implies you have explicitly told SQLFORM.factory to use a table name of 'no_data' (via its table_name argument) and that the upload field name is 'smth'. (The code above would generate a filename starting with 'no_table.file'.)

Note, web2py automatically obtains the original name of the uploaded file and encodes it (using b16encode) into the new filename (it then decodes the original filename when the built-in download mechanism is used). The original filename is also available in form.vars.file.filename. So, you don't necessarily need the user to enter a filename at all. However, if you want to enable the user to enter a filename that may differ from the actual filename and then use the user-entered filename, you can add the following before the form creation:

if 'file' in request.vars and request.vars.file_name:
    request.vars.file.filename = request.vars.file_name

That will re-assign the filename of the uploaded file to the value entered by the user, and web2py will then encode that user-entered filename into the new filename. Note, however, that web2py relies on the filename extension to set the HTTP headers appropriately upon download, so you may want to add some logic to obtain the original filename extension in case the user fails to enter it.

Menu

with request.vars.name_of_file.filename i get the original file name, but how to rename the uploaded. should i do with os.rename? I am uploading different zip files, so they need to be name_of_file.zip THANK YOU – Cypriot

You could also skip the form.accepts and handle the file saving yourself. Don't do this with user uploaded files, though, as you will be open to directory traversal attacks. – Menu

how to set no file encoding by web2py. The thing is i want the uploaded file to be stored in one folder with original file name, cos i have another script then to process it and file name is important for file processing?? – Cypriot

As I said, then don't use form.accepts. The file object will be in request.vars.file.file -- just use regular Python code to save that wherever and however you like. – Menu

Answer 2

If you simply rename the file, this will break the download mechanism. Moreover, sometimes you might want to save the file under a different name than the original. Let's assume you have the following model:

db.define_table("files",
Field("name", unique=True),
Field("file", "upload"))

You need to extend the upload field with customised store and retrieve functions:

Field("file", "upload", custom_store=store_file, custom_retrieve=retrieve_file)

The functions are simply writing/reading a file from a fixed upload directory:

import os
import shutil

def store_file(file, filename=None, path=None):
    path = "applications/app_name/uploads"
    if not os.path.exists(path):
         os.makedirs(path)
    pathfilename = os.path.join(path, filename)
    dest_file = open(pathfilename, 'wb')
    try:
            shutil.copyfileobj(file, dest_file)
    finally:
            dest_file.close()
    return filename

def retrieve_file(filename, path=None):
    path = "applications/app_name/uploads"
    return (filename, open(os.path.join(path, filename), 'rb'))

Now in the controller you need to modify the form.vars before the database insert/update is done and set the file name. If you want to keep the original name of the uploaded file, this is not necessary.

def validate(form):
    # set the uploaded file name equal to a name given in the form
    if form.vars.file is not None:
        form.vars.file.filename = form.vars.name

You also need to define a function to download the file as the build in response.download won't work:

import contenttype as c

def download():
    if not request.args:
        raise HTTP(404)
    name = request.args[-1]
    field = db["files"]["file"]
    try:
        (filename, file) = field.retrieve(name)
    except IOError:
        raise HTTP(404)
    response.headers["Content-Type"] = c.contenttype(name)
    response.headers["Content-Disposition"] = "attachment; filename=%s" % name
    stream = response.stream(file, chunk_size=64*1024, request=request)
    raise HTTP(200, stream, **response.headers)

To connect the dots, you need to build the form. In the example below I'm using the new grid mechanism which is way better than the old school forms (but not yet documented in the book).

upload = lambda filename: URL("download", args=[filename])

def index():
    grid = SQLFORM.grid(db.files, onvalidation=validate, upload=upload)
    return {"grid":grid}

If you don't want all the fanciness of the grid, the equivalent controller code is:

def index():
    if len(request.args):
        form=SQLFORM(db.files, request.args[0], upload=URL("download"))
    else:
        form=SQLFORM(db.files, upload=URL("download"))

    if form.process(onvalidation=validate).accepted:
        response.flash = "files updated"

    return {"form":form}

Clubbable

Hi I implemented the above code which is working fine, but "autodelete=True" is not working. Though the row gets deleted from the database...the physical file remains. – Afghanistan

Answer 3

so i did it :) here is my code

import os
upload_folder ='C:\\Python27\\web2py'
sl = "\\"
path = upload_folder + sl

def display_form():

     form = SQLFORM.factory(
        Field('file_name', requires=IS_NOT_EMPTY()),
        Field('file', 'upload',uploadfolder=upload_folder))


     if form.accepts(request.vars, session):  #.process().accepted:
        session.file_name= form.vars.file_name
        coded_name = form.vars.file 
        orig_name = request.vars.file.filename
        os.rename(path + coded_name, path + orig_name)
        response.flash = u'datoteka naložena'

    elif form.errors:
        response.flash = 'form has errors'
    return dict(form=form)

I know it is probably not the best solution but since it works, I like :)

thank you Anthony

Cypriot