Accessing POST Data from WSGI

Question

I can't seem to figure out how to access POST data using WSGI. I tried the example on the wsgi.org website and it didn't work. I'm using Python 3.0 right now. Please don't recommend a WSGI framework as that is not what I'm looking for.

I would like to figure out how to get it into a fieldstorage object.

Michaelis

FWIW, at this point in time there is still no WSGI specification for Python 3.0, so anything you do will possibly be wasted effort as any final specification update may not be compatible with anyones attempts to implement what it may say for Python 3.0. For WSGI applications you are better off staying with Python 2.X. – Nonparous

@GrahamDumpleton Not anymore: python.org/dev/peps/pep-3333 (Let's not mislead people who read this a bit later like me - saving their time too) – Maffa

@JermoeJ - He wrote the comment in 2009, and you are replying to it in 2013; don't think he was trying to mislead anyone. :) – Calves

Answer 1

Assuming you are trying to get just the POST data into a FieldStorage object:

# env is the environment handed to you by the WSGI server.
# I am removing the query string from the env before passing it to the
# FieldStorage so we only have POST data in there.
post_env = env.copy()
post_env['QUERY_STRING'] = ''
post = cgi.FieldStorage(
    fp=env['wsgi.input'],
    environ=post_env,
    keep_blank_values=True
)

Herein

This doesn't work in Python 3.0 - it has a problem with the wsgi.input returning bytes instead of strings. :( I need a way of doing this in Python 3.0... – Michaelis

What WSGI handler are you using? If I use the built-in CGIHandler it works just fine for me. I have a file "post.cgi" on my local server with the contents at pastebin.com/f40849562 running just fine. – Herein

What io class is the wsgi.input? If it is a BufferedIOBase then you should be able to wrap it in a TextIOWrapper so that the cgi.FieldStorage can use it. – Herein

@Mike, I thought of that too, but that would make it not function properly in the long run as post data can be binary (eg, files). – Michaelis

@Evan, Maybe I'm crazy, but you could wrap the input in a TextIOWrapper and extend the FieldStorage and override the make_file method to return your own wrapper around a file which encodes back to binary data as it writes... If this can't be done an easier way. Which WSGI handler are you using? – Herein

@Mike, I'm just using wsgiref, which on its own took a lot of modification to get working in 3.0. – Michaelis

@Evan: Any particular handler, or the CGIHandler? Cause that one is working just fine for me with 3.0. Maybe I'm not pushing it very hard? – Herein

@Mike, I'm using WSGIServer. I haven't tried it with CGIHandler, but I'd prefer not to ever have to use that since it sort of breaks the point of WSGI in my opinion. – Michaelis

@Evan: Well that is really broken in 3.0, isn't it? I'll play around with it in the near future (cause I am interested in moving to 3.0 for my apps) and post results here. – Herein

Answer 2

body= ''  # b'' for consistency on Python 3.0
try:
    length= int(environ.get('CONTENT_LENGTH', '0'))
except ValueError:
    length= 0
if length!=0:
    body= environ['wsgi.input'].read(length)

Note that WSGI is not yet fully-specified for Python 3.0, and much of the popular WSGI infrastructure has not been converted (or has been 2to3d, but not properly tested). (Even wsgiref.simple_server won't run.) You're in for a rough time doing WSGI on 3.0 today.

Vasty

Yeah I had issues getting wsgiref to work. I ended up implementing the patch. – Michaelis

Answer 3

This worked for me (in Python 3.0):

import urllib.parse

post_input = urllib.parse.parse_qs(environ['wsgi.input'].readline().decode(),True)

Reviviscence

Answer 4

I had the same issue and I invested some time researching a solution.
the complete answer with details and ressources (since the one accepted here didnt work for me on python3, many errors to correct in env library etc):

# the code below is taken from and explained officially here:
# https://wsgi.readthedocs.io/en/latest/specifications/handling_post_forms.html
import cgi
def is_post_request(environ):
    if environ['REQUEST_METHOD'].upper() != 'POST':
        return False
    content_type = environ.get('CONTENT_TYPE', 'application/x-www-form-urlencoded')
    return (content_type.startswith('application/x-www-form-urlencoded' or content_type.startswith('multipart/form-data')))
def get_post_form(environ):
    assert is_post_request(environ)
    input = environ['wsgi.input']
    post_form = environ.get('wsgi.post_form')
    if (post_form is not None
        and post_form[0] is input):
        return post_form[2]
    # This must be done to avoid a bug in cgi.FieldStorage
    environ.setdefault('QUERY_STRING', '')
    fs = cgi.FieldStorage(fp=input,
                          environ=environ,
                          keep_blank_values=1)
    new_input = InputProcessed()
    post_form = (new_input, input, fs)
    environ['wsgi.post_form'] = post_form
    environ['wsgi.input'] = new_input
    return fs
class InputProcessed(object):
    def read(self, *args):
        raise EOFError('The wsgi.input stream has already been consumed')
    readline = readlines = __iter__ = read

# the basic and expected application function for wsgi
# get_post_form(environ) returns a FieldStorage object
# to access the values use the method .getvalue('the_key_name')
# this is explained officially here:
# https://docs.python.org/3/library/cgi.html
# if you don't know what are the keys, use .keys() method and loop through them
def application(environ, start_response):
    start_response('200 OK', [('Content-type', 'text/plain')])
    user = get_post_form(environ).getvalue('user')
    password = get_post_form(environ).getvalue('password')
    output = 'user is: '+user+' and password is: '+password
    return [output.encode()]

Lucan

Answer 5

For cgi is deprecated and slated for removal in Python 3.13, the accepted answer is getting old.

Like @Jack's answer, I prefer to use urllib.parse.parse_qsl (or urllib.parse.parse_qs when some values are consisted of a series of values. cf. How to stop Python parse_qs from parsing single values into lists?).

import urllib.parse

def application(env, start_response):
    params = dict(urllib.parse.parse_qsl(
        env['wsgi.input'].read(int(env['CONTENT_LENGTH'])).decode())
    )

    ...

Faline

Answer 6

I would suggest you look at how some frameworks do it for an example. (I am not recommending any single one, just using them as an example.)

Here is the code from Werkzeug:

http://dev.pocoo.org/projects/werkzeug/browser/werkzeug/wrappers.py#L150

which calls

http://dev.pocoo.org/projects/werkzeug/browser/werkzeug/utils.py#L1420

It's a bit complicated to summarize here, so I won't.

Sadoc

Still doesn't work in Python 3.0, and that's what I'm looking for. Thanks anyways, though. – Michaelis

@FireCrow Looking at how frameworks do seems to be a good idea though. This isn't really advising a framework way. – Maffa

Links are dead, though. – Pointdevice