Print raw HTTP request in Flask or WSGI

Question

I am debugging a microcontroller I've built which is writing raw HTTP requests line by line. I am using Flask for my backend and I would like to see the entire request as it appears in this format:

GET / HTTP/1.1
Content-length: 123
User-agent: blah
...

I know Flask is based on WSGI. Is there anyway to get this to work with Flask?

Chere

Answer 1

With flask you have access to the request object which contains all the HTTP details:

from flask import request

@app.route('/')
def index():
    print(request.headers)

Fee

This only prints the header, not all the HTTP details as specified. Using other properties of request also doesn't help (e.g. data, form, values, etc). To print the HTTP request in its entirety, wrapping around WSGI suggested in @martinpieters' answer is the way to go. – Lickerish

This is not the answer. – Lewie

Answer 2

Yes, Flask is a WSGI application, so it is trivial to wrap your app in an extra layer that logs the request:

import pprint

class LoggingMiddleware(object):
    def __init__(self, app):
        self._app = app

    def __call__(self, env, resp):
        errorlog = env['wsgi.errors']
        pprint.pprint(('REQUEST', env), stream=errorlog)

        def log_response(status, headers, *args):
            pprint.pprint(('RESPONSE', status, headers), stream=errorlog)
            return resp(status, headers, *args)

        return self._app(env, log_response)

This defines a piece of middleware to wrap your Flask application in. The advantage is that it operates entirely independent of Flask, giving you unfiltered insight into what goes in and what comes out.

How you apply the middleware depends on the exact WSGI server you are using; see your WSGI server documentation.

When running Flask with the built-in server (app.run()), do:

if __name__ == '__main__':
    app.wsgi_app = LoggingMiddleware(app.wsgi_app)
    app.run()

The little app.wsgi_app wrapping dance places the LoggingMiddleware around the Flask WSGI application.

The output goes to the wsgi.error stream; where that ends up again depends on your WSGI server; mod_wsgi puts this in the Apache error log for your site, the bundled Flask server prints this to stderr.

Clemmie

How can I get the content of the request for this solution as well? – Odle

@ArthurColombiniGusmão: are you sure you want to do that? The body is potentially very large. You also have to work with a stream, which is like a file that can only be read once, and then have to provide a new stream to the next WSGI component to replace it. env['wsgi.input'] is the stream, you could perhaps copy the data to a tempfile.SpooledTemporaryFile() object, do something with that file, then seek back to the start and set env['input.stream'] to point to it. – Clemmie

Thanks @Martijn Pieters! It may be not the best option then. My goal was to create a canonical snippet to quickly wrap Flask applications with for easy debugging (in controlled environments). Sometimes it seems to be easier to debug code that interfaces with REST APIs by having a middleware like this, but possibly I'm missing on existing solutions. – Odle

Answer 3

suppose if you want complete details,

There is an another way

@app.route('/')
def index():
    print request.__dict__
    #this prints all variables in `dict` format including `headers`

Perennial

That's not gonna work. headers isn't even a key; there is environ key which contains a mix of a different things including headers, but certainly not the full original request. – Suu

actually this does seem to work. when i do pprint.pformat(request.__dict__, depth=5)) after importing pprint I see environ as the first key printed out for me. – Marguerita

Answer 4

Why not?

from flask import Flask, request

app = Flask(__name__)

@app.before_request
def log_request():
    app.logger.debug("Request Headers %s", request.headers)
    return None

# The remaining application code.

I've used the headers but you can use the same aproach to print any request attribute. The docs are here: http://flask.pocoo.org/docs/0.12/api/#flask.Request.

Also you need to setup FLASK_DEBUG=1 to Flask.logger.debug to work, what is nice since you can disable it in production.

Regards,

Uhland

Answer 5

you probably want something like

print(request.headers)
print(request.cookies)
print(request.data)
print(request.args)
print(request.form)
print(request.endpoint)
print(request.method)
print(request.remote_addr)

Generalissimo

I like this answer because it uses only flask objects. Very easy to understand and use. – Krasnodar

print(request.body) and adding try...except may help lot. – Cavanagh

Answer 6

This doesn't use flask but it is fairly simple to setup a socket echo server.

import socket

host = ''
port = 8888
backlog = 5
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(backlog)
while 1:
    client, address = s.accept()
    data = client.recv(size)
    if data:
        client.send(data)
    client.close()

Jeth

One caveat: if client sends "Expect: 110-continue" header, one will not see complete HTTP communication... You should send '100 Continue' back to client and read again body of the request. – Embroidery

Answer 7

if you're using a logger i found this useful

https://docs.python.org/3/library/pprint.html#pprint.pformat

from pprint import pformat
import requests
log.debug(pformat(request.headers))

Caseworm

Thanks. I tried this and didn't see the difference with repr(request.headers). Can you elaborate? – Aylmer