How to get the real IP of a client in a pyramid server behind a nginx proxy

Question

I have a Pyramid application which uses request.environ['REMOTE_ADDR'] in some places.

The application is served by Python Paste on port 6543 and a nginx server listening on port 80 is forwarding requests to the Paste server.

The nginx configuration is inspired by the Pyramid cookbook:

server {

    listen   80; ## listen for ipv4
    listen   [::]:80 default ipv6only=on; ## listen for ipv6

    server_name  localhost;

    access_log  /var/log/nginx/localhost.access.log;

    location / {

        proxy_set_header        Host $host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;
        proxy_pass http://127.0.0.1:6543;

    }

In the Pyramid application the variable request.environ['REMOTE_ADDR'] is now always equal to 127.0.0.1. I see a few strategies to solve this problem but I don't know if there is a recommended way to do that.

Here is what I'm considering:

• add a NewRequest subscriber which replaces request.environ['REMOTE_ADDR'] if necessary:

  if 'HTTP_X_REAL_IP' in event.request.environ: event.request.environ['REMOTE_ADDR'] = event.request.environ['HTTP_X_REAL_IP']

• use a wsgi middleware to modify request.environ before hitting the Pyramid layer.

• something else

Which strategy do you use for deploying Pyramid applications ? What will happen if I have two nginx proxies ? (the first serving the LAN and a second one one a machine directly connected to the internet).

Waldgrave

Answer 1

If you use the paste.deploy.config.PrefixMiddleware in your WSGI pipeline via use = egg:PasteDeploy#prefix, it will automatically translate X-Forwarded-For into REMOTE_ADDR. It is also great for other properties of your reverse proxy, for example it will translate X-Forwarded-Proto into wsgi.url_scheme to ensure that if the user visits with https then generated URLs are also https.

http://pythonpaste.org/deploy/class-paste.deploy.config.PrefixMiddleware.html

Strained

I must have missed something because I first tried with PrefixMiddleware as suggested in the cookbook but it didn't work. I obiously need to try it again. – Waldgrave

ok it turns out that I forgot to insert the [pipeline:main] section. Everything works fine – Waldgrave

Answer 2

I use gevent server behind nginx and I use request.client_addr to get ip address of a client.

Selffulfillment

Answer 3

If you do not have a WSGI pipline, or do not wish to use paste, then add an event handler:

config.add_subscriber(reverseProxyProtocolCorrection,'pyramid.events.NewRequest')

The event handler can read like:

def reverseProxyProtocolCorrection(event):
    event.request.scheme = 'https' if event.request.headers['X-Forwarded-Proto']=='https' else 'http'
    event.request.remote_addr=parseXForward(event.request.headers['X-Forwarded-For'],event.request.remote_addr)

and make sure your proxy set the headers

Cruciate

Answer 4

Even if accepted answer (paste.deploy.config.PrefixMiddleware based) is mostly correct, here is a big problem inside PasteDeploy package. It replaces request.remote_addr with first IP in chain. In other side proxy servers usually append new IP to right side. I will use AWS loadbalancer as example.

Let's say that correct client global IP is 1.1.1.1. But client is adding X-Forwarded-For header with value 192.168.1.10. After passing loadbalancer actuall IP will be appended and resulted header will be looks like:

X-Forwarded-For 192.168.1.10,1.1.1.1

PasteDeploy is using first IP, so 192.168.1.10 will be added as client remote address. What may cause potential vulnerability if your logic is based on client IPs. Last added IP should be used behind loadbalancer to be sure that it is a real global IP of last hop.

Based on PasteDeploy I created own pyramid middleware with correct behavior.

the_middleware.py file stored in the project's root looks like:

class ALBFilterMiddleware:
    def __init__(self, app):
        self.app = app

    def __call__(self, environ, start_response):
        if 'HTTP_X_FORWARDED_FOR' in environ:
            # add LAST IP in chain
            environ['REMOTE_ADDR'] = [
                x.strip(' ') for x in environ.pop('HTTP_X_FORWARDED_FOR').split(',')
            ][-1]
        if 'HTTP_X_FORWARDED_SCHEME' in environ:
            environ['wsgi.url_scheme'] = environ.pop('HTTP_X_FORWARDED_SCHEME')
        elif 'HTTP_X_FORWARDED_PROTO' in environ:
            environ['wsgi.url_scheme'] = environ.pop('HTTP_X_FORWARDED_PROTO')
        return self.app(environ, start_response)


def alb_filter_factory(global_conf, **app_conf):
    def filter(app):
        return ALBFilterMiddleware(app)
    return filter

and application INI config should be like:

; this entrypoint will be used when you build app
; application = get_app('app.ini', 'the_entrypoint_name')
[pipeline:the_entrypoint_name]
pipeline = the_middleware_name your_app

[app:your_app]
use = call:app_file_with_main_func:main

[filter:the_middleware_name]
; the_middleware.py
use = call:the_middleware:alb_filter_factory

Francinafrancine

Answer 5

in nginx:

location / {
    proxy_pass http://yourapp;
        proxy_redirect     off;
        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $remote_addr;
        }

in php i do:

        if($_SERVER["HTTP_X_FORWARDED_FOR"]){
        $ip = $_SERVER["HTTP_X_FORWARDED_FOR"];
    }else{
        $ip = $_SERVER["REMOTE_ADDR"];
      } 

perhaps in python is similar, $ip is your real ip

Millais