Really logging the POST request body (instead of "-") with nginx

Question

I'm trying to log POST body, and add $request_body to the log_format in http clause, but the access_log command just prints "-" as the body after I send POST request using:

curl -d name=xxxx myip/my_location

My log_format (in http clause):

log_format client '$remote_addr - $remote_user $request_time $upstream_response_time '
                  '[$time_local] "$request" $status $body_bytes_sent $request_body "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

My location definition(in server clause):

location = /c.gif {  
  empty_gif;  
  access_log logs/uaa_access.log client;  
}

How can I print the actual POST data from curl?

Rhineland

Sounds like these guys found a solution: #4939882 – Gradey

Answer 1

Nginx doesn't parse the client request body unless it really needs to, so it usually does not fill the $request_body variable.

The exceptions are when:

• it sends the request to a proxy,
• or a fastcgi server.

So you really need to either add the proxy_pass or fastcgi_pass directives to your block.

The easiest way is to send it to Nginx itself as a proxied server, for example with this configuration:

location = /c.gif {  
    access_log logs/uaa_access.log client;
    # add the proper port or IP address if Nginx is not on 127.0.0.1:80
    proxy_pass http://127.0.0.1/post_gif; 
}
location = /post_gif {
    # turn off logging here to avoid double logging
    access_log off;
    empty_gif;  
}

If you only expect to receive some key-pair values, it might be a good idea to limit the request body size:

client_max_body_size 1k;
client_body_buffer_size 1k;
client_body_in_single_buffer on;

I also received "405 Not Allowed" errors when testing using empty_gif; and curl (it was ok from the browser), I switched it to return 200; to properly test with curl.

Lachish

I tried using the same, but in my nginx logs I still find request_body to be empty . – Habakkuk

I want to log the "response content" which is a json, to the nginx log. example: http://<some_ip>/prod/get_sources gives me JSON containing list of sources. How can I log this into the nginx log. – Habakkuk

No, this method can't be used to log the output of your application, just the input it receives, like a REST request using PUT/POST as json values, even form encoded strings. You may forward data from your application to nginx using error log, though. – Eparchy

sorry but I am new to this, can you tell me more about how can I forward the data to nginx. – Habakkuk

Answer 2

For anyone still facing this problem, check your client_body_buffer_size value.

If the req body size is bigger than client_body_buffer_size then nginx will replace it with -.

Teleutospore

Answer 3

Wow I had fun making this work - I simply wanted a way to catch and log SSL posts for a special testing situation. Anyhow, here's how I did it.

Of course the log format needs $request_body somewhere.

In the http (non SSL) portion of the config, I forward to myself - to force the logging. Use whatever paths are good for your purposes.

    listen       8081;
    server_name  localhost;

    location / {
        root   html;
        index  index.html index.htm;
        proxy_pass http://127.0.0.1:8081/kluge; 

    }
    location /kluge {
        root   html;
        index  index.html index.htm;
        access_log off; 

    }

And in the SSL part (which is where I was doing my testing primarily), the same, but forwarding to the http server:

  location / {
        root   html;
        index  index.html index.htm;
        # following required to get things to log... 
        proxy_pass http://127.0.0.1:8081/kluge; 
    }

Works nicely and minimally invasive to the config. Of course I could make it less invasive by not using '/' path but some path unique to my testing requirements.

Thanks to previous posts! I learned a lot about nginx thru this.

Evanescent