Message "X-Accel-Mapping header missing" in Nginx error log

Question

I am running a Rails 3 site on Ubuntu 8.04 with Nginx 1.0.0 and Passenger 3.0.7.

In my Nginx error.log I started seeing the message X-Accel-Mapping header missing quite a lot. Googling lead me to the docs of Rack::Sendfile and to the Nginx docs.

Now, my app can be accessed through several domains and I am using send_file in my app to deliver some files specific to the domain they are requested from, e.g., if you come to domain1.com/favicon.ico I look up the favicon in at public/websites/domain1/favicon.ico. This works fine and I don't think I need/want to get Nginx involved and create some private area where I store those files, as the samples in the Rack::Sendfile docs suggest.

How can I get rid of the error message?

Algarroba

Hi @zoopzoop - did you ever figure this out? I'm having the same problem with my rails 3 app hosted on heroku (which uses ngnix). – Accustom

Same problem here as well. Passenger 3.0.7, nginx 1.0.0, Ubuntu. – Crape

No, I didn't find a solution yet. – Algarroba

Answer 1

this message means that Rack::Sendfile disabled X-Accel-Redirect for you, because you have missing configuration for it in nginx.conf...

I'm using Nginx + Passenger 3 + Rails 3.1.

Gathered information from this pages I've figured it out:

http://wiki.nginx.org/X-accel

http://greenlegos.wordpress.com/2011/09/12/sending-files-with-nginx-x-accel-redirect

http://code.google.com/p/substruct/source/browse/trunk/gems/rack-1.1.0/lib/rack/sendfile.rb?r=355

Serving Large Files Through Nginx via Rails 2.3 Using x-sendfile

I have controller which maps /download/1 requests to storage files which have their own directory structure, like this: storage/00/00/1, storage/01/0f/15 etc. So I need to pass this through Rails, but then I need to use send_file method which will use X-Accel-Redirect to send the final file to the browser through nginx directly.

Within the code I have this:

send_file(
  '/var/www/shared/storage/00/00/01', 
  :disposition => :inline, 
  :filename => @file.name # an absolute path to the file which you want to send
)

I replaced the filename for this example purposes

Now I had to add these lines to my nginx.conf:

server {
    # ... 

    passenger_set_cgi_param HTTP_X_ACCEL_MAPPING /var/www/shared/storage/=/storage/; 
    passenger_pass_header X-Accel-Redirect;

    location /storage {
      root /var/www/shared;
      internal;
    }

    # ...
}

The path /storage is not visible from outside world, it is internal only.

Rack::Sendfile gets the header X-Accel-Mapping, extracts the path from it and replaces /var/www/shared/storage with /storage.... Then it spits out the modified header:

X-Accel-Redirect: /storage/00/00/01

which is then processed by nginx.

I can see this works correctly as the file is downloaded 100x faster than before and no error is shown in the logs.

Hope this helps.

Vermillion

That sounds great but doesn't really help me since, as I mentioned in my question, I neither have nor want to have one folder with all the files... some are stored in a directory in my Rails app that is shared between deploys, some are being downloaded on the fly from somewhere else and stored in the tmp folder etc. I just want to use send_file without having to set up a separate folder that contains all the files. Possible? – Algarroba

Yep, in that case you have to create one internal location for every each of them... but I don't know whats the format for X-Accel-Mapping for it... Or, you could maybe add that internal location like "/=/"... – Vermillion

Answer 2

We used the similar technique as NoICE described, but i replaced the "hard-coded" directory containing all the files with the regular expression describing the folder containing the folders containing the files.

Sounds hard, yeah? Just take a look on these (/etc/nginx/sites-available/my.web.site):

location /assets/(.+-[a-z0-9]+\.\w+) {
    root /home/user/my.web.site/public/assets/$1;
    internal;
}

location /images/(.+)(\?.*)? {
    root /home/user/my.web.site/public/images/$1;
    internal;
}

This should be used with this check:

location / {
    # ...

    if (-f $request_filename) {
        expires max;
        break;
    }

    # ...
}

to prevent the statics from Rails processing.

Acherman

Answer 3

I did by this manual

https://mattbrictson.com/accelerated-rails-downloads

my server sends file path /private_upload/file/123/myfile.txt, the file is in /data/myapp-data/private_upload/file/123/myfile.txt

    # Allow NGINX to serve any file in /data/myapp-data/private_upload
    # via a special internal-only location.
    location /private_upload {
      internal;
      alias /data/myapp-data/private_upload;
    }

    # ---------- BACKEND ----------
    location @backend
    {
        limit_req zone=backend_req_limit_per_ip burst=20 nodelay;
        proxy_pass http://backend;
        proxy_set_header X-Sendfile-Type X-Accel-Redirect;
        proxy_set_header X-Accel-Mapping /=/; # this header is required, it does nothing
        include    /etc/nginx/templates/myapp_proxy.conf;
    }

Pagano