wsgi nginx error: permission denied while connecting to upstream

Question

There seem to be many questions on StackOverflow about this but unfortunately nothing has worked for me.

I'm getting a 502 bad gateway on nginx, and the following on the logs: connect() to ...myproject.sock failed (13: Permission denied) while connecting to upstream

I'm running wsgi and nginx on ubuntu, and I've been following this guide from Digital Ocean. I apparently configured wsgi correctly since uwsgi -s myproject.sock --http 0.0.0.0:8000 --module app --callable app worked, but I keep getting the nginx permission denied error and I have no idea why:

After coming across this question and this other one, I changed the .ini file and added the chown-socket, chmod-socket, uid and gid parameters (also tried just setting the first two, either or, and a couple of different permission settings --and even the most permissive didn't work).

This one seemed promising, but I don't believe selinux is installed on my Ubuntu (running sudo apt-get remove selinux gives "Package 'selinux' is not installed, so not removed" and find / -name "selinux" doesn't show anything). Just in case, though, I tried what this post recommended as well. Uninstalling apparmor (sudo apt-get install apparmor) didn't work either.

Every time I make a change, I run sudo service nginx restart, but I only see the 502 Gateway Error (and the permission denied error when I read the logs).

This is is my nginx configuration file:

server {
    listen 80;
    server_name 104.131.110.156;

    location / {
        include uwsgi_params;
        uwsgi_pass unix:/home/user/myproject/web_server/myproject.sock;
    }
}

.conf file:

description "uWSGI server instance configured to serve myproject"

start on runlevel [2345]
stop on runlevel [!2345]

setuid user
setgid www-data

env PATH=/root/.virtualenvs/my-env/bin
chdir /home/user/myproject/web_server
exec uwsgi --ini /home/user/myproject/web_server/myproject.ini

.ini file:

[uwsgi]
module = wsgi

master = true
processes = 5

socket = /home/user/myproject/web_server/myproject.sock
chown-socket=www-data:www-data
chmod-socket = 664
uid = www-data
gid = www-data

vacuum = true
die-on-term = true

(If it helps, these are the specs of my Digital Ocean machine: Linux 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux)

Please let me know if there's anything I can do, and thank you very much.

Babur

Try placing the socket in /tmp. You could be getting access denied if the nginx user doesn't have permissions to list any of the directories in the socket path. It needs permissions for /home/user, /home/user/myproject, etc – Vaios

Thanks for answering! Unfortunately, though, same error: connect() to unix:/tmp/coaster.sock failed (13: Permission denied) while connecting to upstream – Babur

Pretty sure nginx runs as www-data on Ubuntu, but worth a check... What does /etc/nginx.conf say nginx is running as? Also, what happens if you change php-fpm to use a tcp socket, like 127.0.0.1:9000? – Vaios

Yes, I think it's running as www-data. I thought that there might be a problem with the permissions, so I changed the owners of the directories under user/ to the www-data user by calling sudo chown www-data:www-data DIRNAME but, even after restarting the server and nginx, I still get the same error. I'm also running nginx as root, since I call sudo service nginx restart while on my root account. And I'm not using PHP, but Flask + WSGI, so I don't know if that applies... – Babur

Derp. Sorry, been answering questions about PHP-FPM recently, lost the context. Regarding the tutorial, I notice that in the init file, you're using not using the same env PATH line that the tutorial is using. After you run service myproject start (ie when uwsgi is running), what's the output of ls -l /home/user/myproject/web_server/myproject.sock? – Vaios

Try to use TCP instead of unix socket, in your case it's --socket 127.0.0.1:3031 argument and uwsgi_pass 127.0.0.1:3031; in nginx configuration. It's working better anyway. – Chinkiang

Answer 1

I also followed that tutorial and ran into the same issue. After quite a bit of trial and error, the following steps allowed me to run uWSGI and nginx successfully:

My nginx.config file:

server {
    listen 80;
    server_name localhost;

    location / { try_files @yourapplication; }
    location @yourapplication; {
        include uwsgi_params;
        uwsgi_pass unix:/PATH_TO_PROJECT/PROJECT.sock;
    }
}

My .ini file wasn't working very well, so I decided to take advantage of uWSGI's extensive arguments that are available. Here's what I used:

uwsgi -s /PATH_TO_PROJECT/PROJECT.sock -w wsgi:app -H /PATH_TO_PROJECT/venv --http-processes=4 --chmod-socket=666 --master &

Where:

-s /PATH_TO_PROJECT/PROJECT.sock = the location of my .sock file

-w wsgi:app = the location of my wsgi.py file and app being the name of my Flask object

-H /PATH_TO_PROJECT/venv = the location of my virtual environment

--http-processes=4 = the number of http processes for uWSGI to create

--chmod-socket=666 = the permissions to set on the socket

--master = allow uWSGI to run with its master process manager

& = run uWSGI in the background

Unwieldy

And in production situations what happens? – Prophesy

The --chmod-socket=666 was what I needed! I even put chmod-socket=660 in my case. – Balaam

Where did you put this command which started with "uwsgi"? On the <service_name>.ini? – Marianomaribel

Answer 2

After following all the advice in this thread I was still getting permission errors. The finally missing piece was to correct the nginx user in the /etc/nginx/nginx.conf file:

# old: user  nginx;
user  www-data;

Captainship

FYI ubuntu trusty64 users, nginx is already configured to run as www-data. Hopefully, you read this and I saved you 30 seconds of checking. – Information

Its worth specifying that this param goes outside http, server, etc. (i.e near or at the top) – Papism

Answer 3

To summarize what others have said to solve permission denied error in nginx (which you can look into /var/log/nginx/error.log is usually due to the following:

1. you are writing .sock file at a place nginx does not have permission
2. SELinux is causing the problem

To solve 1: First, don't write .sock file at /tmp as suggested here server fault answer because different services see different /tmp in fedora. You can write at some place such as ~/myproject/mysocket.sock. The nginx user must have access to our application directory in order to access the socket file there. By default, CentOS locks down each user's home directory very restrictively, so we will add the nginx user to our user's group so that we can then open up the minimum permissions necessary to grant access.

You can add the nginx user to your user group with the following command. Substitute your own username for the user in the command:

sudo usermod -a -G $USER nginx

Now, we can give our user group execute permissions on our home directory. This will allow the Nginx process to enter and access content within:

chmod 710 /path/to/project/dir

If the permission denied error is still there: then the hack sudo setenforce 0 will do the trick.

Delcine

sudo setenforce 0 was indeed the hack. – Chamberlin

Answer 4

The path: unix:/PATH_TO_PROJECT/PROJECT.sock should be placed in /tmp this fixed my problem.

Puggree

will not solve problem always see (link)[#22273443 – Delcine

Answer 5

(13: Permission denied)

This indicates that Nginx was unable to connect to the uWSGI socket because of permissions problems. Usually, this happens when the socket is being created in a restricted environment or if the permissions were wrong. While the uWSGI process is able to create the socket file, Nginx is unable to access it.

This can happen if there are limited permissions at any point between the root directory (/) the socket file. We can see the permissions and ownership values of the socket file and each of its parent directories by passing the absolute path to our socket file to the namei command:

namei -nom /PATH_TO_YOUR_SOCKET_FILE/YOUR_SOCKET.sock

The output should be similar to this (Your case might have different folder name)

f: /run/uwsgi/firstsite.sock
 drwxr-xr-x root  root     /
 drwxr-xr-x root  root     run
 drwxr-xr-x sammy www-data uwsgi
 srw-rw---- sammy www-data firstsite.sock

The output displays the permissions of each of the directory components. By looking at the permissions (first column), owner (second column) and group owner (third column), we can figure out what type of access is allowed to the socket file.

In the above example, each of the directories leading up to the socket file have world read and execute permissions (the permissions column for the directories end with r-x instead of ---). The www-data group has group ownership over the socket itself. With these settings, Nginx process should be able to access the socket successfully.

If any of the directories leading up to the socket are not owned by the www-data group or do not have world read and execute permission, Nginx will not be able to access the socket. Usually, this means that the configuration files have a mistake.

So you fix this issue but giving all the upper folder the permission using this command:

chmod 755 directory_name

I know it is late, but to help others overcome the issue faster, I have posted this answer. Hope it helps, good luck.

Propagandize

Answer 6

This may happen if user www-data may not have the permission to create new socket in the given path, so use root user. Relpace user www-data to user root in nginx.conf;

ex: #nginx.conf
#user www-data;
user root;
worker_processes auto;
pid /var/run/nginx.pid;
.............

Zellers

That's brute force method which is rarely good idea. My fix for similar issue was to add this line to nginx.conf file: user www-data www-data; – Aurora

Answer 7

If you have tested all the permissions and it is still not working then maybe SELinux is enabled, this will cause the same behaviour.

Run getenforce and if the result is Enforcing then that will not help.

Quick fix is to disable it, setenforce 0 but a restart is required.

Rosenblast

Answer 8

2 things helped me

I had correct configuration in nginx I also was seeing /tmp/wsgi.sock with my two eyes in the folder but there was still permission denied or directory not exists:

1. In file /lib/systemd/system/nginx.service set PrivateTmp=false and restart nginx (do not forget systemctl daemon reload to refresh config)

2. Run command setenforce 0

Bonus material:

/usr/local/bin/uwsgi --chdir /home/biohazard/myproject -s /tmp/wsgi.sock -w api:app --chmod-socket=777 --master --thunder-lock --http-processes=2 where in api:app api stands for /home/biohazard/myproject/api.py

And

location = /api { rewrite ^ /api/; }
location /api { try_files $uri @api; }
location @api {
  include uwsgi_params;
  uwsgi_pass unix:/tmp/wsgi.sock;
}

Where nginx is serving my http://example.org/api endpoint only

Rector

I forgot how to do it then found my own answer. Thank you me – Rector

Answer 9

Check user field on the first line in nginx.conf file. By default it is www-data. Change the name to user root in nginx.conf file if you logged in as root.

Shakhty

Answer 10

I Am Also Getting Same Issue While Deploying Flask Using Nginx And Gunicorn. I Solved This Issue By putting .Sock file in /temp folder.

Quota

Answer 11

There are many things that can cause this particular error, in my case it was the ownership of my PROJECT.socket file that caused it.

Instead of: srwxr-xr-x 1 yourusername yourusername 0 Nov 4 22:32 PROJECT.sock

it should be: srwxr-xr-x 1 www-data www-data 0 Nov 4 22:32 PROJECT.sock

Just run sudo chown www-data:www-data PROJECT.sock and that's it.

Haywoodhayyim