How to deploy Django/React/Webpack app on Digital Ocean through Passenger/Nginx

Question

I'm trying to deploy a web app built with Django/Redux/React/Webpack on a Digital Ocean droplet. I'm using Phusion Passenger and Nginx on the deployment server.

I used create-react-app to build a Django app which has a frontend that uses React/Redux, and a backend api that uses django-rest-framework. I built the frontend using npm run build.

The Django app is configured to look in the frontend/build folder for its files and everything works as expected, including authentication. It's based on this tutorial: http://v1k45.com/blog/modern-django-part-1-setting-up-django-and-react/

In settings.py:

ALLOWED_HOSTS = ['*']

TEMPLATES = [
... 
       'DIRS': [
            os.path.join(BASE_DIR, 'frontend/build'),
        ],
...
]

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'frontend/build/static'),
]

On my development machine, I activate a Python 3.6 virtual environment and run ./manage.py runserver, and the app is displayed at localhost:3000.

On the deployment server, I've cloned the files into a folder in var/www/ and built the frontend.

I've set up Passenger according to the docs with a file passenger_wsgi.py:

import myapp.wsgi
application = myapp.wsgi.application

And the wsgi.py file is in the djangoapp folder below:

import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myapp.settings')
application = get_wsgi_application()

The Passenger docs only cover a single-part app:

https://www.phusionpassenger.com/library/walkthroughs/start/python.html https://www.phusionpassenger.com/library/walkthroughs/deploy/python/digital_ocean/nginx/oss/xenial/deploy_app.html https://www.phusionpassenger.com/library/deploy/wsgi_spec.html

I've tried cloning the tutorial part 1 code directly onto my server and following the instructions to run it. I got this to work on the server by adding "proxy": "http://localhost:8000" to frontend/package.json. If I run the Django server with ./manage.py runserver --settings=ponynote.production_settings xxx.x.x.x:8000 then the app is correctly served up at myserver:8000. However Passenger is still not serving up the right files.

I have changed wsgi.py to say this:

import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.production_settings")
application = get_wsgi_application()

The page served by Passenger at URL root now appears to have the right links to the js files such as "text/javascript" src="/static/bundles/js/main.a416835a.js, but the links don't work: the expected js is not present. Passenger is failing to serve the js files from static/bundles/js, even though the Django server can find them.

Very grateful for any help or ideas.

Vernacularize

Typically the front end files are built (in create-react app this is done by running npm run build) and then included in Django via the staticfiles workflow. I went through this, and found the django-webpack-loader config a little fiddly - you need to make sure that once you have built your files, that django is looking in the right place for the manifest.json file and the static files. – Heintz

Django is looking in the right place on the development server, so wouldn't that mean that it's in the right place on the production server also? The locations in settings.py are relative and should be equally correct in production. – Vernacularize

I posted a fuller explanation as an answer - it's difficult to troubleshoot exactly what causes the React app not to render, but the answer below should give an outline as to why the tutorial you're following recommends doing things the way they do. – Heintz

Answer 1

The key missing setting was the 'location' setting in the Passenger config file.

Although the Django server serves up the static files, including the build files for your React app, Nginx doesn't see any static files except those in a 'public' directory.

So to deploy a Django app built with Webpack to production, you need to tell Nginx about those files. If you're using Passenger, these settings are probably in a separate Passenger config file. 'alias' is the command to use in this case where the folder has a different name from 'static' (which is where the web page links point).

If you use a virtual environment for your app, you need to specify where Passenger can find the right Python executable.

/etc/nginx/sites-enabled/myapp.conf

server {
    listen 80;
    server_name xx.xx.xx.xx;

    # Tell Passenger where the Python executable is
    passenger_python /var/www/myapp/venv36/bin/python3.6;

    # Tell Nginx and Passenger where your app's 'public' directory is
    # And where to find wsgi.py file
    root /var/www/myapp/myapp/myapp;

    # Tell Nginx where Webpack puts the bundle folder 
    location /static/ {
       autoindex on;
       alias /var/www/myapp/myapp/assets/;
    }

    # Turn on Passenger
    passenger_enabled on;
}

Passenger uses the wsgi.py file as an entry point to your app. You need a passenger_wsgi.py file one level above the wsgi.py file. This tells Passenger where to find the wsgi.py file.

/var/www/myapp/myapp/passenger_wsgi.py

import myapp.wsgi
application = myapp.wsgi.application

/var/www/myapp/myapp/myapp/wsgi.py

If you are using a separate production_settings.py file, make sure this is specified here.

import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.production_settings")
application = get_wsgi_application()

Vernacularize

Answer 2

Create-react-app has a fairly opinionated setup for local and production environments.

Locally, running npm start will run a webpack-dev-server, which you would typically access on port 3000. It runs a local nodejs web server to serve the files. You can route requests to your local Django server via the proxy setting.

It's worth noting that at this point there is little or no connection between your React app and Django. If you use the proxy setting, the only thing connecting the two apps is the routing of any requests not handled by your React app to your Django app via the port.

By default in create-react-app (and as noted in the tutorial you mentioned you are following) in production you would run npm run build which will process your create-react-app files into static JS and CSS files, which are then accessed in Django like static files any other Django app.

One thing Django is missing in order to access the static files is a way to know what files are generated when running npm run build. Running a build will typically result in files output like this:

- css
   |- main.e0c3cfcb.css
   |- main.e0c3cfcb.css.map
- js
   |- 0.eb5a2873.chunk.js
   |- 0.eb5a2873.chunk.js.map
   |- 1.951bae33.chunk.js
   |- 1.951bae33.chunk.js.map

A random hash is added to filenames to ensure cache busting. This is where webpack-bundle-tracker and django-webpack-loader come in. When build files are generated, an accompanying file is also created called manifest.json, listing the files created for the build. This is generated in Webpack and picked up by django-webpack-loader so that Django can know which files to import.

It is possible to run a nodejs server in production, or to use server-side rendering, but if you're following the tutorial you mentioned and using create-react-app default settings, then running npm run build and deploying the static files is the simplest, safest option.

Nothing in any of the Passenger deployment links you mention cover anything beyond deploying a Python/Django app - you would need to manage two apps and deployments to have both Django and React running as servers in production.

Note that the tutorial you mention covers how to get your build files into Django in production, but you will need to ensure that you have webpack-bundle-tracker, django-webpack-loader and your Django staticfiles configuration all configured to work together.

Heintz

Thank you for your explanation. I believe I am following the exact process you suggest - I ran 'npm run build' on the production machine - but I can't figure out why the static files aren't being found on the production server. They are present in the specified folder, and I haven't changed any of the configuration from what works on the development machine, where I am definitely not running a separate app for the client, only the main Django app which is finding the static files and serving them up no problem. I can't at present think how to investigate this any further. – Vernacularize

I can see in the browser that the app has loaded two files: build/static/js/1.ab5b0163.chunk.js and build/static/js/main.53d060d0.chunk.js. However the contents don't look right, each is just html. The corresponding files built on my development machine contain lots of compressed JavaScript. So I'm thinking there is something wrong with my build process. – Vernacularize

what kind of HTML is in those files? – Heintz

Thank you, I have added the HTML to the main question. The next thing I can think of to try is creating the app entirely on the production server to see if that works. – Vernacularize

I'd strongly recommend building the files locally and deploying them to prod with the rest of the Django app. I'm not sure why the js file generated on the server includes the HTML and the entire codebase, but that's obviously not right. – Heintz

That sounds like a much better plan. I'm new to Django and don't know the right way to deploy to prod - I am used to Meteor where you simply build a bundle and unpack it on the server. Can you recommend a method? E.g. should I add the build files to GitHub? Or is there a good tool for directly copying the app from local to prod? – Vernacularize

I've copied the build/static files from my local machine to the server, and have opened 1.xxx.chunk.js to check it has the same content as the local file. However the browser is still seeing the wrong content which is actually the index.html file in public - and I have done a hard reload. It seems like the server is serving public/index.html in place of all my static files? – Vernacularize

the index.html file from create-react-app is not used in production - in prod you'll have a Django template which should have a reference at the bottom to the files generated by Webpack (via django-webpack-loader). You can check the files into GitHub, but if you work with others they will be a constant source of merge conflicts. Currently my team uses Docker to manage environments and deployments. – Heintz

Let us continue this discussion in chat. – Vernacularize