3

Deploying a Django app (to Dreamhost)

I’ve been using Django a lot more and more these days. It’s pretty easy to build and test an app within the testing environment it ships with, but things get trickier when it comes to deploying it to a production environment– several times more tricky when you factor in the different permutations of middleware you need in order to run it (Apache or nginx? Mod_wsgi? Gunicorn? etc.).

Some people get a little dramatic about it (if you repeatedly discover under incident that you need caching, asynchronous tasks and distributed message queues while your database is in flames then sorry, but you are terrible at scoping and planning). There are a lot of Django haters. Ignore them; such plebes are just obstacles in your path to success.

Chances are you’re not launching the next iteration of LinkedIn or a payment processing platform, so if you’re planning on deploying your basic web app to shared hosting such as Dreamhost, Heroku or whatever the Amazon service is that lets you run discrete apps, thankfully you can skip a lot of those architectural decisions since they’ll be made for you. In this tutorial, we’re deploying the pinnacle of high-performance web applications, an app that displays the Dancing Banana gif, to a shared Debian instance on Dreamhost running Apache with Passenger.

Set up hosting and enable Passenger

If you’ve deployed other sites to Dreamhost, you’ve probably glossed over the Passenger checkbox since it never seemed to apply to your PHP site. It matters now. Go ahead and enable Passenger, and have a laugh at the VPS tax Ruby and Node.js users get slapped with for the same service.

Dreamhost Hosting Panel

Enable shell access for your user account

If you don’t already have shell access enabled for your account, add it now. Also for the love of god disallow FTP; unless you’re working with a mainframe from 1970 why is that still an option in 2016…?

Yes I know my password is weak; “I’ll add changing it to my next sprint.”

Dreamhost User Panel

Test your site locally

Make sure everything works when you run locally using the built-in demo server.

Here’s the structure of my demo app, if it matters:

screenshot_2016-10-01_16-14-04

If it works, we need to change a few settings to make your site ready for production deployment.

Dancing Banana

Modify settings.py

You need to do a few things here.

  • Change DEBUG to False, and
  • Set ALLOWED_HOSTS to your domain, plus a leading period [‘.nexxus.xyz’].
  • Change your database settings to whatever you intend to use in production (too niche for me to go into here).
  • Change STATIC_URL to ‘/’
  • Set STATIC_ROOT to the location where all your assets will be found (more on this later).
    • In my case, it’s ‘/home/dh_5hi2et/nexxus.xyz/public’
    • It will be the same for you on Dreamhost with Passenger, just change the username and domain.

I have heard mumblings that if your domain has an underscore in it, it will create problems, but I haven’t seen it myself.

screenshot_2016-10-01_14-14-37

When we initially deploy, you may encounter HTTP 400 – Bad Request errors in the near future. You may take it upon yourself to go hit StackOverflow to troubleshoot the issue before we get to the end of this tutorial. Resist the urge to follow the commoners’ advice to put an asterisk (*) in the allowed_hosts list. This is a very bad thing to do and leads to people blaming Django when their site gets popped. Just like PHP– the insecurity is not with the language, it’s with the people who come up with bad ideas and spread them.

What I can tell you is that though the documentation says ALLOWED_HOSTS should accept just a domain name as an entry, I have found this to be a source of the 400 responses. When I specify the domain with a leading “.” I do not have these issues.

To be fair, Django does have its share of holes– for example, transmission of secure cookies is allowed over HTTP by default. From the documentation:

SESSION_COOKIE_SECURE

Default: False

Whether to use a secure cookie for the session cookie. If this is set to True, the cookie will be marked as “secure,” which means browsers may ensure that the cookie is only sent under an HTTPS connection.

Since it’s trivial for a packet sniffer (e.g. Firesheep) to hijack a user’s session if the session cookie is sent unencrypted, there’s really no good excuse to leave this off. It will prevent you from using sessions on insecure requests and that’s a good thing.

If your site uses HTTPS and sessions, do what you will with this setting. I’m sure there are more snags like this too, so read the documentation– it changes often.

Upload your code

WARNING: Passenger on Dreamhost creates a folder for public-facing web assets within your home directory, named, of all things, “public.” It functions the same as ‘/var/www/html’ on traditional Apache setups.

  • Anything you put in this folder will (barring .htaccess rules working as intended) be visible to the entire internet.
  • DO NOT UPLOAD YOUR CODE TO THIS FOLDER.
  • DO NOT UPLOAD ANYTHING TO THIS FOLDER.

You don’t need to put anything in it for this deployment to work as intended. An automated process we will run in a bit will copy over only the necessary public files needed to run the public-facing side of your site (images, JavaScripts, etc.). Do not do this manually, it is not necessary, and will probably break something and/or compromise your site.

Instead, upload your entire project folder to the home directory of your site.

screenshot_2016-10-01_14-38-13

Migrate your database

Personally I don’t believe in migrating data from dev environments to production, but to each their own– however you do this is going to be specific to your app, so I can’t help you with that.

Create a virtualenv

You don’t have to do this, but if you want to do things right and not have your site break at some random time in the future when Dreamhost updates the default Python version, you will.

Currently Dreamhost is running Python 2.7.3. That’s good enough for me, but if you absolutely need Python 3 or want to try something crazy like PyPy, you’ll need to specify the path to your interpreter of choice. Compiling those is beyond the scope of this tutorial.

If 2.7.3 is good enough for you, within your ~/site directory (/home/dhwhatever/nexxus.xyz in my case), run:

virtualenv PYENV

Or if you want a different interpreter, you’ll need to specify the path to it.

virtualenv PYENV -p /home/dhwhatever/python-version-you-downloaded/bin/python

In either case you don’t have to call it PYENV, but if you do, it will require fewer changes in the next step.

Once complete, run:

source PYENV/bin/activate

This will override the system defaults (Python 2.7.3) and any time you invoke the “python” command it will default to the interpreter you specified.

Download Django

You can download the latest release on Dreamhost’s machine by typing:

pip install django

But you would be better served downloading the same version you conducted development under. Small things do change between versions. If you don’t know what you developed under, open a terminal on your local machine:

(PYPY) [email protected]:~/workspace/bananaguy$ python
Python 2.7.10 (5.0.1+dfsg-4, Apr 08 2016, 00:22:14)
[PyPy 5.0.1 with GCC 5.3.1 20160407] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>> import django
>>>> django.get_version()
'1.9.7'
>>>>

So since I’m running 1.9.7, I would do:

pip install django==1.9.7

Create passenger_wsgi.py

This is one of the trickier parts, since there is no official documentation telling you that this is necessary, yet this is the single thing that determines whether or not your site actually gets served.

In the root of your domain folder (“/home/dhwhatever/nexxus.xyz”) create a file called “passenger_wsgi.py”.

nano passenger_wsgi.py

Pasting in something like this should work. I didn’t write it; this guy did, but I did modify it. It doesn’t follow PEP guidelines; I don’t care, I’m trying to make it easy for you.

Change the first two lines accordingly and if you decided to be difficult and use an alternate interpreter or virtualenv name, make sure everything else accounts for your deviance.

dirpath = '/home/dh_5hi2et/nexxus.xyz' # Leave the trailing slash OFF!
project_name = 'bananaguy'

# -----------------

import sys, os

sys.path.append(dirpath + '/')
sys.path.append(dirpath + '/' + project_name)

INTERP = os.path.expanduser(dirpath + "/PYENV/bin/python")

if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

sys.path.insert(0,dirpath + '/PYENV/bin')
sys.path.insert(0,dirpath + '/PYENV/lib/python2.7/site-packages/django')
sys.path.insert(0,dirpath + '/PYENV/lib/python2.7/site-packages')

os.environ['DJANGO_SETTINGS_MODULE'] = project_name + ".settings"

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Load your static files

I chose a site that displays a gif as opposed to a simple text-based “hello world” for a reason– “hello world” has no assets, and I’d be done already. Chances are your site actually has images and scripts so I’d be doing you a disservice ignoring that.

From the Dreamhost shell, and within your virtual environment, go to your equivalent of ‘/home/dh_5hi2et/nexxus.xyz/bananaguy’ and run:

python manage.py collectstatic

You should see a decent amount of stuff copy over. For an essentially bare app, it will copy over around 60 files pertaining to the admin interface.

(PYENV)[richard-bassett]$ python manage.py collectstatic

You have requested to collect static files at the destination
location as specified in your settings:

    /home/dh_5hi2et/nexxus.xyz/public

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes
Copying '/home/dh_5hi2et/nexxus.xyz/PYENV/local/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/base.css'
Copying '/home/dh_5hi2et/nexxus.xyz/PYENV/local/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/changelists.css'
Copying '/home/dh_5hi2et/nexxus.xyz/PYENV/local/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/dashboard.css'
...
Copying '/home/dh_5hi2et/nexxus.xyz/PYENV/local/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/vendor/xregexp/xregexp.js'
Copying '/home/dh_5hi2et/nexxus.xyz/PYENV/local/lib/python2.7/site-packages/django/contrib/admin/static/admin/js/vendor/xregexp/xregexp.min.js'
Copying '/home/dh_5hi2et/nexxus.xyz/bananaguy/common/static/common/img/banana.gif'

62 static files copied to '/home/dh_5hi2et/nexxus.xyz/public'.
(PYENV)[richard-bassett]$

Then you can check its success by doing:

(PYENV)[richard-bassett]$ ls /home/dh_5hi2et/nexxus.xyz/public
admin  common  favicon.gif  favicon.ico  quickstart.html

Looking good.

Stopping and restarting services

Dreamhost’s Passenger implementation is a bit weird. When the handler launches your wsgi script, it seems to hang– thus every change you make to the site will fail to show up, leaving you ripping apart your code, undoing changes and wondering how to restart Apache.

You don’t need to restart Apache, you need to restart Passenger. When things seem stuck, check for a Passenger script running in the background and kill it.

(PYENV)[richard-bassett]$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
16151480  4478  0.1  0.0 179904 25252 ?        S    12:37   0:00 /home/dh_5hi2et/nexxus.xyz/PYENV/bin/python /dh/passenger/helper-scripts/wsgi-loader.py
16151480  7147  0.0  0.0 126936  1224 pts/0    R+   12:43   0:00 ps aux
16151480 17935  0.0  0.0  71360  1696 ?        S    11:36   0:00 sshd: [email protected]
16151480 17936  0.0  0.0  12776   940 ?        Ss   11:36   0:00 /usr/lib/openssh/sftp-server
16151480 26034  0.0  0.0  71360  1564 ?        R    12:03   0:00 sshd: [email protected]/0
16151480 26035  0.0  0.0 129760  2140 pts/0    Ss   12:03   0:00 -bash
(PYENV)[richard-bassett]$ kill 4478

Then, the way things are supposed to work (but I don’t know if they do) is that you’re supposed to modify the timestamp on a temporary file to signal to Passenger that it should restart.

(PYENV)[richard-bassett]$ pwd
/home/dh_5hi2et/nexxus.xyz
(PYENV)[richard-bassett]$ mkdir tmp && touch tmp/restart.txt

You are now live!

Yeah, it seemed like a pain, but honestly the hard part is over. Now as long as you don’t fundamentally alter your site structure you should never need to mess with wsgi configs or Passenger again.

Success

3 Comments

  1. Just awesome. Saved my life too!

    But here I had to change from:

    sys.path.append(dirpath + ‘/’)
    sys.path.append(dirpath + ‘/’ + project_name)

    to:
    sys.path.append(dirpath + ‘/djangofolder/’)
    sys.path.append(dirpath + ‘/djangofolder/’ + project_name)

Leave a Reply