Getting started with Django

For the past couple of weeks I have been developing a website using Django, a python-based framework for content management. This is after writing the site for some weeks directly in PHP using SQL, without a particular framework. At first the site started out quite simple, so that seemed sensible. As it grew I realised it would someday reach a point of unmanageability, so I have turned to a framework.

Frameworks have several advantages, including cleanly separating the underlying business logic (the model) from the way it is presented (the view); abstracting away from the SQL, instead presenting the objects represented by the SQL; making forms and form validation easier; making user registration easier; making database administration easier; making the URL scheme easier; and having lots of add-ons that make extended functionality easier.  I chose Django because I am a fan of Python.

Django has some great features which make it easy to convert an existing SQL database structure over, with hardly any changes required to the tables. The one sticking point for me has been the user registration. I hesitated for some time before tackling this, because uncharacteristically, the user model in Django 1.4 is fixed, and to extend it requires an additional one-to-one table.  Adding to the uncertainty, whatever you do now will not be supported in Django 1.5, when a more flexible user model will be released.  However, it is fairly easy to implement the extension, and I hope it will be easy to convert to Django 1.5 when the time comes.

So here I want to point out the stumbling blocks that I hit as a total beginner to Django, so that you can help see ahead of time what is involved in learning and using it. My comments come with the caveat that I have been using Django for a matter of weeks, so do not take me as any sort of authority on best practice. I would love feedback on how to do things better!

I think the best way to go is to describe (in a mini-tutorial) how I’ve built my two-phase user registration process, which uses email addresses in place of usernames. You’ll see how I implemented “Sign in” and “Join” pages, how to check a new user’s email is not already in the database, to send an email to the user’s email address when they first join, to only activate the user when they follow a link in that email, and to handle “forgot password” and “stay signed in”.  I’ve done this for a site using jQuery Mobile. Let’s begin!

App or project?

Begin with a project and an app, just like the poll app in the Django tutorial. I originally started building my user registration process as part of this app (I’ll call it pollapp), but have since realised I can separate it out as its own app, which I am calling RTloginapp.  RTloginapp can then be used for other projects where the User model may be different. (Remember to add 'RTloginapp' to the INSTALLED_APPS in settings.py.)

The points where RTloginapp and pollapp need to communicate are:

  • RTloginapp needs customised views, eg. to display the forgot password form, the user signin form etc.  Solution: All the templates are actually put in pollapp. If I was properly packaging up RTloginapp as well, I would put default templates in RTloginapp too. I’d like to have those default templates simply extend a base template so they have more chance of being actually useful… but I’m not sure what the standard practice is here – please let me know if you have an opinion.
  • In the URLs. I have put the RTloginapp URL scheme into pollapp/urls.py, not RTloginapp/urls.py.  This way the url scheme is not fixed for every project, and you can pass the views extra project-specific parameters (the next few points).
  • RTloginapp needs to display a customised first-time user join form, since different projects may need different information about the user. Solution: in urls.py, pass the join view an extra parameter that specifies the join form.
  • RTloginapp needs to save the first-time user’s information when they join. Solution: in urls.py, pass a function as a parameter to the join view. The function saves any extra information.
  • RTloginapp needs to know a few links – the “forgot password” URL and the “activate” URL. These are passed in via urls.py too. You can also optionally pass in a “return” URL which is passed to the templates if required, and any number of additional context settings.

So pollapp/urls.py will look like this when we’re done. Note I’ve added an extra question mark after the final slash in each URL (before the $), so that the final slash in the URL is optional. The official documentation seems to assume URLs will always finish with a slash.

from pollapp.forms import JoinForm
from pollapp.profile import SetJoinerProfile

urlpatterns = patterns('RTloginapp.views',

    url(r'^accounts/?$', 'signin'),
    url(r'^accounts/login/?$', 'signin'),
    url(r'^accounts/join/?$', 'join', {'JoinForm': JoinForm,
        'SetJoinerProfile': SetJoinerProfile,
        "forgot_link":"/accounts/password/reset",
        "activate_link":"/accounts/register"}),
    url(r'^accounts/register/?$', 'activate'),
    url(r'^accounts/signout/?$', 'signout'),
)

urlpatterns += patterns('your-app.views',
		# other urls
)

Django comes with a password-reset facility, which we can use by adding the following import:

    import django.contrib.auth.views

and these URLs:

    url(r'^accounts/password/reset/?$',
        django.contrib.auth.views.password_reset,
        {'post_reset_redirect' : '/accounts/password/reset/done/'}),
    url(r'^accounts/password/reset/done/?$',
        django.contrib.auth.views.password_reset_done),
    url(r'^accounts/password/reset/(?P[0-9A-Za-z]+)-(?P.+)/?$',
        django.contrib.auth.views.password_reset_confirm,
        {'post_reset_redirect' : '/accounts/password/done/'}),
    url(r'^accounts/password/done/?$',
        django.contrib.auth.views.password_reset_complete),

Extending the User model

This applies to Django 1.4. and just follows the documentation. Add this to settings.py:

AUTH_PROFILE_MODULE = 'pollapp.UserProfile'

and this to pollapp/models.py (changing the specifics to your needs! I’ve said here that users belong to a specific “circle”, and have a flag for whether they have read the terms of service):

from django.contrib.auth.models import User

class UserProfile(models.Model):
    # This field is required.
    user = models.OneToOneField(User)
    circle = models.ForeignKey(Circle)
    read_terms = models.BooleanField()

    def __unicode__(self):
        return self.user.email

from django.db.models.signals import post_save

def create_user_profile(sender, instance, created, **kwargs):
    if created:
        UserProfile.objects.create(user=instance)

post_save.connect(create_user_profile, sender=User)

# This next piece is mine - it changes the appearance of the 
# user in the admin panel (useful if you use an email address
# instead of a username, as we will do in the next section)

def user_display(self):
 return "%s %s (%s)" % (self.first_name, self.last_name, self.email)

User.__unicode__ = user_display

And create pollapp/admin.py:

#
# to extend the user model in the admin, as per
# https://docs.djangoproject.com/en/dev/topics/auth/
#
from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.models import User

# Define an inline admin descriptor for UserProfile model
# which acts a bit like a singleton
class UserProfileInline(admin.StackedInline):
    model = models.UserProfile
    can_delete = False
    verbose_name_plural = 'profile'

# Define a new User admin
class UserAdmin(UserAdmin):
    inlines = (UserProfileInline, )

# Re-register UserAdmin
admin.site.unregister(User)
admin.site.register(User, UserAdmin)

While we’re on this subject, we might as well define the function SetJoinerProfile alluded to before (I’ve put this in its own pollapp/profile.py file, hence the earlier from pollapp.profile import SetJoinerProfile):

from circleapp import models

def SetJoinerProfile(new_user, form):
    """
    An extra function required by RTloginapp,
    to set additional user profile data on first joining.
    The function is provided with the new user object and the JoinForm.
    It should finish by saving the user profile.
    """
    new_user_profile = new_user.get_profile()
    circle = form.cleaned_data["circle"]
    new_user_profile.cricle = circle
    new_user_profile.read_terms = form.cleaned_data['read_terms']
    new_user_profile.save()  # is this the standard approach?
    return

Note the last line is to save the new user’s profile. I haven’t found any reference online to needing to do this – I would have thought saving the user would automatically save the profile too (the user is saved by RTloginapp immediately after calling the above function). But I find that it’s not saved if you don’t explicitly save the profile.

Using an email address instead of a username

I personally like using the email address instead of a username – it’s one less thing to have to remember. But Django 1.4 needs a unique username (max of 30 chars, so I can’t really use the email address as the username). My solution is simply to hash the email address and use the first 30 characters as the username. I include a check for hash collisions, to be on the safe side, though I’m sure it’s overkill.

To make this easier, I’ve added a “backend” so that you can look up a user based on their email address, following this post.  This will allow us later to authenticate users with the command:

   user = authenticate(username=email, password=password)

For this you need to add to settings.py:

AUTHENTICATION_BACKENDS = (
    'RTloginapp.accounts.backends.EmailUsernameBackend',
    'django.contrib.auth.backends.ModelBackend'
)

I then added an accounts directory under RTloginapp, with a blank __init__.py file, and a file backends.py which contains:

from django.conf import settings
from django.contrib.auth.models import User
from RTloginapp import utils

class EmailUsernameBackend(object):
    def authenticate(self, username=None, password=None):
        """
        If the username is an email address,
        then get the user with that email address.
        Otherwise get the user with that username.
        """
        if '@' in username:
            (uname, user) = utils.get_username_and_user(email=username)
            # if that couldn't find the user, it's possible the user's
            # username was entered manually, so just look up the email address
            if user is None:
                user = utils.get_object_or_none(User, email=username)
        else:
            user = utils.get_object_or_none(User, username=username)
        if user is not None:
            if user.check_password(password):
                return user
        return None

    def get_user(self, user_id):
        try:
            return User.objects.get(pk=user_id)
        except User.DoesNotExist:
            return None

This uses some utility functions I put together and saved as RTloginapp/utils.py:

def get_object_or_none(model, **kwargs):
    """
    Sample usage: user = get_or_none(User, username='me')
    """
    try:
        return model.objects.get(**kwargs)
    except model.DoesNotExist:
        return None

from django.contrib.auth.models import User
import hashlib

def _get_user(username, email):
    """Used by get_username_and_user"""
    user = get_object_or_none(User, username=username)
    if user is not None:
        if user.email != email:
            # remotely possible the hashes collide
            username = 'x'+username[:29]
            (username, user) = _get_user(username, email)
    return (username, user)

def get_username_and_user(email):
    """
    Pass the user's email address, will return a tuple of the username
    (almost always the hash of the email)
    and the user object if it exists
    """
    username = hashlib.md5(email).hexdigest()[:30]
    return _get_user(username, email)

Outstanding – how do I get the user’s identification in the admin console to show as the email address, instead of the now unintelligible username?

Forms

This is a good place to introduce the JoinForm. Let’s define the minimal form required by the login app, by putting the following into RTloginapp/forms.py. Note the use of widgets to insert placeholder text in the form.

from django import forms

class BaseRTJoinForm(forms.Form):
    name = forms.CharField(max_length=100, label="Your name",
        error_messages={'required': 'Please enter your name.'},
        widget=forms.TextInput(attrs={'placeholder': 'Your name'}))
    email = forms.EmailField(label="Email address",
        error_messages={'required': 'Please enter your email address.'},
        widget=forms.TextInput(attrs={'placeholder': 'Email address'}))
    password1 = forms.CharField(label="Choose a password",
        error_messages={'required': 'Please choose a password.'},
        widget=forms.PasswordInput(render_value=True,
            attrs={'placeholder': 'Choose a password'}))
    password2 = forms.CharField(label="Re-type password",
        error_messages={'required': 'Please re-type your password.'},
        widget=forms.PasswordInput(render_value=True,
            attrs={'placeholder': 'Re-type password'}))

    def clean(self):
        cleaned_data = super(BaseRTJoinForm, self).clean()
        password1 = cleaned_data.get("password1")
        password2 = cleaned_data.get("password2")

        if password1 and password2 and password1!=password2:
            msg = r"Your re-typed password does not match the first one. Please type them both again!"
            self._errors["password1"] = self.error_class([msg])
            del cleaned_data["password1"]
            del cleaned_data["password2"]

        # Always return the full collection of cleaned data.
        return cleaned_data

This just follows the principles outlined in the Django documentation.

Then add the customised join form in pollapp/forms.py, eg.:

from django import forms
from RTloginapp.forms import BaseRTJoinForm
from pollapp import models

class JoinForm(BaseRTJoinForm):
    circle = forms.ModelChoiceField(queryset=models.Circle.objects.all(),
        label="Your circle",
        error_messages={'required': 'Which circle are you in?'},
        widget=forms.Select(attrs={'data-theme': 'c', 'data-inline':'true'}))
    read_terms = forms.BooleanField(label="I agree",
        error_messages={'required': 'Please agree to the terms to join.'},
        widget=forms.CheckboxInput(attrs={'data-theme': 'c', 'data-inline':'true'}))

Sending email

We also need to be able to send emails.  This is covered nicely in this stackoverflow post. Add this to utils.py:

def send_email(email_template_name, to_emails, context_dict={}):
    """
    Sends an email based on a template (e.g. login/email/join.html).
    Requires the following files:
    email_template_name.html - html email content, rendered as a template
    email_template_name.txt - plain text content of the email as a template
    email_template_name-subject.txt - a single line with the subject as a template
    email_template_name-from.txt  - a single email address stating who it is from
    to_emails can either be a single email in a string, or a list.
    """
    from django.core.mail import EmailMultiAlternatives
    from django.template.loader import get_template
    from django.template import Context

    if isinstance(to_emails, basestring):
        to_emails = [to_emails]

    subjecttext = get_template(email_template_name+'-subject.txt')
    fromtext = get_template(email_template_name+'-from.txt')
    plaintext = get_template(email_template_name+'.txt')
    htmly     = get_template(email_template_name+'.html')
    context = Context(context_dict)

    text_from    = fromtext.render(context)
    text_subject = subjecttext.render(context)
    text_content = plaintext.render(context)
    html_content = htmly.render(context)

    msg = EmailMultiAlternatives(text_subject, text_content, text_from, to_emails)
    msg.attach_alternative(html_content, "text/html")
    if text_subject and text_from and msg and to_emails:
        try:
            msg.send()
        except:
            return False
        else:
            return True
    return False

However, to actually send emails, you also need to add to settings.py (changing the host user and password to match a gmail account that you own) as explained in this stackoverflow post:

    EMAIL_USE_TLS = True
    EMAIL_HOST = 'smtp.gmail.com'
    EMAIL_HOST_USER = '@gmail.com'
    EMAIL_HOST_PASSWORD = 'example-password'
    EMAIL_PORT = 587

Outstanding – PHP lets me send an email from any user. Django only lets me send it from the host user above (it seems to ignore the “from” field in the send() call above). Is there a way to send from different email addresses in Django depending on the function, and even ones that are not real, like “do-not-reply@…”  ?

Sites

The password reset process uses Sites.

If you haven’t already, you’ll need to add your sites using python manage.py shell, along the lines of:

   
>>> from django.contrib.sites.models import Site
>>> s = Site.objects.get(pk=1)
>>> s.domain = 'mysite.com'
>>> s.name = 'My Site'
>>> s.save()
>>> s2 = Site(domain='localhost:8000', name='My local host')
>>> s2.save()
>>> s2.id

In my settings.py I also test if DEBUG is True, and set SITE_ID to 2 if so.

The views

That covers all the set up. Now we need to define the behaviour of four pages:

  • Sign in
  • Join (including sending the new user an email so they can activate their account)
  • Activate a new user when they follow the email link
  • Sign out

Let’s go through them in order:

Sign in

from django.shortcuts import get_object_or_404, render, render_to_response
from django.http import HttpResponseRedirect, HttpResponse
from django.contrib.auth import authenticate, login, logout

def signin(request, return_url='/', **kwargs):
    context = kwargs
    context['return_url'] = return_url
    email = request.POST.get('email', '')
    if email=='':
        # first time here (or no email), so display login form
        return render(request, "login/login.html", context)
    else:
        # is it a valid user? note this utilises the emailUsernameBackend
        password = request.POST.get('password', '')
        user = authenticate(username=email, password=password)
        if user is not None and user.is_active:
            # Correct password, and the user is marked "active"
            login(request, user)
            stay = request.POST.get('stay', False)
            if stay:
                request.session.set_expiry(31*24*60*60) # a month if click "stay logged in"
            else:
                request.session.set_expiry(2*60*60) # or else just two hours
            # Redirect to a success page.
            return HttpResponseRedirect(return_url)
        else:
            # Show login form with error message
            context['invalid'] = True
            return render(request, "login/login.html", context)

This clearly depends on a template called login/login.html, which we’ll come to in the next section. I’ll summarise the context for the templates there too.

Outstanding – it would be nice to set the time-limits for logging in in the settings.py file. (Alternatively they could be parameters passed in via urls.py.)

Join

from django import forms
from django.db import IntegrityError

from RTloginapp import models, utils
from RTloginapp.forms import BaseRTJoinForm

def join(request, JoinForm=BaseRTJoinForm, SetJoinerProfile=None, return_url="/", forgot_link="/accounts/password/reset", activate_link="/accounts/register", **kwargs):
    """The user is "active" if they have confirmed their email address by
       responding to the join email.
       The user is "inactive" if they have filled in the join form but not
       yet responded to the email, or if they are removed for some reason.
       An inactive user cannot sign in.
       If a user tries to join using the email address of an inactive user,
       the old user is updated, their name and related info is overwritten,
       and an email is sent out so they can become active.
    """
    context = kwargs
    context['return_url'] = return_url
    if request.method == 'POST':
        form = JoinForm(request.POST)
        context['form'] = form
        if form.is_valid():
            (username, user) = utils.get_username_and_user(form.cleaned_data['email'])
            if user is None:
                new_user = User.objects.create_user(
                                username=username,
                                password=form.cleaned_data['password1'],
                                email=form.cleaned_data['email'])
            elif user.is_active:
                # if the user has an active account,
                # send an email with a link to "forgot password" instead
                context['new_user'] = user
                context['forgot_link'] = request.build_absolute_uri(forgot_link)
                context['email_ok'] = utils.send_email("login/email/already-joined", form.cleaned_data['email'], context_dict=context)
                return render(request, "login/join-email-sent.html", context)
            else:
                # the user is inactive, so update the user instead
                new_user = user
            #
            new_user.first_name = ' '.join(form.cleaned_data['name'].split(' ')[:-1])
            new_user.last_name = form.cleaned_data['name'].split(' ')[-1]
            new_user.is_active = False
            if SetJoinerProfile is not None:
                SetJoinerProfile(new_user, form)
            new_user.save()
            #
            context['new_user'] = new_user
            context['reg_link'] = request.build_absolute_uri(activate_link + "?l="+username)
            context['email_ok'] = utils.send_email("login/email/join", new_user.email, context_dict=context)
            return render(request, "login/join-email-sent.html", context)
    else:
        form = JoinForm()
        context['form'] = form
    return render(request, "login/join.html", context)

How does that look? Please let me know if you see any security problems with it. Note I also split out the first and last names from a single name field.  This split is not perfect, so I plan to always just display the two together.

Activate

A link to this page is emailed to the user when they join, with the username appended as a GET parameter. All the page does is check the GET parameter is valid, and activate the user’s account if so. The emailed link remains valid indefinitely, which may be not be ideal.

def activate(request):
    if request.method == 'GET':
        reg_id = request.GET.get('l', '')
        if reg_id=='':
            return HttpResponseRedirect("/accounts/join")
        else:
            user = utils.get_object_or_none(User, username=reg_id)
            if user is not None:
                if not user.is_active:
                    user.is_active=True
                    user.save()
                    return render(request, "login/activated.html", {'new_user': user})
                else:
                    return render(request, "login/already-active.html", {'new_user': user})
    return HttpResponseRedirect("/accounts/join")

Signout

def signout(request, return_url="/"):
    logout(request)
    return HttpResponseRedirect(return_url)

The templates

Set up a series of website and email templates, which will have the context shown at right:

   login/
        login.html              invalid (only if True)
        join-email-sent.html    form, new_user, email_ok  (and forgot_link or reg_link)
        join.html               form
        already-active.html     new_user (*)
        activated.html          new_user (*)

    login/email/
        already-joined          form, new_user, forgot_link
        join                    form, new_user, reg_link

Each of the two email templates requires four files: .html and .txt for the content, and -from.txt and -subject.txt (these should only be a single line).

For each of the above except *, the context also includes:

                               return_url
                                any keyword args passed in by the url scheme

You will also need the reset-password templates – there are some samples available in the Django documentation:

   registration/
        password_reset_complete.html
        password_reset_confirm.html
        password_reset_done.html
        password_reset_email.html
        password_reset_form.html

This post is already ridiculously long, so I’ll just show a few of the templates I’m using. I put these in pollapp/templates/, and added this directory to settings.py‘s TEMPLATE_DIRS. The details will depend on your base template and what framework you’re using; this is for jQuery Mobile. It is a little painful to have to create all these files.

login/join.html

{% extends "base.html" %}
{% block title %}Join{% endblock %}

{% block content %}</pre>
<form action="" method="post" data-ajax="false">
{% for error in form.non_field_errors %}
<div class="warning">{{ error }}</div>
 {% endfor %}
 {% csrf_token %}
 {% for hidden in form.hidden_fields %} {{ hidden }} {% endfor %}
 {% for field in form.visible_fields %}
 {% for error in field.errors %}
<div class="warning">{{ error }}</div>
 {% endfor %}
 {% ifequal field.auto_id "id_read_terms" %}
 Finally... please read and agree to our
 <a href="/terms" target="readterms" data-ajax="false">terms of service</a>
 {{ field }}
 <label for="{{ field.auto_id }}">{{ field.label }}</label>
 {% else %}
 <label for="{{ field.auto_id }}">{{ field.label }}:</label>
 {{ field }}
 {% endifequal %}
 {% endfor %}
<div class="right"><a href="{{ return_url }}" data-role="button" data-theme="c" data-inline="true" data-ajax="false">Cancel </a>
 <input type="submit" name="joinBtn" value="Join" data-theme="b" data-inline="true" /></div>
</form>
<pre>{% endblock %}

This is a good place to point out a strange thing about templates – I had to use “ifequal“, not “if a==b“; the latter produced an error.

login/signin.html

{% extends "base.html" %}
{% block title %}Sign in{% endblock %}
{% block content %}
  {% if form.errors or invalid %}
  Sorry, that's not a valid username or password
  {% endif %}

  Please sign in</pre>
<form action="/accounts/login/" method="POST" data-ajax="false">
{% csrf_token %}
<fieldset data-role="fieldcontain"><label class="ui-hidden-accessible" for="email">Email</label>
 <input id="email" type="email" name="email" placeholder="Email" size="35/" data-theme="b" />
 <label class="ui-hidden-accessible" for="password">Password</label>
 <span class="left"><input id="password" type="password" name="password" placeholder="Password" data-theme="b" /></span>
 <span class="right"><input type="submit" name="login" value="OK" data-theme="b" data-inline="true" /></span></fieldset>
<fieldset><span class="left">
 <input id="stay" type="checkbox" name="stay" data-theme="a" data-inline="true" data-mini="true" />
 <label for="stay">Stay signed in</label>
 </span>
 <span class="right">
 <a href="/accounts/password/reset" data-ajax="false">Forgot password?</a>
 </span></fieldset>
</form>
<pre>{% endblock %}

login/email/join.html

Thank you for asking to join my site.
To complete your request, please click the following link:
<a href="{{ reg_link }}">{{ reg_link }}</a>
We ask this to make sure that your email address has not been used by someone else.
If you did not ask to join, please ignore this email, or
email our administrators.
Thank you
My Site

Conclusion

If you followed the above, and added a few more templates of your own, you should have a reasonably functioning user registration system. (I make no guarantees though – use it at your own risk!) In fact, if that’s what you want, I would suggest you take a look at django-registration instead, although I think that still requires a username in addition to an email address.

My real aim in writing this up, though, is to answer some of the newbie questions I had about Django and to help you avoid some of the trouble spots. In particular, how to do some very basic things, like:

  • decide what’s an app and what’s a project?
  • use an email address instead of a username for user registration?
  • extend the user model?
  • send (nicely formatted) emails?
  • build forms so that extra validation can be automatically included?
  • include extra fields like “placeholder” in forms?

I hope I’ve helped to answer all these questions in this post.  Let me know!

Also, look out for my next post on Django, which will cover my discoveries using extensions like Django-CMS, South and virtualenv.