Quick guide to creating reusable Django apps

I have developed several Django-CMS sites, e.g. Racing Tadpole, U R The Event Manager, Aquizzical and School Circle, and find myself frequently wanting to reuse pieces of the code.  Of course, this is why Django has the concept of reusable apps, and has a good intro tutorial here.

I am going to embellish on that tutorial and show the actual steps I followed to go from no app, to a single package (possibly containing multiple apps) on the Python Package Index, pypi, that anyone can install with pip, that can be found by others on djangopackages, and with source code on github.  I have already followed this process for cmsplugin-rt, which (in my humble opinion) contains lots of helpful Django-CMS plugins (particularly for Twitter Bootstrap) like buttons and a navbar, but also more generic plugins like Google fonts, Google analytics, Facebook and Twitter buttons.  You can install this easily by typing:

pip install cmsplugin-rt

I now want to repeat that process for a new set of apps, which will include some global settings for Bootstrap.

  1. Create a dummy Django project with all the apps you are going to need (e.g. Django-CMS in my case). 
  2. Write your app, starting with python manage.py startapp myapp 
  3. I have chosen to place this app one directory level down, i.e. cmsapp_rt/bsglobals/, so that I can put more than one app in my package (cmsapp_rt) and have them all installed by pip in one go. Note you need to put an __init__.py file in the cmsapp_rt directory.
  4. Check your app works by including it in the INSTALLED_APPS in your settings.py file, and typing python manage.py runserver.
  5. My projects are in ~/Python/Projects.  My approach is to create a directory like ~/Python/MyPackages, and in it create directories for each pip package. Note as far as I can tell, pip packages like to use hyphens instead of underscores.  So I have a directory ~/Python/MyPackages/cmsapp-rt/.
  6. Following the Django tutorial, type: mv ~/Python/Projects/dummy/cmsapp_rt ~/Python/MyPackages/cmsapp-rt/.
  7. Create the README.txt or README.rstLICENSE and MANIFEST.in files in the ~/Python/MyPackages/cmsapp-rt/ directory, as per the tutorial. Add the docs directory. I also put in a CHANGES file (and add it to the MANIFEST.in file too). Note if you have any fixtures, you need to add them to the MANIFEST.in file in the same was as the templates. Feel free to base yours off mine if it helps.
  8. Create the setup.py file.  Note you can add a URL to github here, e.g.
        url = 'https://github.com/RacingTadpole/cmsapp-rt'

    I also used

        from setuptools import setup
        setup(...
            find_packages packages = find_packages(),
        )

    and

            install_requires = [
                'django-singleton-admin',
            ],

    and most importantly, add to the arguments to setup:

            zip_safe = False,

    which forces the package to be installed as real files, not a zipped up egg. Django seems to struggle with the zipped up eggs.

  9. Commit your work to git – add a .gitignore file which contains at least:
        dist/
        *.egg-info
        build/

    Make yourself a new repository on Github. Then type:

        git init
        git remote add origin https://github.com/user/reponame.git
        git add --all
        git commit -a -m "Initial commit"
        git push -u origin master
  10. You can build the package with python setup.py sdist , as per the tutorial.
  11. Now it’s time to follow this guide from pypi.  The only two things you need from this are:
        python setup.py register
        python setup.py sdist bdist_wininst upload

    The guide says you need to register on the site before that first command, but you can just run the command and it will do that for you.  Also, I get the warning message below, but it doesn’t seem to matter (I have been using a Mac and a linux machine; maybe it is a problem for Windows?):

        Warning: Can't read registry to find the necessary compiler setting
        Make sure that Python modules _winreg, win32api or win32con are installed.
  12. Whenever you make changes to the code, you only need to update README, CHANGES, the version number in setup.py, re-commit it to git (and push it to github), and type:
    python setup.py sdist bdist_wininst upload
  13. At this point, you should be able to install the package using
        pip install projectname

    (or if you are only updating it, add --update on the end).

  14. With any luck, your dummy project will start to work again once you’ve done that, but this time, it will be drawing the app from your system’s packages, and any other project can do so too.  (In practice, you will want to use virtualenv for this.)
  15. Finally, let django packages know about your app.  This is very easy if you have already put your project on github – use the form here.

All done. I have followed this through and just published cmsapp-rt in this way, which you can now install with

    pip install cmsapp-rt

Please let me know if you have any suggestions or improvements!

  

7 thoughts on “Quick guide to creating reusable Django apps”

  1. Thanks a lot for the explanation! I’m in the process of making my first real 3rd party app and have som..philosophical issues that you might be able to shed some light on.

    As I see your workflow, it’s basically:
    1. Set up Django project with the app
    2. Develop the app within that project
    3. Move/Copy the app in order to isolate it (lacking a better term)
    4. Git and package stuff

    But what do you do when you continue work on the app? Again, a list:
    1. (optional) get the code from Github if the code is updated by another developer
    2. Copy the app into the Django project
    3. Do some work
    4. Copy the code back to its isolated project
    5. Update version number etc, then git push

    I mean, the whole copying the code back and forth seems quite clunky. Is there a way to keep the app within the Django project, perhaps that would ease the process a bit. Or do you have a completely different workflow?

    Thanks for sharing!

    1. Hi Martin,
      Sorry not to reply earlier – thanks for your comment. You’re right about my workflow as I continue working on the app. Another approach I use is to directly modify the code in my virtual-env’s lib/python/site-packages, check that my app still works when run (as it now uses the updated version of the app), and then when I’m happy I copy it from the virtual-env into its isolated project. Either way, I agree it’s very clunky.
      I would love to know if you discover a better way!
      cheers
      Arthur

      1. Thanks for your reply Arthur! I see..thanks for explaining! Well, as a lazy programmer (isn’t that one of the reason we do this? :)) I think I need to find a smoother solution. I’m quite unorganized and I think if I worked like that, I would do too many errors :)

        I’ll keep you posted if I find any good solutions! Keep up the good work!

        1. Hi Martin,

          I just found a solution used by django-multiselectfield. He puts an example directory in there, which contains manage.py, run_tests.py and an example directory. This last directory contains settings.py etc, and an app directory with the models.py, tests.py and other app-defining files.

          Not sure if this is a common way of doing it – what do you think?

        2. Arthur,

          I have seen that as well but never tried it in more detail. What I ended up doing was:
          - Create a skeleton project (just basic settings, templates etc). This is to be able to develop the app.
          - Copy/create my app in the skeleton project.
          - Add the root directory to Git/hg, but exclude everything but the app’s files like this
          .hgignore:

          syntax: glob
          *.pyc

          syntax: regexp
          ^django_myapp/
          ^media/
          ^static/
          ^__init__.py
          ^requirements.txt
          ^requirements_local.txt
          ^manage.py

          Then when I push the repo I only get the follwing structure:

          django_myapp
          - myapp
          -
          LICENSE
          MANIFEST.in
          README.rst
          etc

          After that, I can use pip to install it to a production project as an external package:
          pip install -e hg+ssh://hg@bitbucket.org/myuser/django-myapp/#egg=django-myapp

          Does that make sense? For me it works quite well and there’s just a little overhead for pushing and pulling after every commit.

          The only issue I have now is that I’m working on a Windows machine with Virtualbox installed, and I can’t compile a proper package in Virtualbox for some reason (have googled it and it seems to be a bug with Virtualbox).

        3. Thanks for the write up. Concerning the comments I was wondering the same thing and came up with this solution. I don’t know yet if there is a downside to this, but I guess I’ll eventually see along the way… :-)

          I have a django setup with various apps, one of them now an app that I want to make reusable.

          The django project lives here


          /myproject
          |................./makereusable
          |................./someapp
          |................./someotherapp

          I’ve moved the reusable app to a separate location outside the project and followed the django tutorial, leaving me with this structure for that particular app:


          /django-makereusable
          |......................................./.git
          |......................................./.gitignore
          |......................................./
          |......................................./makereusable
          |......................................./makereusable/

          After following along the build and pip install path and confirming that it worked, I used PIP to uninstall my package. Then I headed back to my project and got the app directory back to where it was with a symlink.


          /myproject $ ln -s /django-makereusable/makereusable ./makereusable

          Now I can continue to develop, migrate and test in the same setup right as before. Version control can be handled from within the /django-makereusable tree. Whenever I hit a milestone I can create a new package from it.

          Let’s see how this works out, but right now it does what it should without the need to create separate setups for reusable apps. (also I think that one reusable app per setup can hit some limits at times when a 3rd party eco system relies on more than just one core package)

        4. Hi Stephan,
          Nice idea! I like how there’s no extra clutter with your symlink approach. I imagine that works for most tests in the reusable app, too, as long as the tests don’t need to know anything about the project they’re part of.
          Thanks for sharing – I’ll try it out for my next reusable app (or even next time I need to update one of my existing ones).
          cheers
          Arthur

Comments are closed.