How to develop/include a Django custom reusable app in a new project? Are there some guidelines?

Question

Following tutorial on Django reusable apps things work fine. But I have some questions about the process of developing and packaging a Django app.

1 - In the tutorial, the app is developed first within a project. Then, it is copy-pasted out in another folder for packaging and then included again in the project vía pip. Is this the way for developing Django apps? For example, if I have to include new features or fix bugs, Should I make changes in the project and then copy-paste them to the package folder outside the project?

2 - Assuming that 1 is not the only way to develop an app, I started creating a package folder for my app with this structure:

django-myApp
|--myApp
|  |--models
|     |--file1.py
|     |--file2.py
|--setup.py
|--README.rst

After running python3 setup.py sdist and installing it with pip3 install --user myApp.tar.gz I can successfully import my app from a new Django project shell. But when I run python3 manage.py migrate, tables for models of myApp are not created. I guess it is because there is not migration folder in myApp package, and as far as I know, the only way to create migrations is running makemigrations within a project. Or Am I missing something basic? Can I generate an initial migration module without having the app in a project?

3 - Finally, the main question is: When developing an app, Should I have to start a project, copy out the app folder for packaging, reincluding it vía installing, and then continue developing in the package folder?

Thanks in advance for any comment or guidance.

P.D.: Sorry for my English, also comments about it are well-received

EDIT 1:

An example to highlight my doubt: After finish tutorial, App source code is outside the project and suppose I need to change models. I can change them in App folder, release a new version (e.g. 0.2) and install it. Now, How can I generate migrations for these changes? Should I always have a test project?

Rema

Did you ever get to the bottom of this? I am going through the same process you obviously were and am finding the same thing...My 'package' needs to have a full project, with settings.py and manage.py files, to be able to create the migrations..but they are not part of the app. – Vesicant

Hi, I'm not sure and I can't remember (my fault) how I got migrations works at the end. I think that I had to run a dummy migration, just for initialization purposes or re-create database with current schema, as a new begin of the project. I'm not sure, sorry. – Rema

Answer 1

Additionally, a good workflow during development is to link the reusable app into your Django project. To easily achieve this pip install has the -e, --editable option, which in turn derives from the setuptools Development mode.

Reusable app:

django-myApp
|--myApp
|  |--models
|     |--file1.py
|     |--file2.py
|--setup.py
|--README.rst

Django setup:

my-django-project
|--my_django_project
|  |--settings.py
|  |--urls.py
|  |--wsgi.py
|  |--...
|--manage.py

With your virtualenv activated you can now link your reusable app to the project by running:

(myvenv) $ pip install --editable /path/to/django-myApp

Now, every change that you make in django-myApp is automatically reflected in my-django-project without the need to build/package the reusable app first.

This becomes convenient in many use cases. E.g. imagine developing the app to be compatible with Python 2.x and Python 3.x. With linking, you can install the app into two (or more) different Django setups and run your tests.

Countersignature

Thanks for your response. Using pip develop option also allows my-django-project to use current changes in django-myApp. But when I make changes to models, it cause errors in project. How can I generate migration files for myApp? Because migrate and makemigrations commands do nothing at all when executed within my-django-project. – Rema

That should work as usual, i.e. you can run ./manage.py makemigrations <myapp> from a Django setup that runs under the same virtualenv that you have installed the app into. What error do you get and what OS are you using? – Countersignature

Sorry for the (very) late response. Your suggestion on using development mode with pip fit my needs, and I consider it as the correct answer for my question. I'm not sure how migrations finally worked. – Rema

Answer 2

1. When you package an app, is because you decided it was worth to use it across several projects and had some value on it's own. You can follow the guidelines to keep your app reusable and package once it is in a good state (minimal functionality working). Ok, you got your app ready, it does what you want, so now you want to separate it from your project? Follow the tutorial you linked for the guidelines on the folder structure and upload it to a repository. Add some tests too, they are a must have. You can now use a couple approaches to get it back into your project.

   • Package it and install it from the tar.gz via pip (or directly from repository)
   • Clone your repository as a submodule
   • Keep a copy in your project and update changes manually in the repository

I am not a fan of third one. Second option has its caveats but it can work. I definitely prefer first one.

Create a test project around your new app and use it to develop it's new features (with just the minimum requirements to test it), update your repository and finally update the app in your project via pip. Your app is now something that doesn't depend on your project, so don't tie your app to it.

2. Did you add it to your INSTALLED_APPS? If you did, check that your models.py is accesible...try with the django shell. Try creating the migrations module to see if it works, that doesn't take long anyway.

3. Finally the answer: No, your app is a different project on it's own now, you continue developing on a repository like you would do with any other project and then update like you would do with any other app that got a new version released. You don't need (nor should) touch your virtualenv's folders at all.

These are the steps for an update in your app:

• You found a nasty bug or thought of some nice feature
• You added the tests to cover it
• You updated your code to enhance your app
• You repeat the last couple steps until your app is stable again
• If your app is in Pypi, you release a new package and update it from there, otherwise you update it from the repository.

This obviously takes longer than developing an app inside of your project, but doing it this way helps ensuring a good quality and that it won't introduce problems to your projects.

Rauscher

Answer 3

I ran into a similar situation while developing this library (django-nsync).

I "solved" the issue by creating a makemigrations.py script that sets up a minimal Django settings to include the 'app' and then makes a call to the makemigrations command.

The script looks like this:

def make_migrations():
    from django.core.management import call_command
    call_command('makemigrations', '<YOUR APP NAME>')


if __name__ == '__main__':
    import sys
    # I need to add the 'src' folder to PYTHON PATH
    sys.path.append('./src/')

    try:
        from django.conf import settings

        settings.configure(
            INSTALLED_APPS=[
                '<ANY APPS YOUR APP DEPENDS ON>',
                '<YOUR APP NAME>',
            ],
        )

        import django
        django.setup()

    except ImportError:
        import traceback
        traceback.print_exc()
        raise ImportError('To fix this error, sort out the imports')

    make_migrations()

This script does not require a separate project in order to create the migrations.

NB: I tried to hook this into the setup.py stuff so that whenever you try to build (or more importantly release) the library it checks that the migrations are up to date / in source control. However, I gave up because:

1. I hopefully won't have to do it much for the NSync library.
2. It got messy quickly (like checking how many files in the migrations folder, then running the migrations, then checking again, then deleting the files? or leaving them? should I enforce this at release time? should I check with Git like bumpversion does? blah....)

Vesicant