django models without database

Question

I know the automatic setting is to have any models you define in models.py become database tables.

I am trying to define models that won't be tables. They need to store dynamic data (that we get and configure from APIs), every time a user searches for something. This data needs to be assembled, and then when the user is finished, discarded.

previously I was using database tables for this. It allowed me to do things like "Trips.objects.all" in any view, and pass that to any template, since it all came from one data source. I've heard you can just not "save" the model instantiation, and then it doesn't save to the database, but I need to access this data (that I've assembled in one view), in multiple other views, to manipulate it and display it . . . if i don't save i can't access it, if i do save, then its in a database (which would have concurrency issues with multiple users)

I don't really want to pass around a dictionary/list, and I'm not even sure how i was do that if I had to.

ideas?

Thanks!

Kailey

Somebody smart once said something about premature optimization... how did it go? It isn't obvious what problem you're trying to solve here. You might not even have one! I'd say use Django as it comes, love it for who it is, and come back here with specific problems if you have them. – Opium

This question has lot of merit, not all apps use or need database as a backend and yet they may require model approach. It is not clear how to disable db persistence in Django. – Esthonia

Answer 1

Another option may be to use:

class Meta:
    managed = False

to prevent Django from creating a database table.

https://docs.djangoproject.com/en/2.2/ref/models/options/#managed

Mesitylene

The poster mentioned he wants to use a Django model without it being backed by a table. The Django documentation you linked here specifically mentions that only the migration and deletion operations will not be performed, while all other table-related operations will continue to work as if the table exists. – Ileneileo

This is not accurate. Models with managed = False will still create tables. – Lonna

@zepp.lee Maybe you were referring to an older version, but the current docs explicitly say "If [managed =] False, no database table creation or deletion operations will be performed for this model. This is useful if the model represents an existing table or a database view that has been created by some other means." (docs.djangoproject.com/en/3.1/ref/models/options/#managed). So the model's functionality assumes that there is a database table mapped to the model for querying, etc., but it doesn't do anything to ensure the database table's existence (i.e. doesn't manage it for you) – Recuperative

Answer 2

Just sounds like a regular Class to me.

You can put it into models.py if you like, just don't subclass it on django.db.models.Model. Or you can put it in any python file imported into the scope of whereever you want to use it.

Perhaps use the middleware to instantiate it when request comes in and discard when request is finished. One access strategy might be to attach it to the request object itself but ymmv.

Dreibund

Where do I best put this regular class? models.py? – Lactic

Using models instead of regular classes can sometimes be advantageous, i.e. when using something like django-rest-swagger to generate API docs. – Phillada

models.py is an appropriate place for such a model – Phillada

@jakob In python you can put the classes in any .py file. To use a class import it. That's what all those from mymodule import MyClass statements are doing. Checkout packages – Dreibund

We're using this option, gathering regular classes in a module called domain_models.py – Quantum

How to serialize a regular class? – Indevout

Answer 3

Unlike SQLAlchemy, django's ORM does not support querying on the model without a database backend.

Your choices are limited to using a SQLite in-memory database, or to use third party applications like dqms which provide a pure in-memory backend for django's ORM.

Onward

Answer 4

So this is 11 years ago, but I ran into the exact same problem as the original poster and came up with a clever solution I thought I'd share:

models.py:

import requests
from collections import UserList
from django.core.cache import caches
from django.db import models

CACHE = caches["default"]

class MyTransientModelManager(models.Manager):
    cache_key = "cached-transient-models"
    cache_sentinel = object()
    cache_timeout = 60 * 10

    def get_queryset(self):
        transient_models_data = CACHE.get(self.cache_key, self.cache_sentinel)
        if transient_models_data is self.cache_sentinel:
            response = requests.get("some/remote/api")
            response.raise_for_status()
            transient_models_data = response.json()            
            CACHE.set(self.cache_key, transient_models_data, self.cache_timeout)

        return MyTransientModelQueryset([
            MyTransientModel(**data)
            for data in transient_models_data
        ])


class MyTransientModelQueryset(UserList):
    # custom filters go here
    pass


class MyTransientModel(models.Model):
    class Meta:
        managed = False

    objects = MyTransientModelManager.from_queryset(MyTransientModelQuerySet)()

    id = models.IntegerField(primary_key=True)
    foo = models.CharField(max_length=255)
    bar = models.TextField(null=True)

serializers.py:

from rest_framework import serializers
from my_app.models import MyTransientModel

class MyTransientModelSerializer(serializers.ModelSerializer):
    class Meta:
        model = MyTransientModel
        fields = ["id", "foo", "bar"]

views.py:

from rest_framework.exceptions import APIException
from rest_framework.generics import ListAPIView
from rest_framework.permissions import AllowAny

from my_app.models import MyTransientModel
from my_app.serializers import MyTransientModelSerializer


class MyTransientModelView(ListAPIView):
    permission_classes = [AllowAny]
    serializer_class = MyTransientModelSerializer

    def get_queryset(self):
        try:
            queryset = MyTransientModel.objects.all()
            return queryset
        except Exception as e:
            raise APIException(e) from e

Commons

This seems like exactly what I need. But it seems like you have 2 repeated MyTransientModelManager classes. Is the second one supposed to be MyTransientModelQuerySet? – Humph

@Humph - Yes you are correct. I have fixed my error. Glad my answer helped you. – Commons

Also MyTransientModelManager.get_queryset() should return MyTransientModelQuerySet() – Humph

@Humph - D'oh! You're right again. I guess I made a bunch of copy-and-paste errors. I've fixed the code again. – Commons

Answer 5

Use Django's cache framework to store data and share it between views.

Harvest

Answer 6

Try to use database or file based sessions.

Divaricate

Already tried, if you store instances of a model in a session, you can retrieve it, but you can't use all the normal methods on the model . . . for instance "order_by" . . . i suppose that is because those translate to sql and we don't have a database anymore, but still . . perhaps i'm searching for something that doesn't exist – Kailey

Answer 7

You need Caching, which will store your data in Memory and will be seperate application.

With Django, you can use various caching backend such as memcache, database-backend, redis etc. Since you want some basic query and sorting capability, I would recommend Redis. Redis has high performance (not higher than memcache), supports datastructures (string/hash/lists/sets/sorted-set).

Redis will not replace the database, but will fit good as Key-Value Database Model, where you have to prepare the key to efficiently query the data, since Redis supports querying on keys only.

For example, user 'john.doe' data is: key1 = val1
The key would be - john.doe:data:key1
Now I can query all the data for for this user as - redis.keys("john.doe:data:*")

Redis Commands are available at http://redis.io/commands

Django Redis Cache Backend : https://github.com/sebleier/django-redis-cache/

Kickshaw

Answer 8

Delete migration directory

If you delete the migrations directory of the app, django ignores creating migrations for this app. Migration related functionality like permissions are then not working anymore for the whole app. This way you can create ModelForms, DRF Serializers or such from a Model that has not tables in database.

This is not a answer to the question but for the title. Often i stumbled upon this thread and maybe it can save someones time.

Jeromyjerreed

Answer 9

I make my bed to MongoDB or any other nosql; persisting and deleting data is incredibly fast, you can use django-norel(mongodb) for that.

http://django-mongodb.org/

Eldenelder

This is different from what is being asked. – Urita