Django run tasks (possibly) in the far future

Question

Suppose I have a model Event. I want to send a notification (email, push, whatever) to all invited users once the event has elapsed. Something along the lines of:

class Event(models.Model):
    start = models.DateTimeField(...)
    end = models.DateTimeField(...)
    invited = models.ManyToManyField(model=User)

    def onEventElapsed(self):
        for user in self.invited:
           my_notification_backend.sendMessage(target=user, message="Event has elapsed")

Now, of course, the crucial part is to invoke onEventElapsed whenever timezone.now() >= event.end. Keep in mind, end could be months away from the current date.

I have thought about two basic ways of doing this:

1. Use a periodic cron job (say, every five minutes or so) which checks if any events have elapsed within the last five minutes and executes my method.

2. Use celery and schedule onEventElapsed using the eta parameter to be run in the future (within the models save method).

Considering option 1, a potential solution could be django-celery-beat. However, it seems a bit odd to run a task at a fixed interval for sending notifications. In addition I came up with a (potential) issue that would (probably) result in a not-so elegant solution:

• Check every five minutes for events that have elapsed in the previous five minutes? seems shaky, maybe some events are missed (or others get their notifications send twice?). Potential workaroung: add a boolean field to the model that is set to True once notifications have been sent.

Then again, option 2 also has its problems:

• Manually take care of the situation when an event start/end datetime is moved. When using celery, one would have to store the taskID (easy, ofc) and revoke the task once the dates have changed and issue a new task. But I have read, that celery has (design-specific) problems when dealing with tasks that are run in the future: Open Issue on github. I realize how this happens and why it is everything but trivial to solve.

Now, I have come across some libraries which could potentially solve my problem:

• celery_longterm_scheduler (But does this mean I cannot use celery as I would have before, because of the differend Scheduler class? This also ties into the possible usage of django-celery-beat... Using any of the two frameworks, is it still possible to queue jobs (that are just a bit longer-running but not months away?)
• django-apscheduler, uses apscheduler. However, I was unable to find any information on how it would handle tasks that are run in the far future.

Is there a fundemantal flaw with the way I am approaching this? Im glad for any inputs you might have.

Notice: I know this is likely to be somehwat opinion based, however, maybe there is a very basic thing that I have missed, regardless of what could be considered by some as ugly or elegant.

Rotberg

I would say your approach depends on how soon after the elapsed event the end user needs notification. I have had a similar problem in which the user only needed to know the following day for any appointment missed the previous day. So in this case I ran a cron job at midnight, and had, as you suggested, a boolean field to tag whether notifications had been sent. It was a very simple and computationally inexpensive way to do it. – Nannienanning

In my opinion, the answer is about how many events you need to send. If you have hundreds of events to be sent every day, it doesn't matter how far in the future is a single event: using the first solution (adapting the recurrence time basing on your needs) you can run the task reading updated data. – Daunt

@HaydenEastwood Its not crucial that the person recieves it immediately, but within 2-5 minutes within the end date should be fine. So you did something similar to my opion 1? – Rotberg

@Daunt so you would option 1? Potentially there could be hundreds of events per day, yes. – Rotberg

@Rotberg Yes, I think option 1 is a good solution for your case. As usual, it is a tradeoff between consistency (read the correct date when start/end datetime is moved) and availability (the task is delayed until the next job run). – Daunt

In addition, scheduling tasks in a dbase that are potentially months away could be memory-heavy... Do you have any recommendation regarding frameworks? especially using something celery based vs apscheduler? – Rotberg

@Rotberg Yes - I think a simple cron call with a field in the database for whether message was sent would be a good fit for your case. – Nannienanning

Thank you both. When I finish this, I will post an update with the solution I came up with. – Rotberg

Dramatiq use another aproach than Celery when sheduling tasks (not memory hungry on worker) and could works in your case, see dramatiq.io/guide.html#scheduling-messages. But as they say -- message broker isn't DB -- when you need planning of long-term event your first solution is better. So you can combine both: put events in MB, say by 1 day, and by expiration they would goes to DB and will be sended via cron. – Corpuz

Yes I already looked ad dramatiq as a possible solution. AFAIK its simply a better celery, right? – Rotberg

Answer 1

We're doing something like this in the company i work for, and the solution is quite simple.

Have a cron / celery beat that runs every hour to check if any notification needs to be sent. Then send those notifications and mark them as done. This way, even if your notification time is years ahead, it will still be sent. Using ETA is NOT the way to go for a very long wait time, your cache / amqp might loose the data.

You can reduce your interval depending on your needs, but do make sure they dont overlap.

If one hour is too huge of a time difference, then what you can do is, run a scheduler every hour. Logic would be something like

1. run a task (lets call this scheduler task) hourly that gets all notifications that needs to be sent in the next hour (via celery beat) -
2. Schedule those notifications via apply_async(eta) - this will be the actual sending

Using that methodology would get you both of best worlds (eta and beat)

Antinomian