Shared XMPP connection between Celery workers

Question

My web app needs to be able to send XMPP messages (Facebook Chat), and I thought Celery might be a good solution for this. A task would consist of querying the database and sending the XMPP message to a number of users. However, with that approach I would have to connect to the XMPP server every time I run a task, which is not a great idea.

From the Facebook Chat API docs:

Best Practices

• Your Facebook Chat integration should only be used for sessions that are expected to be long-lived. Clients should not rapidly churn on and off.

Is there a way to share an XMPP connection between workers so I don't have to reconnect every time I want to send a message? Or, is there a better solution?

Omentum

Answer 1

You can create a connection globally in your celery task module and use it from your tasks to send messages. In that case the connection will be established at start-up and will be shared between worker processes.

import socket 
from celery.task import task

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
s.connect(('localhost', 9999)) 

@task
def echo(arg):
    s.send(arg) 
    return s.recv()

Kitty

Yeah, this is the idea I'm going for. Thanks. – Omentum

Answer 2

How about a long-running background job whose job it would be to receive messages from other short-lived processes and push them onto an XMPP socket?

Eridanus