From the OP:
I haven't been able to find documentation on what the JSON message should look like. There are some HTTP headers mentioned in the official documentation. But there's no mention of header to specify the desired endpoint in the worker environment.
summary
Use SQS.Client.send_message() from boto3 to send messages to the SQS queue.
- The SQS message can look like anything you want (not necessarily JSON).
- You do not need to worry about the HTTP headers, unless you want to.
- You can specify a single endpoint in the worker environment configuration using
HTTP path. Afaik only tasks from a cron.yaml can specify alternative endpoints using the path value.
At the endpoint you can apply logic based on the request body and/or headers to distinguish tasks.
details
The worker docs are indeed sparse and a bit confusing.
Here's what I've been able to deduce (disclaimer: there may be some inaccuracies):
HTTP headers and message content
The docs mention a lot of HTTP headers, and these are also described in the answers by user3643324 and Jun.
These HTTP headers are set by the SQS Daemon (sqsd). You can use them in your perform_job(), if you want, but you do not have to.
More importantly, you do not need to set these headers yourself when you send a custom message to the worker's SQS queue in your trigger_worker_job().
In fact, the sqsd picks up all messages from the worker queue, regardless of content. The content does not need to be JSON either.
Here's a relevant snippet from the docs:
[...] When the daemon pulls an item from the queue, it sends an HTTP POST request locally to http://localhost/ on port 80 with the contents of the queue message in the body. All that your application needs to do is perform the long-running task in response to the POST. You can configure the daemon to post to a different path, use a MIME type other than application/JSON, connect to an existing queue, or [...]
normal messages
Basically, any message sent to the SQS queue, by anyone, will be pulled down by the worker's sqsd. Note that an SQS message has a body and optional attributes.
In the simple case of a message without any attributes, the message body is posted to localhost at the path specified under HTTP path in the worker environment configuration. The default path is /, but in the OP's case that would become /perform_job.
If the SQS message does contain attributes, those are added to the POST request as custom HTTP headers, as mentioned in the docs and the other answers.
messages scheduled via cron.yaml
If your app source includes a cron.yaml, that's a special case. Each task in cron.yaml specifies a path, which can be different from the worker environment's HTTP path value. The sqsd adds this path as an attribute to the SQS message when sending to the SQS queue. If the same (or another) sqsd pulls that message down from the SQS queue, it will POST the message body to the specified path, with the cron-related HTTP headers mentioned in the docs.
simplified example
So, the OP's example can be adjusted as follows, using send_message() from boto3:
sqs_client = boto3.client('sqs')
@post('/trigger_job')
def trigger_worker_job():
# send a JSON message to sqs to trigger the '/perform_job'
result = sqs_client.send_message(
QueueUrl='https://<my worker queue url>',
MessageBody='Would you please perform a job for me?',
)
# check `result` if necessary...
...
return HTTPResponse(status=200, body={'Msg': 'Sent message'})
and on the worker side, assuming HTTP path for the worker env is configured as /perform_job:
@post('/perform_job')
def perform_job():
request_body = ... # get the body from the POST request somehow
if request_body == 'Would you please perform a job for me?':
# perform the actual job
...
return HTTPResponse(status=200, body={'Msg': 'Success'})
If necessary, you can adjust the logic to test for certain HTTP headers.
Also see how it's done in the elastic beanstalk python sample app: python.zip
