How to automatically assign a publisher role and subscriber role on dead letter subscription with pubsub?

Question

I'm upgrading a queue processing system that previously used RabbitMQ.

I'm currently importing the dead letter and exponential backoff functionality we had in place with RabbitMQ, however I seem to be encountering a couple of issues.

The primary issue being, that when I create a new subscription with a dead letter policy, it seem to not have the associated permissions required in order for Google to forward messages to my dead letter subscription.

As seen below when inspecting the subscription details, google has highlighted that the pubsub service account for the project needs the publisher role and the subscriber role in order to publish and forward to the dead letter topic.

The option is given in the UI to add these, however I need to have this handled entirely via my consumer that I'm running, as this needs to be an automated process as these consumers are run by SupervisorD in the background.

Does anyone have any idea what exactly I'm missing, I've gone through all the documentation, but it's not entirely clear, I've tried adding the permissions mentioned in IAM to the associated member, but no dice.

Zamir

Answer 1

In order to forward messages to dead-letter topics, Pub/Sub must have permission to do the following:

• Publish messages to the dead-letter topic.
• Acknowledge forwarded messages, which removes them from the subscription.

If you want to grant the required permissions per docummentation via the consumer that you are running, I recommend by doing it via the Cloud SDK by first adding the publisher role:

PUBSUB_SERVICE_ACCOUNT="service-${project-number}@gcp-sa-pubsub.iam.gserviceaccount.com"

gcloud pubsub topics add-iam-policy-binding dead-letter-topic-id \
    --member="serviceAccount:$PUBSUB_SERVICE_ACCOUNT"\
    --role="roles/pubsub.publisher"
    

And then, the subscriber role:

PUBSUB_SERVICE_ACCOUNT="service-${project-number}@gcp-sa-pubsub.iam.gserviceaccount.com"

gcloud pubsub subscriptions add-iam-policy-binding subscription-id \
    --member="serviceAccount:$PUBSUB_SERVICE_ACCOUNT"\
    --role="roles/pubsub.subscriber"
    

With that permissions, you can now track the delivery attempts using the language of your preference, for example, for Python:

from concurrent.futures import TimeoutError
from google.cloud import pubsub_v1

# TODO(developer)
# project_id = "your-project-id"
# subscription_id = "your-subscription-id"

subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(project_id, subscription_id)

def callback(message):
    print("Received message: {}".format(message))
    print("With delivery attempts: {}".format(message.delivery_attempt))
    message.ack()

streaming_pull_future = subscriber.subscribe(subscription_path, callback=callback)
print("Listening for messages on {}..\n".format(subscription_path))

# Wrap subscriber in a 'with' block to automatically call close() when done.
with subscriber:
    # When `timeout` is not set, result() will block indefinitely,
    # unless an exception is encountered first.
    try:
        streaming_pull_future.result(timeout=timeout)
    except TimeoutError:
        streaming_pull_future.cancel()

Shepard

Answer 2

According with the documentation, you have to grant the Service Agent (sa) Pubsub service account of the project that host the source topic with the role Pubsub Publisher on the dead letter topic.

-> Authorize the source project to publish a message in the Dead letter topic

You also have to grant the Service Agent (sa) Pubsub service account of the project that host the deadletter topic with the role Pubsub Subscriber on the souscription on which the dead letter feature is configured.

-> Authorize to acknowledge the Dead letter message posted in the dead letter topic.

here the pattern of the Service Agent PubSub service account

service-${project-number}@gcp-sa-pubsub.iam.gserviceaccount.com

Frondescence

can you please help me on this question – Sampan

cloud.google.com/pubsub/docs/… – Barite

Answer 3

I was investigating this as I also needed the roles to be granted automatically on the consumer side.

I found that this is possible to achieve from code using gcloud API. I am ensuring the roles are granted when creating the subscription on the consumer side. Sharing my c# implementation:

private async Task GrantDeadLetteringPublisherRole()
{
    var topic = TopicName.FromProjectTopic(_projectId, _deadLetterTopicName);
            
    var deadletterTopicResource = $"projects/{_projectId}/topics/{topic.TopicId}";
            
    var publisherServiceApiClient = await PublisherServiceApiClient.CreateAsync();
    
    var policy = await publisherServiceApiClient.IAMPolicyClient.GetIamPolicyAsync(
        new Google.Cloud.Iam.V1.GetIamPolicyRequest
        {
            Resource = deadletterTopicResource
        });
    
    if (policy.Bindings.All(x => x.Role != "roles/pubsub.publisher"))
    {
        policy.Bindings.Add(new Google.Cloud.Iam.V1.Binding
        {
            Role = "roles/pubsub.publisher",
            Members = { $"serviceAccount:service-{_projectNumber}@gcp-sa-pubsub.iam.gserviceaccount.com" }
        });

        await publisherServiceApiClient.IAMPolicyClient.SetIamPolicyAsync(
            new Google.Cloud.Iam.V1.SetIamPolicyRequest
            {
                Resource = deadletterTopicResource,
                Policy = policy
            });

        Console.WriteLine("Publisher policy updated");
    }
    else
    {
        Console.WriteLine("Publisher policy already up to date");
    }
}

private async Task GrantDeadLetteringSubscriberRole(SubscriptionName subscriptionName)
{
    var subscriberServiceApiClient = await SubscriberServiceApiClient.CreateAsync();
    
    var subscriptionResource = $"projects/{_projectId}/subscriptions/{subscriptionName.SubscriptionId}";

    var policy = await subscriberServiceApiClient.IAMPolicyClient.GetIamPolicyAsync(
        new Google.Cloud.Iam.V1.GetIamPolicyRequest
        {
            Resource = subscriptionResource
        });

    if (policy.Bindings.All(x => x.Role != "roles/pubsub.subscriber"))
    {
        policy.Bindings.Add(new Google.Cloud.Iam.V1.Binding
        {
            Role = "roles/pubsub.subscriber",
            Members = { $"serviceAccount:service-{_projectNumber}@gcp-sa-pubsub.iam.gserviceaccount.com" }
        });

        await subscriberServiceApiClient.IAMPolicyClient.SetIamPolicyAsync(
            new Google.Cloud.Iam.V1.SetIamPolicyRequest
            {
                Resource = subscriptionResource,
                Policy = policy
            });

        Console.WriteLine("Subscriber policy updated");
    }
    else
    {
        Console.WriteLine("Subscriber policy already up to date");
    }
}

Relevant: How to add subscriber role and publisher role to deadletter for google cloud pubsub using nodeJS?

Barite