Define RabbitMQ policies in configuration file

Question

I'd like to define mirroring for all my queues by default. I currently have to use rabbitmqctl once the node is up:

rabbitmqctl set_policy ha-all "" '{"ha-mode":"all"}'

If one of my nodes fail, I'd have to remember to re-execute that code on restart.

Is there a way to automatically configure my node to use mirrored queues?

Tramway

Answer 1

Policy CAN be specified in a definition file, which can be referred to from your config file.

Example of how I have set a specific policy (not sure if ha can be specified in policy):

/etc/rabbitmq/rabbitmq.config

[
{rabbit,
    [{vm_memory_high_watermark, 0.8}]
},
{rabbitmq_management,
    [{listener, [{port, 15672}]},
     {load_definitions, "/etc/rabbitmq/rabbitmq_definitions.json"},
     {http_log_dir, "/var/log/rabbitmq/management_http.log"}]
}
].

/etc/rabbitmq/rabbitmq_definitions.json

{       "users":[
            {"name":"rabbot","password_hash":"Cvse5iGOg20UqUq7Za9D1tatOJnMVDru4GHtxqc02g7zj5ur","tags":""},
            {"name":"rabnet","password_hash":"CqqG2fwvH6xz64NpibGJx2M7ZCyFnR1BQBM+C0KH2qRPmVxF","tags":"administrator"}],
    "vhosts":[
            {"name":"/"}],
    "permissions":[
            {"user":"viabot","vhost":"VIA","configure":".*","write":".*","read":".*"},
            {"user":"vianet","vhost":"VIA","configure":".*","write":".*","read":".*"}],
    "parameters":[],
    "policies":[
            {"vhost":"VIA","name":"DLX","pattern":".*","apply-to":"queues","definition":{"dead-letter-exchange":"dead_letter"},"priority":0}
            ],
    "queues":[
            {"name":"store_to_es","vhost":"VIA","durable":true,"auto_delete":false,"arguments":{}},
            {"name":"store_to_mongodb","vhost":"VIA","durable":true,"auto_delete":false,"arguments":{}}
            ],
    "exchanges":[
            {"name":"data_incoming","vhost":"VIA","type":"fanout","durable":true,"auto_delete":false,"internal":false,"arguments":{}},
            {"name":"sms_incoming","vhost":"VIA","type":"fanout","durable":true,"auto_delete":false,"internal":false,"arguments":{}}
            ],
    "bindings":[
            {"source":"data_incoming","vhost":"VIA","destination":"store_to_es","destination_type":"queue","routing_key":"","arguments":{}},
            {"source":"sms_incoming","vhost":"VIA","destination":"store_to_mongodb","destination_type":"queue","routing_key":"","arguments":{}}
    ]
}

I am sharing this config file and definitions file as it was impossible to figure it out from the RabbitMQ web site.

Note: This config worked on RabbitMQ 3.6.1 running on Ubuntu 14.04  

Outfight

Answer 2

To add more details to IvanD's answer, this is how I did it.

First: sudo nano /etc/rabbitmq/rabbitmq.config (this command might be different depending on your OS)

[
  {rabbit,
    [
        {default_vhost,<<"/">>},
        {default_user,<<"someuser">>},
        {default_pass,<<"somepassword">>},
        {default_permissions, [<<".*">>, <<".*">>, <<".*">>]},
        {default_user_tags, [administrator]}
    ]
  },
  {rabbitmq_management,
    [{listener, [{port, 15672}]},
        {load_definitions, "/etc/rabbitmq/rabbitmq_definitions.json"},
        {http_log_dir, "/var/log/rabbitmq/management_http.log"}]
  }
].

Then create the additional json: sudo nano /etc/rabbitmq/rabbitmq_definitions.json (this command might be different depending on your OS). Its content:

{
  "vhosts":[
        {"name":"/"}
  ],
  "policies":[
        {"vhost":"/","name":"ha","pattern":"", "definition":{"ha-mode":"all","ha-sync-mode":"automatic","ha-sync-batch-size":5}}
  ]
}

IMPORTANT NOTE: The ha-sync-batch-size is only supported in RabbitMQ versions above 3.6.0! https://www.rabbitmq.com/ha.html#sync-batch-size If your Rabbit is older than that, remove the setting from the rabbitmq_definitions.json.

I'm using Ubuntu 14.04 Trusty and RabbitMQ v.3.6.2.

Te

Answer 3

Yes, You can load policies, queues, exchanges, bindings, users and much more directly on restart using definition import at load time.

1. Preconfigure you Rabbit

   Go to http://localhost:15672 Admin -> Policies page and create a new policy you need:

   You may also preconfigure queues and other things.

2. Dump definitions

   curl -s -H "Accept:application/json" -u guest:guest http://localhost:15672/api/definitions > definitions.json

   You can also dump your definitions using web interface. Open Overview tab, scroll down:

   (!!!) Alter definitions.json, so as to keep things you need on restart only.

   There will be section withing your policies, keep it:

   ...
   "policies": [
     {
       "vhost": "/",
       "name": "ha-all",
       "pattern": "",
       "apply-to": "all",
       "definition": {
         "ha-mode": "all"
       },
       "priority": 0
     }
   ]
   ...
   

3. Load definitions on restart

   Put that definitions.json nearby your rabbit and add this line to rabbit.conf. No need to use old format:

   management.load_definitions = /path/to/definitions.json
   

So as to load something after restart use cli tools.

---

Paying tribute to @IvanD since my answer is pretty much the same as his own but reveals detailed steps and new config format usage. Not enough space in comment.

Allene

Answer 4

Policy can't be set in the rabbitmq.config file. One workaround is to start rmq using an init script and put the rabbitmqctl command inside there so that it is run whenever rmq starts or restarts.

Equivalence

Answer 5

Finally I found something which worked: No need of configMap or rabbitmq.config files. Under

containers:
...
...
...

    lifecycle:
              postStart:
                exec:
                  command: ["/bin/sh","-c","rabbitmq-plugins --offline enable rabbitmq_management;until rabbitmqctl node_health_check; do  sleep 5;done;rabbitmqctl set_policy ha-all \".\" '{\"ha-mode\":\"all\", \"ha-sync-mode\":\"automatic\"}' --apply-to all --priority 0;"]

Largent