How to Handle Runtime Configuration of Symfony2 Using Consul Service Discovery

Question

Our team is presently exploring the idea of service discovery for a Symfony2 application using Consul. Being in the relative frontier, there's very little out there in the way of discussion. So far we've discovered:

• Runtime configuration has previously been shot down.
• A bundle exists to handle such a use case, but has also hasn't seen a lot of activity as of late.
• One of the contributors of said bundle suggested that External Parameters might be the answer to the problem.
• Sensio has created its own Consul SDK. However, there seems to be little in the way of documentation/official blog articles re: Symfony2 integration
• Consol provides watches which can be triggered on various changes

Current thoughts are to explore utilizing the Consul watchers to re-trigger a cache build along with external parameters. That said, there is some concern on the overhead of such an operation if services change semi-frequently.

Based on the above, and knowledge of Consul/Symfony internals, would that be an advisable approach? If not, why, and what alternatives are available?

Chemesh

Answer 1

In the company I work, we took a different route.

Instead of fighting against Symfony to accept runtime configuration (something it should, like Spring Data Consul, for example), we decided to make Consul update Symfony configuration, in a similar in concept, different in implementation than Frank did.

We installed Consul and Consul Template. We create a K/V entry pair that contains the entire parameters.yml file. Example:

Key: eblock/config/parameters.yml

parameters:
    router.request_context.host: dev.eblock.ca
    router.request_context.scheme: http
    router.request_context.base_url: /

Then a consul template configuration file was added at location /opt/consul-template/config/eblock.cfg:

template {
    source = "/opt/consul-template/templates/eblock-parameters.yml.ctmpl"
    destination = "/var/www/eblock/app/config/parameters.yml"
    command = "/opt/eblock/scripts/parameters_updated.sh"
}

The contents of ctmpl file are:

{{key "eblock/config/parameters.yml"}}

Finally, our parameters_updated.sh script does:

#!/bin/bash

readonly PROGNAME=$(basename "$0")
readonly LOCKFILE_DIR=/tmp
readonly LOCK_FD=201

lock() {
    local prefix=$1
    local fd=${2:-$LOCK_FD}
    local lock_file=$LOCKFILE_DIR/$prefix.lock

    # create lock file
    eval "exec $fd>$lock_file"

    # acquire the lock
    flock -n $fd \
        && return 0 \
            || return 1
}

lock $PROGNAME || exit 0

export HOME=/root
logger "Starting composer install" && \
/usr/local/bin/composer install -d=/var/www/eblock/ --no-interaction && \
logger "Running composer dump-autoload" && \
/usr/local/bin/composer dump-autoload -d=/var/www/eblock/--optimize && \
logger "Running app/console c:c/c:w" && \
/usr/bin/php /var/www/eblock/app/console c:c -e=prod --no-warmup && \
/usr/bin/php /var/www/eblock/app/console c:w -e=prod && \
logger "Running doctrine commands" && \
/usr/bin/php /var/www/eblock/app/console doctrine:database:create --env=prod --if-not-exists && \
/usr/bin/php /var/www/eblock/app/console doctrine:migrations:migrate -n --env=prod && \
logger "Restarting php-fpm" && \
/bin/systemctl restart php-fpm &

Knowing that both consul and consul-template services are up, as soon as your value changes in the specified key for consul template, it'll dump the file into configured destination and run the command for parameters updated.

It works like a charm. =)

Wyck

Answer 2

A simple KV watcher that puts the value into parameters.yml, triggers a cache:clear is the simplest option in my opinion and also provides the benefit of compilation so that it doesn't have to go to Consul each time to check if values are updated. Like you said, some overhead but seems to be ok if you don't change your parameters every 5 minutes.

We're exploring that option now but if you made any progress on this, an update would be appreciated.

[Update 2016-02-23] We've implemented the idea I mentioned above and it works as expected: well. Mind you, we change our parameters only on deploy of a new version (because we also use service discovery by Consul so no need to update service lists in parameters). We mostly did it because it saves us the boring job of changing parameters on several servers. As usual: this might not work for you but I think you would be safe if, like I said before, you don't change your parameters every 5 mins :)

Extramarital

If you have a NEW question, please ask it by clicking the Ask Question button. If you have sufficient reputation, you may upvote the question. Alternatively, "star" it as a favorite and you will be notified of any new answers. – Rhizogenic

I might misunderstand but this for me is a question relevant to this topic. I'm asking for an update on his own question, not for anyone else to answer it perse (although that would be nice) – Extramarital

So this is a comment then, and comments may not be posted as answers on StackOverflow. Users earn the privilege of commenting by participating through questions, answers and editing activities. – Rhizogenic

Can't agree with you more but since I don't have the points, I can't and also it seems kind of silly to first 'earn points' to be able to stick to the rules? – Extramarital

Please read the link in my previous comments. It explains why. – Rhizogenic

@Extramarital it is kind of a chicken-egg scenario, comments. I had a bit of challenge with that myself. To your question, we have not continued exploring this as of yet. It got put on the backburner because our Consul values are remaining pretty static for the moment. If you end up going your suggested route and update this answer w/ your findings, I think it could prove a great answer more in line w/ what Tunaki is mentioning. I think removing the initial paragraph and cleaning up the second as more statement than question get a good part of the way there. – Chemesh

So I hope everyone is pleased with this chicken so I can go place some eggs around :) Also, seriously, is it really necessary for four people to place all kinds of comments about how wrong a post is? Doesn't make StackOverflow seem like a very welcoming place. Just my 2 cents – Extramarital