What's the process of storing the configuration for a 12-factor application?

Question

I have been building my application mostly as a 12-factor application and am now looking at the configuration part.

Right now as it stands, I have separate configuration files for development and production and through the build process, we either build a development or production image. The code is 100% the same; the only thing that changes is the configuration.

Now I 100% understand that in a 12-factor application the configuration should come from external sources, such as: environment variables, or maybe a safe store like a vault, etc.

So what the various articles and blogs fail to mention about the configuration is how the configuration is stored/processed. If the code is separated in its own Git repository and it doesn't have any configuration stored with it, then how do we handle the configuration?

Do we store the actual configuration values in a separate Git repository and then somehow merge/push/execute those on the target environment (Kubernetes configuration map, marathon JSON configuration, Vault, etc...) through the build process using some kind of trigger?

Reckon

Answer 1

There is not a standard, but I've been observing some common behaviors like:

1. Sensitive information never gets on the versioning system, especially Git which is a DVCS (you can clone the repository for other locations). If you don't follow, remember that our existing "security system" is based on the incapacity of reading cryptographic information in a certain time, but at a certain point you might be able to read the information. Usually on Kubernetes I see operators, managing the service account across multiple namespaces and then other only referring the service account, tools like KMS, Cert manager, Vault and etc. are welcome

2. Configuration like environment variables, endpoints, are stored and versioned with their own "lifecycle".

12factor does not mean to separate the configuration of your application from your repository. Instead, suggesting not to put it into your application (like in your container or even binary distribution).

In fact, if you want to use a separate repository only for configuration you can do it, but if you want to put aside your project source code the configuration, you can do it as well. It is more a decision based on the size of the project, complexity, segregation of duties and team context. (IMHO)

In my case of study, for instance, it makes sense to separate the configuration in a dedicated repository as the production environment has more than 50 clusters, each one with their own isolation stack. Also, there are different teams managing their own services and using common backing services (database, API, streams, etc.). In my opinion, as long as things gets more complex and cross-shared, it makes more sense to separate the configuration in an independent repository, as there are several teams and resources over multiple clusters.

Unwilling

Currently right now, we have buildImageDev and buildImageProd tasks which uses conf-dev.json and conf-prod.json and that lives with the code. Code is 100% the same only the config file name changes. But there have been cases where people have made cut and paste mistakes of copying dev values into prod values. Other option is to do the same as we treat 3rd party applications and use separate repos for holding configs for each one of those apps. Examples: Kafka, Elasticsearch etc... So each of those configs have their own repos. So we can treat our apps that way too. – Reckon

In that case I would suggest you, on your image spec (dockerfile) describe a VOLUME to put your config file, makes the app (process) read that, otherwise assume default values And during orchestration, mount a configmap by the key on your podSpec, and makes a configmap different for each environment. Optionally if your app supports envvar, try to stick on envvar parametrization over config file, as it might give you a little extra flexibility – Unwilling

Yeah right now I copy the config file into the image. But where would you store all the configmap values, same repo or separate repo? – Reckon