Can you use environment variables in config file for fluentd

Question

I was wondering how to use env vars in the Fluentd config, I tried:

<match **>
type elasticsearch
logstash_format true
logstash_prefix $ENV_VAR
host ***
port ***
include_tag_key true
tag_key _key
</match>

but it doesn't work, any idea?

Mistranslate

Answer 1

EDIT:

Here is a much better solution:

If you pass "--use-v1-config" option to Fluentd, this is possible with the "#{ENV['env_var_name']" like this:

<match foobar.**> # ENV["FOO"] is foobar
  type elasticsearch
  logstash_prefix "#{ENV['FOO']}"
  logstash_format true
  include_tag_key true
  tag_key _key
  host ****
  port ****
</match>

Old, kludgey answer is here.

1. Install fluent-plugin-record-reformer and fluent-plugin-forest
2. Update your config as follows.

<match hello.world>
  type record_reformer
  tag ${ENV["FOO"]}.${tag_prefix[-1]} # adding the env variable as a tag prefix
</match>

<match foobar.**> # ENV["FOO"] is foobar
  type forest
  subtype elasticsearch
  <template>
    logstash_prefix ${tag_parts[0]}
    logstash_format true
    include_tag_key true
    tag_key _key
    host ****
    port ****
  </template>
</match>

In particular, do NOT use <match **> there. That would catch all events and will lead to behaviors that are hard to debug.

Bookstack

From groups.google.com/forum/#!topic/fluentd/z-1vIsQ4kHU you need to keep the environment variables for --use-v1-config in /etc/default/td-agent like this: export FOO="foobar" – Chthonian

/etc/default/td-agent is for debian family. For redhat family, use /etc/sysconfig/td-agent instead. – Omentum

--use-v1-config is now the default setting, so you don't have to specify it manually. – Martel

Found this today. Fluentd FAQ and search for "How can I use environment variables" – Pierpont