I'm trying to tail multiple logs in fluentd with the following configuration: <source> type tail tag es.workers.worker1 format /^\[(?<timestamp>.*? .*?) (?<log_level>[INFO|E...

I have a Pod with two containers. apiVersion: v1 kind: Pod metadata: name: test spec: containers: - name: my-container image: google/my-container:v1 - name: third-party image: google/third-p...

Remove time and tag from fluentd output plugin stdout with json Fluentd's output plugin produces output like: 2017-11-28 11:43:13.814351757 +0900 tag: {"field1":"value1","field2":"value2"} So ti...

I am new to fluentd. I have configured the basic fluentd setup I need and deployed this to my kubernetes cluster as a daemon set. I'm seeing logs shipped to my 3rd party logging solution. However I...

I want make a log management system through EFK. I'm trying for days now to get my multiline mycat log parser to work with fluent-bit. My fluentbit configuration: parsers.conf: [PARSER] Name m...

Objective: Get some logging/monitoring on Googles Stackdriver from a Kuberntes HA cluster that is on premises, version 1.11.2. I have been able to send logs to Elasticsearch using Fluentd Daemonse...

I'm using Fluentd to transfer the data into Elasticsearch. td-agent.conf ## ElasticSearch <match es.**> type elasticsearch target_index_key @target_index logstash_format true flush_in...

I have logs like the following: { "log": { "header": { "key": "value", "nested": "{\"key1\":\"value\",\"key2\":\"value\"}", "dateTime": "2019-05-08T20:58:06+00:00" }, "body": { "path": "/r...

I have deployed Elastic-search container in aws using eks kubernetes cluster. The memory usage of the container keeps increasing even though there are only 3 indices and not used heavily. I am dump...

I'd like to parse ingress nginx logs using fluentd in Kubernetes. That was quite easy in Logstash, but I'm confused regarding fluentd syntax. Right now I have the following rules: <source> ...

I'm a bit confused at how to setup error reporting in kubernetes, so errors are visible in Google Cloud Console / Stackdriver "Error Reporting"? According to documentation https://cloud.google.com...

I'm using the image gcr.io/google-containers/fluentd-elasticsearch (v2.3.1) in order to make fluentd collect some logs and send them to Elastic search. I'm using the below configuration for fluentd...

I'm using Fluentd with Elasticsearch for logs from Kubernetes but I noticed that some JSON logs cannot be correctly indexed because JSON is stored as string. Logs from kubectl logs look like: {"t...

I am trying to run fluentd as a daemonset on kubernetes cluster (GKE). The config is getting parsed successfully, then the plugins are receiving shutdown signal with few warn messages. There are no...

I have a Fluentd instance, and I need it to send my logs matching the fv-back-* tags to Elasticsearch and Amazon S3. Is there a way to configure Fluentd to send data to both of these outputs? Righ...

I am using fluentd daemonset to get kubernetes logs to Elasticsearch/Kibana which is working fine. Now the problem is that there are 3 4 application running in kubernetes which have different log p...

The use case is this: I've several java applications running which all have to interact with different (each one has a specific target) elasticsearch indices. For instance an application A uses the...

I've been reading recently about Fluentd and Fluent-bit as tools for log unifying and collection. The documentation says it supports a few Linux distributions but I couldn't find any reference to A...

I've been looking for a while for fluentd output plugin for tcp which is also ssl secured that doesn't force my receiver to be from a specific kind. In my case, my receiver is logstash. Here are a...

I have a container cluster in Google Container Engine with Stackdriver logging agent enabled. It is correctly pulling stdout logs from my containers. Now I would like to change the fluentd config t...

I'm a noob to both fluentd and elasticsearch, and I'm wondering if it's possible for fluentd to capture specific logs (in this case, custom audit logs generated by our apps) from stdout - use stdou...

Issue The fluentd daemonset manifest in Kubernetes Logging with Fluentd will cause an authorization error if RBAC is enabled. $ kubectl logs fluentd-4nzv7 -n kube-system 2018-01-06 11:28:10 +0000...

I am really new to kubernetes and have testing app with redis and mongodb running in GCE. I would like to grap my log files with fluentd and send them to logz: I use the following fluentd config f...

I am trying to create a centralized logging system using fluentd for a docker environment. Currently, i able to send the docker log to fluentd using fluentd docker logging driver which is a much cl...

I am trying to read from the scribe server using flunetd and output those logs to be stored in logstash for now. I know it's very stupid to log the scribe_central logs to another central logger, bu...