See also dockerhub tags page: https://hub.docker.com/r/fluent/fluentd-kubernetes-daemonset/tags
v1.10.4-debian-elasticsearch7-1.0,v1.10-debian-elasticsearch7-1,v1-debian-elasticsearchdocker-image/v1.10/debian-elasticsearch7/Dockerfilev1.10.4-debian-elasticsearch6-1.0,v1.10-debian-elasticsearch6-1docker-image/v1.10/debian-elasticsearch6/Dockerfilev1.10.4-debian-loggly-1.0,v1.10-debian-loggly-1docker-image/v1.10/debian-loggly/Dockerfilev1.10.4-debian-logentries-1.0,v1.10-debian-logentries-1docker-image/v1.10/debian-logentries/Dockerfilev1.10.4-debian-cloudwatch-1.0,v1.10-debian-cloudwatch-1docker-image/v1.10/debian-cloudwatch/Dockerfilev1.10.4-debian-stackdriver-1.0,v1.10-debian-stackdriver-1docker-image/v1.10/debian-stackdriver/Dockerfilev1.10.4-debian-s3-1.0,v1.10-debian-s3-1docker-image/v1.10/debian-s3/Dockerfilev1.10.4-debian-syslog-1.0,v1.10-debian-syslog-1docker-image/v1.10/debian-syslog/Dockerfilev1.10.4-debian-forward-1.0,v1.10-debian-forward-1docker-image/v1.10/debian-forward/Dockerfilev1.10.4-debian-gcs-1.0,v1.10-debian-gcs-1docker-image/v1.10/debian-gcs/Dockerfilev1.10.4-debian-graylog-1.0,v1.10-debian-graylog-1docker-image/v1.10/debian-graylog/Dockerfilev1.10.4-debian-papertrail-1.0,v1.10-debian-papertrail-1docker-image/v1.10/debian-papertrail/Dockerfilev1.10.4-debian-logzio-1.0,v1.10-debian-logzio-1docker-image/v1.10/debian-logzio/Dockerfilev1.10.4-debian-kafka-1.0,v1.10-debian-kafka-2docker-image/v1.10/debian-kafka/Dockerfilev1.10.4-debian-kinesis-1.0,v1.10-debian-kinesis-1docker-image/v1.10/debian-kinesis/Dockerfile
You can also use v1-debian-PLUGIN tag to refer latest v1 image, e.g. v1-debian-elasticsearch. On production, strict tag is better to avoid unexpected update.
See dockerhub's tags page for older tags.
v0.12 development has been ended. These images are never updated.
v0.12-debian-elasticsearchdocker-image/v0.12/debian-elasticsearch/Dockerfilev0.12-debian-logglydocker-image/v0.12/debian-loggly/Dockerfilev0.12-debian-logentriesdocker-image/v0.12/debian-logentries/Dockerfilev0.12-debian-cloudwatchdocker-image/v0.12/debian-cloudwatch/Dockerfilev0.12-debian-stackdriverdocker-image/v0.12/debian-stackdriver/Dockerfilev0.12-debian-s3docker-image/v0.12/debian-s3/Dockerfilev0.12-debian-gcsdocker-image/v0.12/debian-gcs/Dockerfilev0.12-debian-papertraildocker-image/v0.12/debian-papertrail/Dockerfilev0.12-debian-syslogdocker-image/v0.12/debian-syslog/Dockerfilev0.12-debian-graylogdocker-image/v0.12/debian-graylog/Dockerfilev0.12-debian-logziodocker-image/v0.12/debian-logzio/Dockerfilev0.12-debian-kafkadocker-image/v0.12/debian-kafka/Dockerfilev0.12-debian-splunkhecdocker-image/v0.12/debian-splunkhec/Dockerfilev0.12-debian-kinesisdocker-image/v0.12/debian-kinesis/Dockerfile
v0.12-alpine-elasticsearchdocker-image/v0.12/alpine-elasticsearch/Dockerfilev0.12-alpine-logglydocker-image/v0.12/alpine-loggly/Dockerfilev0.12-alpine-logentriesdocker-image/v0.12/alpine-logentries/Dockerfilev0.12-alpine-cloudwatchdocker-image/v0.12/alpine-cloudwatch/Dockerfilev0.12-alpine-stackdriverdocker-image/v0.12/alpine-stackdriver/Dockerfilev0.12-alpine-s3docker-image/v0.12/alpine-s3/Dockerfilev0.12-alpine-gcsdocker-image/v0.12/alpine-gcs/Dockerfilev0.12-alpine-papertraildocker-image/v0.12/alpine-papertrail/Dockerfilev0.12-alpine-syslogdocker-image/v0.12/alpine-syslog/Dockerfilev0.12-alpine-graylogdocker-image/v0.12/alpine-graylog/Dockerfilev0.12-alpine-logziodocker-image/v0.12/alpine-logzio/Dockerfilev0.12-alpine-kafkadocker-image/v0.12/alpine-kafka/Dockerfilev0.12-alpine-kinesisdocker-image/v0.12/alpine-kinesis/Dockerfilev0.12-alpine-splunkhecdocker-image/v0.12/alpine-splunkhec/Dockerfile
Fluentd is an open source data collector, which lets you unify the data collection and consumption for a better use and understanding of data.
Fluentd versioning is as follows:
| Series | Description |
|---|---|
| v1.x | current stable |
| v0.12 | Old stable, no longer updated |
Default YAML uses latest v1 images like fluent/fluentd-kubernetes-daemonset:v1-debian-kafka. If you want to avoid unexpected image update, specify exact version for image like fluent/fluentd-kubernetes-daemonset:v1.8.0-debian-kafka-1.0.
This is for v0.12 images.
In Kubernetes and default setting, fluentd needs root permission to read logs in /var/log and write pos_file to /var/log.
To avoid permission error, you need to set FLUENT_UID environment variable to 0 in your Kubernetes configuration.
These images have default configuration and support some environment variables for parameters but it sometimes doesn't fit your case. If you want to use your configuration, use ConfigMap feature.
Each image has following configurations:
- fluent.conf: Destination setting, Elaticsearch, kafka and etc.
- kubernetes.conf: k8s specific setting.
tailinput for log files andkubernetes_metadatafilter - prometheus.conf: prometheus plugin for fluentd monitoring
- systemd.conf: systemd plugin for collecting systemd-journal log. See also "Disable systemd input" section.
Overwrite conf file via ConfigMap. See also several examples:
Since v1.9.3 or later images.
You can exclude container logs from /var/log/containers/ with FLUENT_CONTAINER_TAIL_EXCLUDE_PATH.
If you have a trouble with specific log, use this envvar, e.g. ["/var/log/containers/logname-*"].
exclude_pathparameter document: https://docs.fluentd.org/input/tail#exclude_path- Fluentd log issue with backslash: fluent/fluentd#2545
If you don't setup systemd in the container, fluentd shows following messages by default configuration.
[warn]: #0 [in_systemd_bootkube] Systemd::JournalError: No such file or directory retrying in 1s
[warn]: #0 [in_systemd_kubelet] Systemd::JournalError: No such file or directory retrying in 1s
[warn]: #0 [in_systemd_docker] Systemd::JournalError: No such file or directory retrying in 1s
You can suppress these messages by setting disable to FLUENTD_SYSTEMD_CONF environment variable in your kubernetes configuration.
By default, latest images launch prometheus plugins to monitor fluentd.
You can disable prometheus input plugin by setting disable to FLUENTD_PROMETHEUS_CONF environment variable in your kubernetes configuration.
This is for older images. Latest elasticsearch images don't use sed.
By historical reason, elasaticsearch image executes sed command during startup phase when FLUENT_ELASTICSEARCH_USER or FLUENT_ELASTICSEARCH_PASSWORD is specified. This sometimes causes a problem with read only mount.
To avoid this problem, set "true" to FLUENT_ELASTICSEARCH_SED_DISABLE environment variable in your kubernetes configuration.
This daemonset setting mounts /var/log as service account fluentd so you need to run containers as privileged container.
Here is command example:
oc project kube-system
oc create -f https://raw.githubusercontent.com/fluent/fluentd-kubernetes-daemonset/master/fluentd-daemonset-elasticsearch-rbac.yaml
oc adm policy add-scc-to-user privileged -z fluentd
oc patch ds fluentd -p "spec:
template:
spec:
containers:
- name: fluentd
securityContext:
privileged: true"
oc delete pod -l k8s-app = fluentd-logging
This is from nekop's japanese article.
zookeeper gem doesn't work on Debian 10, so kafka image doesn't include zookeeper gem.
Some images are contributed by users. If you have a problem/question for following images, ask it to contributors.
- cloudwatch : @so0k
- papertrail : @alexouzounis
- kafka : @erhudy
- graylog : @rtnpro
- gcs : @andor-pierdelacabeza
- Amazon Kinesis : @shiftky
- logz.io : @SaMnCo / @jamielennox
- splunkhec: @FutureSharks
Currently, we don't accept new destination request. See fluent/fluentd-kubernetes-daemonset#293
Kubernetes Logging with Fluentd
We can't notice comments in the DockerHub so don't use them for reporting issues or asking question.
If you have any problems with or questions about this image, please contact us through a GitHub issue.
Update templates files instead of docker-image files.
docker-image files are automatically generated from templates.
Note: This file is generated from templates/README.md.erb