How to troubleshoot and drill down FluentD configuration issues
- Place your configs into folder
fluentd-config
- Add prints to debug the current event at specific points with:
<filter in.**> @type stdout @id debug1 </filter>
- Add prints of customized messages (see here) at specific points by :
<filter in.**> @type stdout @id debug1-with-msg <format> @type sprintf sprintf_format "Debugging the tag:${tag}\n" </format> </filter>
- Optional: Enable running with verbose
- Start by running
sudo docker-compose up --build
Install FluentCat by installing fluentD
or
use FluentCat by execing into the running fluentd container with
sudo docker-compose exec fluentd bash
Send log:
echo '{"event":"message"}' | fluent-cat fromfluentcat
Send unparsed FluentBit log:
echo '{"log":"{\"event\":\"message\"}"}' | fluent-cat fromfluentcat
Append to file (watched by fluentbit):
echo '{"event":"message"}' >> ./fluentbit/input-fluentbit.json
echo '{"@timestamp":"2019-02-04T23:09:52.884742296+0100","event":"message"}' >> ./fluentbit/input-fluentbit.json
sudo docker-compose exec fluentd bash
echo '{"log":"{\"@timestamp\":\"2019-02-04T23:09:52.884742296+0100\",\"event\":\"message\", \"foo\":\"bar\"}"}' | fluent-cat fromfluentcat
echo '{"@timestamp":"2019-02-04T23:09:52.8847422960100","event":"message"}' >> ./fluentbit/input-fluentbit.json
sudo docker-compose exec fluentd bash
echo '{"log":"{\"@timestamp\":\"2019-02-04T23:09:52.8847422960100\",\"event\":\"message\", \"foo\":\"bar\"}"}' | fluent-cat fromfluentcat
-
FluentD Config validation without start use
fluentd --dry-run
-
fluent-plugin-debug (Unmaintained) Fluentd plugin to investigate incoming messages in a short-hand.
- Follow instructions here
- Open in VisualStudioCode
- Install ruby Extension
- Add debug config by creating file
.vscode/launch.json
{ "configurations": [ { "name": "Debug Local File", "type": "Ruby", "request": "launch", "cwd": "${workspaceRoot}", "program": "${workspaceRoot}/lib/fluent/command/fluentd.rb", "args": [ "-c", "myFluent.conf" //, "-vv" ], "env": { "RUBYLIB":"lib" } } ] }
- Know limitation: Stopping VSCode Debugging does not stop fluentd process. Run
kill $(ps aux | grep -v grep | grep fluentd.rb | awk '{print $2}')
to stop it