This repository contains the code, configurations, test data, and utilities for Manning's book Logs and Telemetry using Fluent Bit, as well as the book's examples and solutions. We have incorporated some additional examples within the chapter folders.
A quick summary of the book's chapters:
- Introduction to Fluent Bit
- From Zero to Hello World
- Capturing Inputs
- Getting inputs from Containers and Kubernetes
- Outputting events
- Parsing to extract more meaning
- Filtering and transforming events
- Stream Processors for time series calculations and filtering
- Building processors and Fluent Bit extension options
- Building Plugins
- Putting Fluent Bit into action - an enterprise use case
Additional read-me documents are incorporated into the different folders to provide domain- or chapter-specific information.
The chapters container demo configurations to illustrate different aspects of Fluent Bit. This includes for some chapters scripts which wrap utilities / tools in a container.
It is important to note that a couple of demos CAN'T run on (Mac and Windows) as not all plugins support all platforms. We have focussed on all the demos working for Linux as this is the typical platform for containerized/Kubernetes solutions. The root of the issue is that there are a few plugins that have not been built for macOS (particularly for Apple silicon), and for Windows some of the OS level services do not have a direct equivalent.
We've noted the constraints in the text of the book, and the book also has a table showing which plugins are supported by which platforms.
Fluent Bit is slowly pushing towards YAML as the configuration standard, although with a couple of exceptions, the classic and YAML configuration formats work for all plugins. As a result, we have provided YAML versions of the configuration files (although during development, we've largely worked with the classic format first).
In addition to the content in the book, we have also incorporated some extra resources, such as additional configurations and details, such as Rail Road diagrams. They are in a folder called extras or, where more appropriate, in the relevant folder.
Details of the railroad diagrams can be found here, and an example diagram:
This book covers Fluent Bit v2 onwards, focusing on its application and configuration. It's designed to cater to your specific interests in logging and monitoring, even if you haven't read Logging In Action.
It is being written as a partner title to Logging In Action (although you don't need to have read Logging in Action) to benefit from this book.
The book uses several third-party tools, all of which are referenced in Appendix A. The Log Simulator can be a very helpful tool for developing and testing Fluent Bit configurations (although it is currently limited to logs). All the resources can be found in the Log Generator GitHub repository.
The docker scripts we've provided to run the Log Simulator (are in the folder SimulatorConfig in the relevant chapter folders). These scripts exist to just simplify the Docker command - primarily mapping the volumes and environment variable configuration settings for a specific scenario.
The command does include the parameters -ti --init . This means that the Docker container will terminate with the use of ctrl-c. These parameters do the following:
-titells Docker to use a pseudo tty (terminal input) and to run interactively. As a result, you will see everything the Log Simulator sends to the console.--initcommand modifies the way the process is started within the container so that the
The use of this -ti --init is nicely explained here and you can read more about how --init works here.
For more information, you can either go to my blog