KubeCon + CloudNativeCon North America 2024

Creating better data pipelines is constantly challenging when "better" is defined by performance, low resource usage, and total ecosystem integration. In this session, we will dive deep into Fluent Bit, a CNCF graduate project under the Fluentd umbrella that helps you build scalable data pipelines to manage all your needs for the collection and processing of telemetry data by integrating multiple data sources and formats and reliably sending it to your desired endpoints or vendors for analysis. Attendees of this session will learn about best practices for telemetry data handling and important concepts such as buffering, backpressure, monitoring, and retry logic, among many others. Fluent Bit, which was primarily known for managing logs, now also handles metrics and traces. It fully integrates with applications based on Prometheus or OpenTelemetry formats.