KubeCon + CloudNativeCon North America 2024

At Intuit, rapid development is essential for swift feature updates and fixes. Yet, 33% of last year's incidents were due to new deployments, highlighting the need for a progressive delivery system with automated rollback capabilities. However, traditional static thresholds fall short for Intuit's ~2500 services, each with unique patterns across multiple key performance metrics. To tackle this, Intuit has implemented an ML-based progressive delivery system that utilizes Prometheus to monitor multivariate metrics, offering a comprehensive view of application health and performance during deployments. The talk will present a case study application, identify its critical metrics, and showcase how Intuit leverages Numaproj and its out-of-the-box ML models to generate anomaly scores during deployments using Argo Rollouts. This strategy enables Intuit to quickly identify and address issues using AIOps techniques, ensuring a smooth and dependable customer experience.