KubeCon + CloudNativeCon North America 2024

Kubernetes deployments frequently utilize custom resources beyond just CPU and memory, such as GPUs, which are essential for AI/ML workloads. While the Metrics API offers insights into CPU and memory usage at both the pod and node levels, it does not provide similar information for custom resources. Although resource requests for custom resources are specified in the pod spec, there is no visibility into how efficiently these resources are utilized at the node and cluster levels. To address this gap, we developed a Prometheus Node Resource Exporter tailored to monitor custom resources. Our case study focuses on evaluating the efficiency of Kubernetes schedulers when handling a high volume of AI/ML jobs, using GPU occupancy on the nodes as the primary indicator. In this lightning talk, we will present a comparative analysis of several scheduling frameworks based on the metrics collected by our custom exporter.