KubeCon + CloudNativeCon North America 2024

With the skyrocketing demand for GPUs and problems with obtaining the hardware in requested quantities in desired locations, the need for multicluster batch jobs is stronger than ever.

During this talk we will show how you can automatically find the needed capacity across multiple clusters, regions or clouds, dispatch the jobs there and monitor their status. We will discuss the setup in both fixed-size on-prem environments, fully autoscaled clusters running on clouds and mixed, hybrid environments. In the end we will present what a recent effort for a multicluster setup looks like at CERN, do a quick (but impressive) demo, and share the lessons learned during the deployment.