KubeCon + CloudNativeCon North America 2024

Did you know that major Cloud providers cap the amount of volumes you could attach to a node? You may be tempted to use a ConfigMap or a Secret instead; however, did you know that Kubernetes caps the size of all resources to 1 MiB? What if you need arbitrarily large data? After all, reaching these limits may effectively render your application completely useless and for commonly used operators, constraints are exacerbated. In this talk, we cover all built-in storage mechanisms and their pitfalls. Not only can your data be large, but what about auto-scaled workloads which access the same data? We explore patterns that we at Google explored while working on our open source Prometheus operator, including: variable expansion, compression, sharding, projected volumes and dynamically mounting resources. We discuss how to shape the user configuration surface and how to make your data available at scale. Especially a must-see for anyone distributing configurations in their operator!