The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon North America 2024 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.
Please note: This schedule is automatically displayed in Mountain Standard Time (UTC -7). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date." The schedule is subject to change and session seating is available on a first-come, first-served basis.
Sign up or log in to bookmark your favorites and sync them to your phone or calendar.
Join us for an event dedicated to advancing batch and serving workloads for High Performance Computing (HPC) and seamlessly integrating AI/ML into Kubernetes. Whether you are new to the MLOps world or a seasoned practitioner, come hear from maintainers and end users about how Kubeflow and other ML platform tools are making cloud native the best option to manage your machine learning workloads. This gathering is tailored for a diverse range of technical enthusiasts, including open source contributors, practitioners, researchers, and end-users, all united by a common goal: enhancing Kubernetes as the ultimate infrastructure management tool for research, training, and production. To learn more please visit the event's website.
Running Database-as-a-Service (DBaaS) in the cloud is a common practice for organizations, and more are seeking to offer DBaaS on Kubernetes. Benefits include cost efficiencies, as well as providing a faster, more scalable development environment. While it has many benefits, managing a DBaaS on Kubernetes can be challenging. In this panel, database experts from the Data on Kubernetes Community will discuss how to get started with Kubernetes and operators to run DBaaS, storage and security requirements, common patterns for deployment and Day 2 operations, how to leverage AI for DBaaS, and pitfalls to avoid. They will also share real world experiences from users running DBaaS on Kubernetes.
Melissa Logan has worked in tech for 24 years and is currently director of the Data on Kubernetes and Data Mesh Learning communities, and founder of Constantia.io - a tech community and communications company. Constantia works with data and open source companies to provide marketing... Read More →
Gabriele, a co-founder of 2ndQuadrant and open-source advocate, has been instrumental in PostgreSQL's global growth. Focused on enhancing business continuity for large-scale databases, he has championed stateful workloads in cloud-native environments since 2019. As a co-founder and... Read More →
Deepthi is the Technical lead for Vitess, a CNCF graduated open source project. She also leads the Vitess engineering team at PlanetScale which offers a database service built on Vitess. She brings over 20 years of experience building scalable systems to this role. She enjoys speaking... Read More →
Sergey is a passionate technology “driver”. After graduation worked in various fields: internet service provider, financial sector and M&A business. Main focal points were infrastructure and products around it. At Percona as a Group Product Manager drives forward Kubernetes and... Read More →
This isn't your typical technical talk. We'll take you on a step-by-step adventure, starting from a humble single database in a container and adding components one by one, just like we did. You'll witness firsthand how we tackled real-world challenges, from storage and scaling to monitoring and UI design, to create an Open Source Cloud Native database platform. You'll walk away with a deep understanding of how Kubernetes can be used to orchestrate complex and stateful applications (like databases clusters). Join us and discover how you can break free from vendor lock-in, save costs, and build a database that's truly yours. This is your chance to learn from our triumphs and tribulations, and be inspired to create your own open source success story.
Sergey is a passionate technology “driver”. After graduation worked in various fields: internet service provider, financial sector and M&A business. Main focal points were infrastructure and products around it. At Percona as a Group Product Manager drives forward Kubernetes and... Read More →
As deep learning continues to evolve, the demand for handling petascale datasets efficiently becomes paramount. Current cloud storage solutions often struggle with the speed (throughput) and cost-effectiveness required for these massive datasets, particularly due to the random access needs of machine learning workloads. This talk introduces AIStore (AIS) as a fast-tier storage solution designed to overcome these challenges by offering a fast, scalable, cost-effective tier for deep learning data. AIS features linear scalability with each added storage node - in fact, with each added drive. In this presentation, we will explore the architecture and benefits of AIStore, focusing on its linear scalability and high performance. This session will feature detailed benchmarks and use cases comparing the performance of accessing cloud datasets with and without AIStore, highlighting AIS's ability to deliver high per-GPU throughput and stable latencies.
I'm a developer on the AIStore project, focused mainly on our Kubernetes deployments and Python SDK for training workflows. Most recently, I've been working on the AIS K8s Operator and Helm charts, improving AIStore deployment and lifecycle management. My past experience has been... Read More →
Abhishek Gaikwad is a Software Engineer at NVIDIA with a Master of Science degree in Computer Science from San Jose State University. As a key developer and maintainer of AIStore, Abhishek has played a crucial role in its design, development, and management. His contributions include... Read More →
Building data streaming platforms using open source tools like Strimzi, Kafka, Kafka Connect, Confluent Community licensed tools like Schema Registry and KSqlDB on K8s
Application of core software engineering principles in architecting open source data platforms and its benefits
Principal Software Engineer, Palo Alto Networks Inc
Priyanka J. Naik is at Palo Alto Networks Inc. where she works on secure networking software. In her career of 17 years, she has worked in Citrix Systems, Appfolio, with projects and work ranging on products like GoToMeeting, GoToTraining, GoToWebinar, and on data platforms. Interests... Read More →
As generative AI revolutionizes industries, reliance on vector databases - crucial for managing and querying high-dimensional data - has skyrocketed. These databases are often deployed on Kubernetes for its scalability and orchestration capabilities. However, ensuring robust backup and disaster recovery for these stateful applications presents unique challenges. Join Pavan and Shwetha as they discuss the critical need for an effective data protection strategy for vector databases in Kubernetes environments, emphasizing its importance in maintaining data integrity and availability. Attendees will learn about the growing significance of vector databases driven by AI applications and the specific considerations for their reliable deployment and management in cloud-native settings. Through a practical demonstration, this session will introduce Kanister, a CNCF Sandbox project, showcasing how it simplifies the complex process of backing up and recovering vector databases on Kubernetes.
Pavan joined Kasten by Veeam in March 2018, where he leads the open-source efforts and manages a team of cloud-native engineers developing innovative solutions for data protection in Kubernetes. He has previously worked in data protection and networking at NetApp and Aryaka. Pavan... Read More →
Shwetha Subramanian is a 2+ year experienced software professional, armed with a Master’s in Computer Science (Machine Learning track) from Columbia University, currently working as an SWE in the Kasten team at Veeam. An inherently curious individual, she is on a journey of learning... Read More →
At LinkedIn, we develop many stateful systems and run them on tens of thousands of machines in our datacenters. As we move LinkedIn’s infrastructure to Kubernetes, we quickly realized that StatefulSet was not going to be enough to support running critical stateful systems and satisfy the safety and durability goals of the teams developing stateful systems. We've built first-class support for running stateful workloads on bare metal where the stateful systems can coordinate with Kubernetes to stay available and ensure durability. With our design, we support planned/unplanned maintenance, swapping out hardware, and allow stateful systems to customize their rollout policies natively on Kubernetes. This talk covers: - Our LiStatefulSet API. - How we allow apps to customize safety checks and deployment policies via an ApplicationClusterManager, our pluggable policy engine. - The ApplicationClusterManager protocol that allows coordination of the lifecycle of workloads with Kubernetes.
Michael is a Staff Software Engineer at LinkedIn, currently making management and deployment of sharded systems a touch less painful on Kubernetes. In his free time he enjoys spending time with his cat, inhaling chocolate, and playing tennis.
Running stateful applications on Kubernetes can provide many of the same advantages as stateless applications. In this talk, Stephen and Chris will share some thoughts on managing stateful applications as part of a CD Pipeline so that applications - and the application's data - can be versioned and deployed safely and repeatedly. This talk will discuss managing persistent data within kubernetes, as well as managing structural changes to a database as part of a CD process. With Kubernetes and liquibase, we can provide something better than before: A more testable, repeatable, and open way to deploy stateful applications. This talk features a practical demo of how CD tooling can empower users to automate data migrations within Kubernetes.
Chris Crow works as a cloud architect at Portworx. He has worked previously as an education, systems administrator. He is a lifelong open-source enthusiast.
With over 26 years of technology experience, Stephen focuses on solving problems encountered in his previous roles. Currently he is focused on database devops at harness. He has worn hats ranging from network administrator, to database administrator, to software engineer, to product... Read More →
Today, storage technologies play a fundamental role in the realm of AI/ML. Read performance is essential for swiftly moving datasets from storage to AI accelerators. However, the rapid enhancement of AI accelerators' performance often outpaces I/O, bottlenecks the training. Due to the scheduling of pods in Kubernetes across multiple nodes, utilizing node-local storage effectively presents a challenge. To address this, we introduce a distributed cache system built atop node-local storages, designed for AI/ML workloads. This cache system has been successfully deployed on our on-premise 1024+ GPUs Kubernetes cluster within a multi-tenancy environment. Throughout our two-year experience operating this cache system, we have overcome numerous hurdles across several components, including the I/O library, load balancers, and the storage backend. We will share the challenges and the solutions we implemented, leading to a system delivering 50+ GB/s throughput and less than 2ms latency.
Toru is a machine learning platform engineer at Preferred Networks in Japan. He is the creator and lead developer of youki, an OCI Runtime in Rust, and a maintainer of the OCI Runtime Specification. Additionally, he serves as a reviewer for runwasi and is involved in developing a world that utilizes containers and Wasm. Additionally, he is a member of the Kubernetes org and is especially interested in... Read More →
He is currently a machine learning platform engineer at Preferred Networks in Japan. His research and engineering interests include a range of high-performance computing (distributed deep learning, networking/RDMA, and storage technologies), performance engineering, and Kubernete... Read More →
Autoscaling is an important part of modern cloud-native architecture. It allows applications to handle a big load at peak times while helping to optimize costs and make deployments more green and sustainable at the same time. Apache Kafka is well known for its scalability. It can grow with your project from a small cluster up to hundreds of brokers. But it was not very elastic for a long time and using dynamic autoscaling with it was very hard. This talk will guide the attendees through the main challenges of auto-scaling Apache Kafka on Kubernetes. It will show how these challenges can be solved with the help of new features added recently in Strimzi and Apache Kafka projects such as auto-rebalancing, node pools, or tiered storage. And it will help the users get started with the auto-scaling of Apache Kafka.
Jakub works at Red Hat as Senior Principal Software Engineer. He has long-term experience with messaging and currently focuses mainly on Apache Kafka and its integration with Kubernetes. He is one of the maintainers of the Strimzi project which provides tooling for running Apache... Read More →
As Kubernetes becomes the leading platform for data processing, mastering the deployment and management of Apache Spark on it is crucial. In this presentation, you'll hear from the new maintainers of the Kubeflow Spark Operator project, who will provide an overview of scaling the Spark Operator on Kubernetes, emphasizing best practices to optimize performance and efficiency. Attendees will explore the migration of the Spark Operator repository from Google to Kubeflow, gaining insights into the roadmap and key takeaways. The session will cover strategies for achieving multi-tenancy, managing multiple Spark Operator instances for large-scale deployments, ensuring robust security, and performing seamless upgrades. Participants will learn advanced techniques to maximize their Spark on Kubernetes deployments, making their data processing pipelines more efficient, reliable, and secure. This talk is for Data, ML, DevOps, and MLOps pros to enhance their Spark on Kubernetes skills.
Chaoran Yu is a software engineer at Apple. He leads a team that builds and operates a large-scale batch analytics data platform to meet the demanding requirements of data scientists and engineers. His passion lies in delivering the best value to stakeholders through best-of-breed... Read More →
Vara Bonthu is a dedicated technology professional and Worldwide Tech Leader for Data on EKS, specializing in assisting AWS customers ranging from strategic accounts to diverse organizations. He is passionate about open-source technologies, Data Analytics, AI/ML, and Kubernetes, and... Read More →
Building a new solution and contemplating whether or not the OSS path is right for you? Wondering where to get started with a large cloud initiative and where the pitfalls may lie? Curious to know all the benefits waiting if your organization embraces a rich CNCF ecosystem? In this talk we will discuss the trade-offs between building a product on a full OSS platform vs. a DIY approach. We will delve into the issues of working with internal stakeholders or partners to embrace an OSS community and will cover the benefits and scaling factors that come when embracing open standards. We will use the recent integration of NVIDIA NIM into KServe as a case study and talk through the trials and tribulations that paid off in a win-win-win situation for our solutions, the OSS projects, and our users. We will cover Kubeflow, Knative, Istio, KServe, and wg-serve as well as a network of companies building enterprise K8s platforms and enterprise AI applications on top of these foundations.
I lead AI at Canonical, the publisher of Ubuntu and a provider of open source security, support and services. With a background in data science across industries like retail and telecommunications, I help enterprises make data-driven decisions with AI. I am passionate about amplifying... Read More →
Tessa Pham is a Senior Software Engineer on Bloomberg's Cloud Native Compute Services organization. She works on building an inference platform for Bloomberg’s Data Science Platform, used by engineers and data scientists for training, deploying and serving ML models. Tessa is a... Read More →
Johnu George is a Technical Director at Nutanix with a background in distributed systems and large-scale hybrid data pipelines. He is an active in open-source and has steered several industry collaborations on projects like Kubeflow, Apache Mnemonic and Knative. His research interests... Read More →
Adam Tetelman is a principal architect at NVIDIA leading cloud native initiatives and CNCF engagements across the company; building inference platforms for NVIDIA AI Enterprise and DGX Cloud. He has degrees in computational robotics, computer & systems engineering, and cognitive science... Read More →
Taneem is an engineering leader at Red Hat where his organization is responsible for building and delivering Model Serving, Responsible AI, and Model Registry solution in OpenShift AI.
TiDB is a distributed SQL database that we built to solve the scalability problems of traditional SQL databases such as MySQL and PostgreSQL. Using TiDB, users do not need to shard their data across multiple MySQL or PostgreSQL database instances, nor do they need to sacrifice some key database features such as JOIN and transactions. Users only need to add storage nodes and computing nodes to the cluster as needed. However, we also encountered many scalability challenges when building TiKV - the stateful storage layer of TiDB. Challenges such as workload skew issues making it difficult to scale performance, management challenges of millions of dynamic data partitions, latency impact during scaling, interference between different workloads when consolidating multiple workloads into the same cluster, etc. In this talk, I will provide an in-depth look at these challenges and our solutions.
Director of Engineering at PingCAP, TiKV maintainer and committer, RocksDB contributor, the author of "MariaDB Principles and Implementation". Mainly engaged in the design and development of cloud-native large-scale distributed storage systems, data platforms, 10+ years of experience... Read More →
The day has come! Kubernetes has won the hearts and minds of your leadership and entire organizations, and everyone wants to benefit. Projects are launched to migrate legacy apps, run proprietary systems, and even use virtual machines in your Kubernetes infrastructure! But wait a minute. VMs and good' ol RDBMS are not microservices developed with 12 factors in mind where data is either hosted on an external service or replicated by the application. How are we going to warranty the availability of these applications and systems? Do I need to do a backup of these things? What if my business is fragmented across edge, on-prem, and public clouds? Members from SIG Storage will guide you through the options to compose with, including the latest CSI features, Kubernetes architecture design, and even hardware solutions. We will evaluate the benefits to consider and the pitfalls to avoid when implementing stateful workloads in Kubernetes on multiple sites.
Jan is a Senior Principal Software Engineer at Red Hat working on storage aspects of Kubernetes. He started developing Kubernetes more than 8 years ago, and is one of the founding members of SIG-Storage. He’s the author of PersistentVolume controller, dynamic provisioning and StorageClass... Read More →
Father of 2, living in France. Nerd since 1996 when Quake alpha version leaked, Linux user since 2001, Kubernetes enthusiast since 2016, member of Kubernetes SIG Storage since 2023.
When Jeff Bezos commissioned Amazon S3 he called it "malloc for the web"; since then many people have considered cloud object storage to be a weird kind of non Posix filesystem, but also a great backing store for websites or storing lots of data. Recently more and more applications are being built with object storage as the entire persistence layer. This started with analytics databases such as Snowflake and Databricks, and the open source Delta Lake and Apache Iceberg projects. More recently the use is spreading to even more applications, from observability to streaming data and more. In this talk we look at why it is becoming so popular, the benefits, downsides and performance characteristics, and how and when to use it effectively.
Justin is the CTO of Docker, recently a member of the CNCF TOC, and has been working in the container ecosystem and in supply chain security for many years.
This presentation helps simplify the container storage landscape to assist K8s users make educated cloud storage choices based on their workload requirements and data strategy. You already know K8s is a an open-source platform that orchestrates containerized applications. But what type of cloud storage should one deploy for stateless and stateful applications to ensure persistent data across various operational scenarios? Different storage types cater to specific use cases within K8s environments. Organizations often require persistent storage to run K8s for stateful use cases such as Large-Scale Application Deployment, High-Performance Computing (HPC), AI/ML, Microservices Management, CI/CD Pipelines, and Big Data Processing. Because Block, File, and Object Storage are used in varying ways for containerized workloads, this talk will explain use cases for each storage type and educate the attendees so their selection of storage supports their applications and overall data strategy.
Tom McDonald is a Senior Workload Storage Specialist at AWS. Starting with an Atari 400 and re-programming tapes, Tom began a long interest in increasing performance on any storage service. With 20 years of experience in the Upstream Energy domain, file systems and High-Performance... Read More →
In the age of big compute, the definition of privacy has transformed as re-identification from anonymized datasets has become easier. This session explores the challenges and solutions in navigating privacy concerns in high-dimensional data environments. Attendees will learn about the risks of re-identification, the importance of unicity in data sets, and how Privacy Enhancing Technologies (PETs) and Confidential Computing can mitigate these risks. Discover how these advancements can help protect sensitive data, ensure compliance, and foster a more secure data ecosystem in cloud-native environments.
Technical Community Architect, Confidential Computing Consortium, Linux Foundation
Sal is an advocate for open source, passionate about helping engineers, ethical hackers, and digital enthusiasts navigate modern software development. With over a decade of experience building cloud-native machine learning pipelines in healthcare and tech for good sectors, Sal now... Read More →
Kubernetes once favored Etcd as a database for all cluster data. Back then, relational databases lacked the availability and scalability characteristics required by Kubernetes. However, as Etcd encountered challenges with various Kubernetes workloads, relational databases continued to evolve. This session is a practical guide for deploying fault-tolerant and scalable Kubernetes clusters on distributed PostgreSQL. We’ll begin with Kine, which integrates into the Kubernetes architecture, enabling relational databases for cluster metadata management. Then, we’ll use Kine to deploy Kubernetes on a single-server PostgreSQL instance. After that, we’ll migrate to a multi-node PostgreSQL instance, allowing Kubernetes to tolerate zone and region outages and scale to thousands of nodes on demand.
Denis started his software engineering career at Sun Microsystems and Oracle, where he built JVM/JDK and led one of the Java development groups. After learning Java from the inside, he joined the world of distributed systems and databases, where he has remained ever since. His experience... Read More →
