2025-06-16 –, Frannz Salon
K8s StatefulSets present significant hurdles for scaling and migrating large-scale cloud database workloads. We'll cover scaling strategies beyond vanilla StatefulSets and share lessons on executing zero-downtime live migrations using custom controllers, durable execution workflows, and tackling complex synchronization problems in ClickHouse Cloud.
This is a densely packed technical talk that teaches you Auto Scaling architecture, Kubernetes StatefulSets and their limitations, various scaling strategies and statefulset alternatives. We also look at building custom kubernetes controllers for the purpose of changing our orchestration code-path, and investigate leveraging durable execution workflows like Temporal for managing zero downtime migrations.
You will understand the Pros and Cons of Break-First and Make-First scaling models and which to use when. We focus on the challenges that prevent doing Make-first with traditional StatefulSets. We discuss open source projects such as Advanced StatefulSets, OpenKruise and a custom Multi-StatefulSet approach. We go into the story of moving from one mode of orchestrating StatefulSet to another via a Live Migration, without breaking the running queries. Finally we end with some ClickHouse specific problems we encountered during the migrations and how we solved them.
Store, Scale, Operations
Level:Intermediate
Manish Gill works at ClickHouse Inc, where he is managing the AutoScaling team for ClickHouse Cloud. He is based out of Berlin and is deeply interested in Databases and Cloud challenges and still considers himself new to Kubernetes.
In a past life, he worked in an ML research team doing Traffic prediction for at Global Scale and was a Data Engineer for more than half a decade before that.