06-11, 12:00–12:40 (Europe/Berlin), Palais Atelier
I'll talk about all the mistakes we made and all the lessons we learned developing a data platform on top of a dozen or so Kubernetes operators so you don't have to go through the same ordeal.
We have written more than 10 operators for Kubernetes.
When we started our journey in 2020, we thought lots of people surely must have done this already.
We could not have been more wrong.
As it turns out, most people write a single operator: For their product. All of them slightly different. Everyone has their own way of specifying Pod overrides, storage, TLS or LDAP support, log aggregation, monitoring, and so on.
This is before we have entered the land of horror that is Kerberos, multi-homing and various other fun topics like certifications.
We knew that we would be writing more than one we made sure to move as much functionality into a common framework as possible. That is also the reason we did not pick Go as our programming language of choice but are using Rust instead. Its support for generic (which Go lacked at that point) made it much easier to extract common functionality.
In this talk we will tell you about some of the things we learned while writing our operators, pitfalls, open questions, and a plea to collaborate on some standards in the operator world. We – as a community – can do better than today's state, I am sure of it.
But you will not only hear the developer’s side, but you will also hear the side of a user and partner of ours who appreciates the ease of use across multiple operators and utilize the Stackable platform to build on prem data platforms for their customers.
Would we do it again? Yes.
Would we do it differently? Also, yes.
Lars has been a Big Data freelancer (when it was hip!) for 12 years before founding the consulting company OpenCore, followed by the product company Stackable where he is currently the CTO. He's spoken at conferences worldwide, is an open-source enthusiast, a committer in various projects, a member of the Apache Software Foundation, and now relegated to do mostly non-technical tasks by his team. He's a father of two and lives in Germany.
Jannik has built the Business Unit Big Data & Analytics from Ground up to around 30 Employees. He has studied Computer Science and is despite his business role still an enthusiastic techie. Jannik is also lecturer at a local University since over 5 years for the bachelors degree computer science.