Berlin Buzzwords 2024

End-to-End Encryption for Streaming Data Pipelines
2024-06-10 , Palais Atelier

This talk explains what it takes to bring end-to-end encryption to streaming data pipelines built on top of Apache Kafka and Flink. A live demo illustrates how to encrypt/decrypt sensitive payload fields by means of single message transformations and user-defined functions without any custom code.


Security is a key concern for Apache Kafka: authentication, authorization, and over-the-wire encryption help to ensure the confidentiality of your data. This notwithstanding, workloads with very sensitive data might require end-to-end encryption between Kafka client applications.

This talk introduces Kryptonite for Kafka, written and open-sourced by the speaker. It's a community project that performs client-side field-level cryptography for payloads produced to and consumed from Kafka topics. A demo scenario built on top of Kafka Connect and Flink SQL illustrates how to encrypt and decrypt sensitive payload fields by means of applying turn-key ready message transformations as well as user-defined functions without the need for any custom code.

Join this session to learn how Kryptonite for Kafka makes your streaming data pipelines more secure, safeguarding your most sensitive payload fields against any form of uncontrolled or illegal access on the brokers.

Hans-Peter Grahsl is a Developer Advocate at Red Hat. He is an open-source community enthusiast and in particular passionate about event-driven architectures, distributed stream processing systems and data engineering. For his code contributions, conference talks and blog post writing at the intersection of the Apache Kafka and MongoDB communities, Hans-Peter received multiple community recognition awards and became one of the founding members of the MongoDB Champions Program in 2020. He is a regular speaker at international tech-related and developer conferences for several years.