0.18 bbuzz24 2024-06-09 2024-06-11 3 00:05 https://program.berlinbuzzwords.de Europe/Berlin Palais Atelier Workshop 2024-06-09T15:00:00+02:00 15:00 03:00 Barcamps are informal sessions, a kind of "un-conference", with a schedule decided on the day. It is all driven by the interests and expertise of those who attend so each one is different, but ours are always great! bbuzz24-48574-barcamp Nick Burch en Although the barcamp doesn't have a strict schedule, it won't be completely devoid of structure! #bbuzz barcamps are dynamic events, focused on the overall Berlin Buzzwords topics, tackling the same challenges but in a different format. At the barcamp each session runs for 30 minutes giving enough time to get into the meat of a topic, but without a chance of anyone getting bored. These are participatory sessions and more inclusive than regular conference talks, with everyone taking part. You can help by leading the session, by giving some insights, by asking some great questions, or maybe just with your enthusiasm. The barcamp will be coordinated and moderated by Nick Burch. Registration starts from 2:30pm false https://program.berlinbuzzwords.de/bbuzz24/talk/A7PSPW/ Kesselhaus #BBuzz 2024-06-10T09:30:00+02:00 09:30 00:05 Join us as we kick off Berlin Buzzwords 2024! bbuzz24-48820-opening-session en false https://program.berlinbuzzwords.de/bbuzz24/talk/WVFMC7/ Kesselhaus Keynote 2024-06-10T09:35:00+02:00 09:35 00:45 At the turn of the century, the internet ushered in the era of the digital commons - a realm of collectively created and managed resources that are open to the public. Over the past decade, the digital landscape has undergone a profound transformation. bbuzz24-48222-the-paradox-of-open-can-digital-commons-offer-a-way-forward Zuzanna Warso en The early ideals of openness and decentralization have given way to a reality in which a handful of companies exert disproportionate control over the digital ecosystem and exploit its resources for their own gain. In this era, the concept of openness presents a paradox - it both challenges and enables the concentrations of power that define the current digital landscape. In this talk, I will explore the paradox of open and consider whether and under what conditions digital commons can provide a solution. I will discuss some strategies for reimagining openness to ensure its resilience and effectiveness in the face of evolving digital challenges. false https://program.berlinbuzzwords.de/bbuzz24/talk/VE3XYF/ Kesselhaus Short Talk 2024-06-10T10:40:00+02:00 10:40 00:20 Discover how melding user-traffic signals with LLM-derived relevance labels can significantly improve learning to rank models. This talk unveils a novel approach to enhancing search relevance for query-product pairs, offering a glimpse into the future of e-commerce search technology. bbuzz24-45770-synergy-of-signals-traffic-logs-meet-llm-labels Stefana Serban en In the domain of search technology, the precision of matching user queries with the most relevant products is paramount. This session ventures into the forefront of relevance engineering, spotlighting a technique that marries unclear signals from user-traffic logs with programmatically obtained relevance labels from large language models (LLMs). We focus on improving the labels used for learning to rank models, central for mature search systems. We'll dissect the process of integrating user-generated data with synthetic relevance labels, underscoring the synergy between real-world behaviors and artificial intelligence insights. This blend not only addresses the inherent ambiguity in user queries but also significantly amplifies the performance and accuracy of search results. The presentation will guide you through the intricacies of collecting, processing, and integrating these diverse data sources. We'll navigate through the challenges this novel approach presents, proposing robust solutions and best practices for implementing this strategy effectively. false https://program.berlinbuzzwords.de/bbuzz24/talk/SYFPSS/ Kesselhaus Talk 2024-06-10T11:10:00+02:00 11:10 00:40 The successes (and failures) of applying Machine Learning to improve Reddit's search ranking bbuzz24-45751-learning-to-rank-for-reddit-search-a-project-retro Doug TurnbullCharles Njoroge en In today’s AI based world, Reddit stands out as a deep catalog of human, subjective information. Whether product reviews or the deeply personal - Reddit searchers want to connect with other humans, not generic AI based answers. We at Reddit would like the site-search experience to be better, so you don’t need to add “Reddit” to your Google search. That’s what we’re trying to do with Learning to Rank: turning relevance into a repeatable, data-driven solution. The journey hasn’t been an easy one. We want to share our painful lessons learned working with training data, developing features, the Solr Learning to Rank plugin, scaling Learning to Rank to 1000s of QPS, and more. Hopefully, you can learn from the egg we constantly found on our faces! See how our scrappy, understaffed team has been slowly turning LTR from a science project into a repeatable process of constant, data-informed improvement. From a lab to an assembly line, come and learn from our painful lessons big and small. false https://program.berlinbuzzwords.de/bbuzz24/talk/WKMGWH/ Kesselhaus Talk 2024-06-10T12:00:00+02:00 12:00 00:40 This talk focuses on how we leveraged Nvidia’s open source cuVS library to accelerate Apache Lucene’s vector search capabilities on the GPU. cuVS contains several algorithms for approximate nearest neighbors and clustering on the GPU and we show how it pairs well with Lucene, which has long been the bbuzz24-45736-cuvs-and-lucene-gpu-based-vector-search Corey J. NoletVivek Narang en By 2025, it is believed that 80% of all data will be unstructured. Meanwhile, modern advances in large language models and generative AI have acted as a catalyst for the use of vector embeddings, which are impacting nearly every industry. Vector search is growing in popularity and, thanks to these advances, the need for efficient and scalable semantic search is becoming more evident. GPUs have become synonymous with AI over the past decade, but even more intriguing is the advancement of software capabilities that can leverage GPUs to accelerate more general-purpose data processing workloads like vector search. cuVS, from Nvidia [1], is a CUDA-based library containing state of the art implementations of several algorithms for approximate nearest neighbors and clustering on the GPU. Apache Lucene [2] is an open-source search library that is at the core of popular search engines like MongoDB, Elasticsearch, OpenSearch and Apache Solr. This talk will have two main parts, 1) Introduction to Nvidia’s cuVS library, its history, approximate nearest neighbor search algorithm types, their implementations and comparison, the novel graph-based CAGRA algorithm, and the cuVS future roadmap. 2) Integration of cuVS into Apache Lucene to power GPU accelerated vector search, the motivations, challenges, the roadmap associated with this integration, and potential future directions for turbo-charging Lucene on the GPU. We will provide benchmarks and lessons we've learned along the way, and hope that this will spark a new trend where GPUs can be used to accelerate other compute-heavy search, analytics and database tasks in the future. [1] - cuVS, https://github.com/rapidsai/cuvs, was formerly part of RAFT, https://github.com/rapidsai/raft [2] - https://github.com/apache/lucene false https://program.berlinbuzzwords.de/bbuzz24/talk/PTFLE7/ Kesselhaus Talk 2024-06-10T14:00:00+02:00 14:00 00:40 Discover Restate: A new tool for event-driven apps, blending Kafka's power with simplicity for handling complex transactions, async tasks, and microservices. Dive into its unique approach to tackle scalability, resilience, debuggability, and state management challenges in our session. bbuzz24-45950-fixin-the-hard-bits-of-event-processing-with-restate-kafka Stephan Ewen en Event-driven applications are still hard in 2024, despite all their strong and desirable properties (scalability, resilience, ability to replay, ...). While streaming analytics are supported by a wave of powerful tools (like Flink, KStreams, Materialize, RisingWave, ...), the transactional side of event-driven applications is not quite as far. Complex per-event control flow, complicated state semantics, and interactions with many external dependencies (APIs/services/...) make those applications challenging. Observability and debuggability are still ways behind what RPC-based architectures offer with today's tools. In this talk we present 'Restate' (https://restate.dev/), a new open source tool we have been working on. Restate complements systems like Kafka when building event-driven workflows, async tasks, distributed signaling, or even full microservices architectures. While Restate internally is an event-driven system (sharing some similarities with a Kafka/Flink/KStreams crossover), it has a very simple programming model: Durable Execution (for event-handlers and RPC), Durable Promises (for signaling), and Virtual Objects. Those three constructs are the resilient encapsulations of the core ingredients of every distributed application: compute, communication, and state. In the talk, we will first we'll look practically at common challenging patterns: Where do those tend to break current architectures? How can we address those issues with Kafka/Restate? - burst-y loads / rapid scaling - head-of-the-line waiting on long event processing times - combining Kafka events with webhooks and other signals - Transactionally updating databases - high-fan-out routing to consumer APIs - state machines with long-running transitions - delaying/postponing certain events Second, we take a look at similarities, synergies, and differences between Restate, Kafka, and Stream-Processing. We look at the way applications are expressed, how the systems organize event logs and event processing, and how this mirrors the differences between analytical- and transactional applications respectively. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/MEDCT3/ Kesselhaus Short Talk 2024-06-10T14:50:00+02:00 14:50 00:20 The demand for advanced search and data retrieval capabilities is growing exponentially: The rise of AI applications, along with the unprecedented scaling of data is leading to workloads that are pushing traditional software based search to its limits. As cloud computing costs are skyrocketing, and queries are becoming more complex, organizations are often forced to compromise results relevancy given the strict real-time latency requirements of 100ms. To address this issue, a new dimension of search must be introduced: domain-specific computing; it focuses on designing a dedicated chip for search, to consistently achieve ten times faster search at a billion-scale, all at a fraction of the infrastructure cost. bbuzz24-50148-shattering-the-limits-of-search-with-domain-specific-computing Ohad Levi en Maintaining latency below 100ms at scale depends on several factors, such as the level of throughput (QPS), complexity of the queries, the hardware and network infrastructure, and the indexing strategy being used. With all of the above, maintaining real-time latency is extremely challenging at a large scale for a number of reasons: • Millions to millions of documents are becoming standard • Both BM25 and k-NN algorithms are compute intensive and requires extensive compute power at scale • High query throughput (QPS) during surges in demand • Real-time database Updates conducted on the DB in parallel to search queries A major contributor to the underperformance of existing software based solutions is the fact that these solutions are dependent on general purpose CPUs, those are designed to handle a wide range of tasks, but they are not optimized for any specific task. The outcome is suboptimal search processing power. This is where Domain-specific computing comes into play. Domain-specific architectures are tailored for specific workloads, significantly boosting processing speeds by tens to hundreds of times. These purpose-built chips excel in performing thousands of computations simultaneously, making them ideal for high-throughput tasks like data processing and search indexing. Additionally, they can offload processing from the CPU, handling specific operations such as term matching, sorting, and filtering, which reduces CPU workload and enhances query performance. Purpose built chips like FPGAs have become available in the cloud (e.g. AWS) and are compatible with Elasticsearch. It supports dynamic scaling to efficiently manage fluctuating traffic volumes and delivers rapid search results, enhancing user experiences and boosting customer satisfaction. Overall, domain-specific computing pushes search and data retrieval capabilities to new heights, transcending traditional data distribution methods for efficiency. It minimizes software stack overhead and enables organizations to achieve the performance needed at scale and with consistency. <br> --- <br> This session is sponsored by <a href="https://hyper-space.io/">Hyperspace</a> false https://program.berlinbuzzwords.de/bbuzz24/talk/8WLZYZ/ Kesselhaus Talk 2024-06-10T15:20:00+02:00 15:20 00:40 Flink's SQL engine is the workhorse behind many event processing platforms. We take a deep look into the internals and take the stack apart! Optimizer phases, streaming primitives, watermarks, CDC, and upsert keys. Let me give you a feeling for the power of a simple streaming SQL query. bbuzz24-45650-flink-s-sql-engine-let-s-open-the-engine-room Timo Walther en Apache Flink aims to make stream processing easy and accessible for everyone. It should come as no surprise that a high level of abstraction puts more load on the core. Flink's SQL engine is the workhorse behind many on-prem and managed SQL platforms. Yet very few users know what is really going on under the hood when submitting a SQL query. In this talk, we take a deep look into the internals of Flink SQL. Let's take the stack apart! We start with some SQL text and go all the way down to Flink's streaming primitives. I will go through the individual optimizer phases. You will learn how event-time operations are tracked when declaring a watermark, how state is managed when using different kinds of joins, and how changelog modes and upsert keys travel through topology when reading from a Change Data Capture connector. After this talk, you may not be able to write an optimizer rule, but you should, at least, get a feeling for the power of a simple streaming SQL query. false https://program.berlinbuzzwords.de/bbuzz24/talk/ZHGJLB/ Kesselhaus Talk 2024-06-10T16:30:00+02:00 16:30 00:40 Gear up for a speed competition, it's ChatGPT versus yours truly, racing to build a data visualisation platform from scratch at top speed. Who will win? We’ll be taking on the challenge of reconstructing an energy generation visualisation platform, drawing inspiration from the likes of GridWatch. bbuzz24-42299-can-chatgpt-build-a-data-platform-faster-than-a-developer Chloé Caron en Gear up for a speed competition, it's ChatGPT versus yours truly, racing to build a data visualisation platform from scratch at top speed. Who will win? We’ll be taking on the challenge of reconstructing an energy generation visualisation platform, drawing inspiration from the likes of GridWatch and the Energy Dashboard. We’ll dig into some key steps: 1. **Finding our data source**: Unearth the data gold 2. **Extracting our data**: Release the data from its digital prison 3. **Transforming it**: Turn it into the most useful version of itself 4. (Optional) **Storing the data**: Keep it nice and safe 5. **Creating the visualisation**: Craft the masterpiece that will tell the story But it’s not just a race - once we’ve determined the winner, we will explore some vital insights, delving into cost and trade-offs among the available tools. At the end of the talk, you’ll have a better understanding on how to build your own platform. false https://program.berlinbuzzwords.de/bbuzz24/talk/YFSZUF/ Kesselhaus Talk 2024-06-10T17:20:00+02:00 17:20 00:40 Yes it's possible to create an 8-bit computer from scratch on breadboards at home with basic components and that's what I will demonstrate live with lots of blinking LEDs ;) bbuzz24-43583-build-your-8-bit-computer-from-scratch Olivier HUBER en Do we really understand how a CPU works? I will display and live demo a 4 lines computer program in binary (not even assembly) on a machine created on breadboards. There will be blinking LEDs in a fun and passionate presentation of this homemade 8-bit computer created on breadboards. Please try this at home ;) That is my challenge with this project of bringing the joy of understanding computer science to the masses. As usual, it will be be fun and exciting and be ready for some surprises! false https://program.berlinbuzzwords.de/bbuzz24/talk/BVGPGZ/ Kesselhaus #BBuzz 2024-06-10T18:00:00+02:00 18:00 03:00 Join us for snacks & drinks in an relaxed atmosphere. Meet old and new friends, business contacts or get to know other participants of Berlin Buzzwords! The Get-Together takes place directly in front of Palais Kulturbrauerei. bbuzz24-48821-get-together en false https://program.berlinbuzzwords.de/bbuzz24/talk/PFADZG/ Maschinenhaus Short Talk 2024-06-10T10:40:00+02:00 10:40 00:20 We’re taking a stab at a definitive - yet nuanced! - answer to “how much heap do I need?” and “is GC a problem and how can I fix it?” for Solr/Elasticsearch/OpenSearch deployments. bbuzz24-43209-heap-sizing-and-gc-tuning-for-solr-and-friends Radu GheorgheRafał Kuć en If the Java heap is too low, you may hit an OOM or circuit breakers. Or maybe the GC latency spikes, causing cluster instability. GC latency can also be high when cleaning a heap that’s too large. But even if it doesn’t, you might be wasting memory that can be used by the OS to cache index files. Choosing good heap size and GC options (i.e. for your use-case) is important for both stability and performance. In this session, we’ll start by looking at what uses heap (we’ve got some cool production stories in this area) and how garbage is being generated and cleaned up. Then, we’ll zoom into what happens when you run out of heap: you either simply need more, or your GC fell behind. We’ll look at how you can know whether you need more and which GC options allow you to improve throughput. Sometimes at the expense of median latency, but usually p99 or maximum latency is what we really care about. The aim is for participants to understand how to use metrics and knowledge about the use-case to choose the heap size and GC options. Instead of using rules of thumb like “use half of your RAM but no more than 30GB heap” or `ParallelGCThreads=4`. There’s a grain of truth in those, but they can be very limiting on large-scale clusters (and in some small clusters, too). We’ll cover both the origins and the limitations of these rules of thumb as well. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/UFHCEJ/ Maschinenhaus Talk 2024-06-10T11:10:00+02:00 11:10 00:40 We explore the usage of AI, especially NLP techniques and LLM, to enhance Apache Solr data accessibility. We propose translating natural language queries into structured Solr queries using LLM and metadata to improve search and user experience. We’ll discuss the results and future directions. bbuzz24-44754-from-natural-language-to-structured-solr-queries-using-llms Ilaria PetretiAnna Ruggero en This talk draws on experimentation to enable AI applications with Solr. One important use case is to use AI for better accessibility and discoverability of the data: while User eXperience techniques, lexical search improvements, and data harmonization can take organizations to a good level of accessibility, a structural (or “cognitive” gap) remains between the data user needs and the data producer constraints. That is where AI – and most importantly, Natural Language Processing and Large Language Model techniques – could make a difference. This natural language, conversational engine could facilitate access and usage of the data leveraging the semantics of any data source. The objective of the presentation is to propose a technical approach and a way forward to achieve this goal. The key concept is to enable users to express their search queries in natural language, which the LLM then enriches, interprets, and translates into structured queries based on the Solr index's metadata. This approach leverages the LLM's ability to understand the nuances of natural language and the structure of documents within Apache Solr. The LLM acts as an intermediary agent, offering a transparent experience to users automatically and potentially uncovering relevant documents that conventional search methods might overlook. The presentation will include the results of this experimental work, lessons learned, best practices, and the scope of future work that should improve the approach and make it production-ready. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/7VSFFK/ Maschinenhaus Talk 2024-06-10T12:00:00+02:00 12:00 00:40 S3 is cheap; but S3 is slow. Stream procesisng systems want to maintain internal states at low cost, but cannot tolerate high latency. How could we build a stream processing system on top of S3? I'll tell you what we learned over the last three years. bbuzz24-45565-s3-as-the-state-store-for-stream-processing-systems Yingjun Wu en S3 is cheap; but S3 is slow. Stream procesisng systems want to maintain internal states at low cost, but cannot tolerate high latency. Could we build a stream processing system on top of S3? We started building RisingWave, a stream processing system, from scratch three years ago. It uses S3 as the state store and implements a storage and compute separation architecture to reduce performance costs. While the idea looks appealing, implementing it and supporting production workloads is full of challenges. In this talk, I will share with you lessons we've learned from building a production-ready system that can achieve: 1) Efficient execution; 2) Fast failure recovery; 3) Dynamic scaling, at a reasonably low cost. false https://program.berlinbuzzwords.de/bbuzz24/talk/LLXN8V/ Maschinenhaus Talk 2024-06-10T14:00:00+02:00 14:00 00:40 Many of the putatively novel challenges of building systems around LLMs are analogous to problems we've solved for conventional ML systems. This talk will show you why the things you already know about building ML systems are still relevant for LLM systems — and where the true novelty of LLMs lies. bbuzz24-45722-large-language-models-are-not-a-paradigm-shift William Benton en Hundreds of millions of people have used generative large language models (LLMs) to entertain themselves and improve their productivity. Sophisticated enterprises have scrambled to quickly put LLM initiatives and applications in place. It's hard not to wonder if LLMs are finally the technology that will make AI "real" to a broad audience after years of steady progress in machine learning (ML) applications. This talk will argue that the emergence of LLMs does not represent the paradigm shift that it might appear to. As we'll see, many of the putatively novel challenges that LLM applications have strong analogies to challenges faced by conventional ML systems. We'll discuss the differences between LLM applications and ML applications, and see how many of these are important but not fundamental. We'll discuss the challenges inherent to getting the best out of LLMs and learn how AI systems can go wrong, how to build them responsibly, and why the aspects of LLMs that are truly new give us a reason to ultimately believe at least some of the hype after all. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/EDT99N/ Maschinenhaus Short Talk 2024-06-10T14:50:00+02:00 14:50 00:20 Looking to reduce your home energy consumption, to move to more ecological heating, or just want to know what's going on in your house? You need data, and graphs! Learn how cheap hardware plus devops observability tools let you do that! bbuzz24-45741-monitoring-your-home-with-devops-observability-tools Nick Burch en If you want to reduce your home energy consumption, are looking to move to more ecological heating, or just want to know what's going on in your house, the first step is getting some data. Smart thermometers and smart power sockets are pretty cheap these days, but then what? There are several commercial cloud solutions out there, but we saw from last year's talk by Steve Loughran how bad they can be. We have several good Open Source tools for Home Automation, but they can be a bit much if you just want to capture some simple data. Luckily, there's not much difference between monitoring the power consumption of a rack and of a house. There's little change between graphing the temperature of a CPU and a room. The observability tools you're (hopefully!) already using in the DevOps world to monitor your systems can also help your home. Join us to learn about the cheap hardware you need to capture the data, and how to feed it into standard observability tools. There will be code snippets. And of course, there will be graphs! false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/BQTZHK/ Maschinenhaus Talk 2024-06-10T15:20:00+02:00 15:20 00:40 This session explores different fallback strategies for RAG applications, enabling you to build robust LLM systems for various scenarios, using open source LLM framework Haystack. bbuzz24-45801-improve-llm-based-applications-with-fallback-mechanisms Bilge Yücel en Large Language Model (LLM)-based systems have demonstrated remarkable advancements in various natural language processing (NLP) tasks, particularly through the Retrieval Augmented Generation (RAG) approach. This approach addresses some of the pitfalls associated with LLMs, such as hallucination or issues related to the recentness of its training data. However, RAG systems may encounter other challenges in real-world scenarios, including handling out-of-domain queries (e.g., requesting medical advice from a finance app), struggling to generate meaningful answers from retrieved data, or failing to provide any answer at all. To address these situations effectively, it is necessary to implement a fallback mechanism capable of gracefully handling such scenarios. This fallback mechanism can incorporate alternative strategies, such as conducting a web search with the same query to retrieve more up-to-date information or utilizing alternative information sources (such as Slack, Notion, Google Drive, etc.) to gather more relevant data and generate a satisfactory or comprehensive response. However, the question arises: how can we determine if the response is inadequate? During this session, we will explore various fallback mechanism techniques and ensure that our system can assess the adequacy of a response and improve it if necessary without human intervention. On the practical side, we will use the open source LLM framework Haystack to implement end-to-end RAG systems. By the end of this talk, you will have learned to select the appropriate fallback method for your use case, enabling you to develop more dependable and versatile LLM-based systems and implement them effectively using [Haystack](https://github.com/deepset-ai/haystack). false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/JMLFV8/ Maschinenhaus Talk 2024-06-10T16:30:00+02:00 16:30 00:40 How do you track data lineage in an open-source, realtime analytics pipeline? I'll show in a demo! bbuzz24-45653-let-s-do-data-lineage-in-kafka-flink-and-druid Hellmar Becker en Data lineage means you can track the data bits in your system and know at any time where they come from and how exactly they have been processed. Enterprise systems need to be able to prove lineage for compliance reasons, but in general, lineage is also a big part of data discoverability and governance. In this talk, I am going to connect a few Raspberry Pi's that collect ADS-B (aircraft radar) data to a KFD (Kafka-Flink-Druid) stack for analytical processing. I will deliver the data through Kafka, cleanse and enrich them with Flink, and run analytical queries on the result with Druid. I am going to track data lineage through Kafka metadata, and I am going to show how that information can be maintained throughout the processing pipeline. This relies on using Kafka headers, an underused feature of Kafka that also integrates readily and easily with Druid! You will learn how data lineage can be implemented using the open source KFD stack and readily available data sources, so you, too, can try out enterprise style data lineage processing, and prepare yourself at home for a question that will arise in any enterprise data engineering project! false https://program.berlinbuzzwords.de/bbuzz24/talk/DPY8W3/ Maschinenhaus Talk 2024-06-10T17:20:00+02:00 17:20 00:40 Learn a new cloud architecture that separates streaming ingest compute from query compute for real-time applications and how it addresses the age-old issue of compute contention. bbuzz24-44910-how-we-isolate-streaming-ingest-from-search-using-rocksdb Igor Canadi en In this presentation, we delve into how we disaggregated streaming ingest compute from query compute for real-time applications. Up until this point, real-time architectures were designed for data streaming ingestion and query processing to be executed within the same cluster. This co-location, while beneficial for fresh data access, often leads to competition for compute resources, especially during spikes in either data ingestion or query activities. There are strategies for mitigating compute contention, including replication and overprovisioning clusters, but they do not fully address the issue. We describe how we built a new cloud architecture using RocksDB, a key-value store with a log-structured merge-tree architecture. The design disaggregates compute from storage, enabling simultaneous querying of shared real-time data across multiple clusters. The architecture also separates streaming ingest compute and query compute by replicating the in-memory state of the RocksDB memtable across different compute clusters, ensuring the accessibility of the latest data in single-digit milliseconds. Learn how this architecture is being used for search, real-time analytics and AI-powered search applications. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/ULUBJE/ Palais Atelier Short Talk 2024-06-10T10:40:00+02:00 10:40 00:20 At mobile.de we are striving to provide a better and faster search. We use a backend ML system to learn changing user interests and optimise search experience. Based on learning to rank using XGBoost, we discuss current search relevance ranking framework and how it ranks millions of searches daily. bbuzz24-50638-better-search-relevance-using-learning-to-rank-at-mobile-de Manish Saraswat en At mobile.de, we continuously strive to provide our users with a better, faster and a unique search experience. Machine learning and Python play a key role in providing this experience. Every day, millions of people visit mobile.de to find their dream car. The user journey typically starts by entering a search query and later refining it based on their requirements. If the user finds a relevant listing, they contact the seller to purchase the vehicle. Our search engine is responsible for matching users with the right sellers. In this talk, I will talk about: - Introduction - Why search is important - How learning to rank helps ? - Current challenges with our ranking models - Proposed solution - How do we deploy our ranking models ? (Under strict latency SLA <30ms) - AB Test results - Key Learnings - How can we improve further <br> --- <br> This session is sponsored by <a href="https://www.mobile.de/careers/">mobile.de</a> false https://program.berlinbuzzwords.de/bbuzz24/talk/MARKUE/ Palais Atelier Talk 2024-06-10T11:10:00+02:00 11:10 00:40 Let’s rebuild the world of Apache Kafka® brick-by-brick through the lens of LEGO®. You’ll explore Kafka’s inventory of pieces, learn the ins and outs of core elements, and see just how they ‘click’ together with other APIs and tools. The only question left is: what will YOU make with Kafka? bbuzz24-42322-brick-by-brick-exploring-the-elements-of-apache-kafka Danica Fine en Have you hit a brick wall when it comes to learning Apache Kafka? Do you wish that grasping Kafka were as easy, intuitive, and fun as building your favorite LEGO® set? Why shouldn’t it be!? Let’s rebuild the world of Kafka brick-by-brick starting from the basic building blocks of the technology. We’ll leave no plate unturned as we introduce events, brokers, topics, and partitions––fundamental elements that affect how data is stored inside of this powerful distributed event streaming platform. From there, explore the wider inventory of pieces in the ecosystem––APIs and tools like Kafka Streams and Kafka Connect––that you can use to migrate, stream, and transform your data. By the end of the session, you’ll know the ins and outs of the components that form the basis of Kafka, how they ‘click’ together, and what you can build with them. At that point, only one question should remain––what will YOU make with Kafka? false https://program.berlinbuzzwords.de/bbuzz24/talk/KEKJYS/ Palais Atelier Talk 2024-06-10T12:00:00+02:00 12:00 00:40 Hybrid search combines traditional keyword-based search with vector-based search. The result sets are merged and a single ranked list of items is returned. Reciprocal Rank Fusion is one of the most popular algorithms for such a task. This talk presents the work done to bring it to Apache Solr. bbuzz24-44853-hybrid-search-with-apache-solr-reciprocal-rank-fusion Alessandro Benedetti en Vector-based search gained incredible popularity in the last few years: Large Language Models fine-tuned for sentence similarity proved to be quite effective in encoding text to vectors and representing some of the semantics of sentences in a numerical form. These vectors can be used to run a K-nearest neighbour search and look for documents/paragraphs close to the query in a n-dimensional vector space, effectively mimicking a similarity search in the semantic space (Apache Solr KNN Query Parser). Although exciting, vector-based search nowadays still presents some limitations: - it’s very difficult to explain (e.g. why is document A returned and why at position K?) - it doesn’t care about exact keyword matching (and users still rely on keyword searches a lot) Hybrid search comes to the rescue, combining lexical (traditional keyword-based) search with neural (vector-based) search. So, what does it mean to combine these two worlds? It starts with the retrieval of two sets of candidates: - one set of results coming from lexical matches with the query keywords - a set of results coming from the K-Nearest Neighbours search with the query vector The result sets are merged and a single ranked list of documents is returned to the user. Reciprocal Rank Fusion (RRF) is one of the most popular algorithms for such a task. This talk introduces the foundation algorithms involved with RRF and walks you through the work done to implement them in Apache Solr, with a focus on the difficulties of the process, the distributed support(SolrCloud), the main components affected and the limitations faced. The audience is expected to learn more about this interesting approach, the challenges in it and how the contribution process works for an Open Source search project as complex as Apache Solr. false https://program.berlinbuzzwords.de/bbuzz24/talk/NCPYUH/ Palais Atelier Talk 2024-06-10T14:00:00+02:00 14:00 00:40 We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines. bbuzz24-45884-end-to-end-pipeline-agility Lars Albertsson en A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more. A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream. Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/UVWWJF/ Palais Atelier Short Talk 2024-06-10T14:50:00+02:00 14:50 00:20 This talk explains what it takes to bring end-to-end encryption to streaming data pipelines built on top of Apache Kafka and Flink. A live demo illustrates how to encrypt/decrypt sensitive payload fields by means of single message transformations and user-defined functions without any custom code. bbuzz24-45545-end-to-end-encryption-for-streaming-data-pipelines Hans-Peter Grahsl en Security is a key concern for Apache Kafka: authentication, authorization, and over-the-wire encryption help to ensure the confidentiality of your data. This notwithstanding, workloads with very sensitive data might require end-to-end encryption between Kafka client applications. This talk introduces Kryptonite for Kafka, written and open-sourced by the speaker. It's a community project that performs client-side field-level cryptography for payloads produced to and consumed from Kafka topics. A demo scenario built on top of Kafka Connect and Flink SQL illustrates how to encrypt and decrypt sensitive payload fields by means of applying turn-key ready message transformations as well as user-defined functions without the need for any custom code. Join this session to learn how Kryptonite for Kafka makes your streaming data pipelines more secure, safeguarding your most sensitive payload fields against any form of uncontrolled or illegal access on the brokers. false https://program.berlinbuzzwords.de/bbuzz24/talk/YBGVAY/ Palais Atelier Talk 2024-06-10T15:20:00+02:00 15:20 00:40 In this talk we detail how Yelp evolved its Streaming Data Platform at scale while supporting its growing business needs coupled with the fast changing streaming landscape. bbuzz24-45636-evolving-yelp-s-streaming-at-scale Ashish Khatkar en Yelp relies on near real-time data to enable critical functionality ranging from in-flight business data moving across different data stores, to operational and user data used to train machine learning models. In this talk, we will detail the evolution of Yelp's streaming data platform. - We will delve into the past to review the data platform we built abstracting storage and compute, and the middleware we implemented to tie everything together. We will also visit observability and maintenance challenges derived from hyper-adoption. - In the present, we've standardized on SQL for data pipelines, except for advanced use cases. - Looking towards the future, we're transitioning from custom to open-source data formats and registry, leveraging Flink. We're also exploring Streamhouse to address current challenges. This talk will provide insights into the challenges faced in building and scaling a streaming data platform, as well as the strategies employed to overcome them, making it a valuable session for anyone interested in data engineering and streaming technologies. false https://program.berlinbuzzwords.de/bbuzz24/talk/H8VLKT/ Palais Atelier Talk 2024-06-10T16:30:00+02:00 16:30 00:40 Dense Vector Search is not the only route to improve your search relevance. Empower your existing sparse keyword search with semantic search capabilities by leveraging text expansion, metadata enrichment, and query re-writing techniques. bbuzz24-45772-rediscover-your-keyword-search-expand-enrich-and-rewrite Praveen Mohan PrasadHajer Bouafif en This presentation discusses about advanced methods to enhance keyword search by leveraging machine learning and large language models. The techniques involve utilising sparse models for document and query expansion, implementing metadata enrichment for text and images, and incorporating query rewriting using Large Language models. Through a live demonstration on a retail search scenario, we will illustrate how these approaches provide more relevant results by addressing the challenges of keyword search, such as vocabulary mismatches, unmatched queries against document attributes, and non-contextual search. The goal is to empower your existing keyword search system with semantic search capabilities without totally transforming to a costly and un-explainable, dense vector search system. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/WLKNUS/ Palais Atelier Talk 2024-06-10T17:20:00+02:00 17:20 00:40 Apache Pinot is a really fast database that's powering real-time applications across various industries, much akin to how people use OLTP databases. So what's the difference ? Are there scenarios where Pinot can replace an existing OLTP database ? In this talk we will explore this idea. bbuzz24-44812-can-apache-pinot-replace-your-oltp-database Chinmay Soman en Apache Pinot is a blazingly fast analytics database that’s powering mission critical applications in Stripe, Doordash, Uber, Cisco, 7-11 and more. It was designed to support very high query throughput (think of 1000s or 100s of 1000s of queries per second) with sub-second query latency. It also supports real time upserts and deletes in real time which makes it an ideal fit for generating accurate insights. This brings us to the question - can you use Apache Pinot in lieu of an OLTP database such as MySQL or Postgres. The answer as you would imagine is - it depends. In this talk we will do a deep dive into what makes Pinot so fast. We’ll take a closer look at what use cases are a good fit for Pinot vs OLTP and also where it complements existing solutions. Finally, we’ll see this all in action with a small demo. false https://program.berlinbuzzwords.de/bbuzz24/talk/P7ULUB/ Frannz Salon Short Talk 2024-06-10T10:40:00+02:00 10:40 00:20 TopK Queries are SQL queries that contain both an order by and a limit clause. The optimization presented in this talk uses runtime information to skip scanning data that could not be a part of the final result, thereby significantly accelerating these types of queries. bbuzz24-45558-accelerating-topk-queries Juliane Waack en This talk is about a recently added optimization to the Snowflake Cloud Database for speeding up TopK queries. In it the audience will learn about how we use runtime information to skip scanning data partitions that will not contribute to the result of the query. TopK Queries are SQL queries that contain both an order by and a limit clause. This is a very common query pattern. Examples are queries looking for the last k events (order by time descending), the k cheapest products (order by price ascending) or simply any limit query that is expected to deterministically return the same k rows (order by any unique identifier). In contrast to limit queries without an order by, the query engine cannot simply return the first k rows it has scanned. Instead it has to take all data into consideration to find the k smallest or largest rows. We can, however, come up with an upper / lower bound for values that can make it into the result after having seen at least k rows. This boundary can then be used to filter out rows that will not be a part of the result. Our optimization takes this a step further and additionally uses the boundary for pruning entire data partitions. This means that we compare it with metadata for these partitions and based on this comparison may decide to skip downloading and scanning a partition entirely if none of its contained rows can become a part of the result. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/3DTQJB/ Frannz Salon Talk 2024-06-10T11:10:00+02:00 11:10 00:40 What's the role of Orchestrators in the Modern Data Stack?. Trying new orchestrators? Do we need Orchestrators at all? Or what "Orchestrator" can do for you had been redefined ? Meet "new" orchestrator in town ..... "Apache Airlfow 2.x" with ways of orchestration you had not realized you can do. bbuzz24-45816-new-workflow-orchestrator-in-town-apache-airflow-2-x Jarek Potiuk en Efficient and well managed Orchestrating your Data processing Pipelines is crucial to provide the back-bone of the modern Data processing needs. With all the new rage of LLMS and AI, it's more important than ever to make sure that you can wrap your head around all the old, new and upcoming data processing tools and services you use, but also track the provenance, lineage of the data, and make it easy to author and mange those for distributed teams. This talk will bring to the light how modern Airflow 2.8+ provides proven and also modernised, easy and nice to use ways on how to do it. Quite often you hear about the "new" orchestrator that aims to solve your orchestration needs. You can also often hear how it compares to Airflow. However those comparisions often overlook the fact that since Airflow 2.0 has been introduced, it continues to evolve and piece-by-piece modernize itself and respond to the needs of processing even more data, and interacting with even more systems that were not even existing a year ago (think LLMs) - while continue harnessing the powers of other tools you already used in the past and bind them all together. New UI, New ways of writing your orchestration tasks, new ways to test them, tracking lineage, simpler authoring and intracting with object storages. And the comparision often overlook that if you start your journey with Airflow today, your experience will be quite a bit different than even 2 years ago (what usually most comparisions talk about). This talk will highlight some of the important ones. Also you might be surprised but this is all happening without breaking compatibility - hence you can still you use it in the "old way". But maybe it's time to learn new ways? false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/8DMSUU/ Frannz Salon Talk 2024-06-10T12:00:00+02:00 12:00 00:40 OpenTelemetry is a standard for tracing across multiple components. Let’s see how to set it up across different stacks bbuzz24-44654-practical-introduction-to-opentelemetry-tracing Nicolas Fränkel en Tracking a request’s flow across different components in distributed systems is essential. With the rise of microservices, their importance has risen to critical levels. Some proprietary tools for tracking have been used already: Jaeger and Zipkin naturally come to mind. Observability is built on three pillars: logging, metrics, and tracing. OpenTelemetry is a joint effort to bring an open standard to them. Jaeger and Zipkin joined the effort so that they are now OpenTelemetry compatible. In this talk, I’ll describe the above in more detail and showcase a (simple) use case to demo how you could benefit from OpenTelemetry in your distributed architecture. false https://program.berlinbuzzwords.de/bbuzz24/talk/HMQMDH/ Frannz Salon Talk 2024-06-10T14:00:00+02:00 14:00 00:40 The talk explores managing transaction and validity times in data storage, using examples like retroactive creation of shipping documents. It highlights how bitemporal databases aid in this process and discusses the complexities of implementing such solutions. bbuzz24-42336-back-to-the-future-time-travel-with-bitemporal-databases Tim Zöller en For our data storage, there are two time axes that we, as software developers, must manage with great effort: the transaction time and the validity time of a data record. While the transaction time records when a data set was written to storage, the validity time indicates from when to when a data record should be considered in the program logic. When both of these time axes intersect, the complexity of our persistence and query logic often increases exponentially. An example can be a retroactively recorded salary increase: An employee in our company is granted a salary increase on March 14th, which is retroactively effective from January. The validity period is clear, but how do we explain to the tax office why the lower amount was transferred in January and February? What validity could we assume at the respective transaction time, and how can we replicate the case in our software? Databases that can map bitemporality can help us manage these two time axes. In this talk, I will illustrate how they can be used, which use cases we can represent with them, and how complex such solutions can become if we build them ourselves with application logic or database triggers. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/HEHE38/ Frannz Salon Short Talk 2024-06-10T14:50:00+02:00 14:50 00:20 The foundations of machine learning were built on offline batch processing techniques for model training and inference. As organisations become more dependent on real-time data, the technological trend for machine learning in production is moving towards adding an online stream processing approach. bbuzz24-45804-moving-from-offline-to-online-machine-learning-with-river Tun Shwe en The foundations of machine learning were built on offline batch processing techniques for model training and inference. As organisations become more dependent on real-time data, the technological trend for machine learning in production is moving towards adding an online stream processing approach. This has benefits such as lower computational requirements due to being able to incrementally learn from a stream of data points, which enables the continual upgrading of models by adapting to real-time changes in data. This has wide applications in industries such as cyber security, banking, healthcare, IIoT and any industry that involves processing large volumes of high throughput data and adapting predictive capability with real-time data feeds. This capability was once only in the realm of Java developers but now it's available to Python developers. You’ll leave this talk with an understanding of the differences between offline and online machine learning, how to complement one with the other and enough streaming concepts and best practices needed get started on your online ML journey with River, an open source Python ML library. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/XMDVVM/ Frannz Salon Talk 2024-06-10T15:20:00+02:00 15:20 00:40 From relevance guesswork to granular search experimentation at scale. Evaluate modifications to your search, such as incorporating synonyms, adjusting field-boosts, adding vector search, or product assortment modifications, on real data rather than relying solely on intuition. bbuzz24-44792-improving-search-scale-with-efficient-query-experimentation Andreas Wagner en Measuring the effectiveness of changes in search functionality, whether significant or minor, remains a persistent challenge. Even experienced teams often rely on search relevance feedback labels to gauge outcomes. While Web-scale Experimentation, such as A/B or Multivariate Testing, is widely practiced, only a few companies utilize it to enhance their search systems incrementally on a query-level basis. A/B Testing in the realm of search presents unique complexities. It involves various considerations and lacks a universal approach. Challenges include selecting appropriate metrics for evaluation, determining suitable randomization units, and addressing imbalanced and sparse experimental data to establish precise decision boundaries, especially with limited data availability. Over time, we've developed, refined, and operated our "Query Testing" capability. I aim to share insights gleaned from this journey. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/7D7YWT/ Frannz Salon Talk 2024-06-10T16:30:00+02:00 16:30 00:40 A great chance to hear the experience about the future of work from the biggest all-remote company in the world. Arm yourself with the best-practices toolbox for the async work tips and tricks and leverage the context from the distributed work leaders. bbuzz24-43253-remote-work-is-here-to-stay-and-what-s-next Radovan Bacovic en A great chance to hear the experience about the future of work from the biggest all-remote company in the world. Arm yourself with the best-practices toolbox for the async work tips and tricks and leverage the context from the distributed work leaders. From the GitLab Data Team member, a first-face story about how the asynchronous way of working helps us manage the “chaos” (as folks usually think about remote work) in the 24/7 work environment. Vibrant session to expose the picture of the pioneers and advocates of all-remote philosophy. Forget working hours, timesheets, offices, and emails and join the ride. Will show you how this way of work brings joy to me and my 1600 colleagues from 56 countries and how we serve 30+ million customers with exactly 0 offices - Is remote work just a perk? - Various approaches to remote strategies - Why “just” remote work is not enough? - What’s the next level for remote work? false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/7J3NMW/ Frannz Salon Talk 2024-06-10T17:20:00+02:00 17:20 00:40 This talk will discuss approaches to evaluate LLMs at the end-to-end and task levels, focusing on use cases such as question-answering (RAG). We will also cover metric selection and ways to generate datasets using LLMs. bbuzz24-45609-advancements-in-evaluating-large-language-model-applications Petr Polezhaev en The topic of generative AI has gained significant popularity in recent times. With the help of proprietary and open source LLMs, many companies, start-ups, and researchers are exploring the potential of these models to develop new products or enhance existing ones. They are utilizing prompt-engineering, fine-tuning, and model chaining techniques to implement their solutions. LLMs are widely used for various tasks such as text summarization, translation, stylization, information extraction, question answering, and content generation. However, the evaluation of these models is often limited to human-based assessments, which is not scalable and restricts the ability to conduct experiments. In this talk, we will delve into the approaches for performing evaluation at both the end-to-end and task levels, taking into consideration different types of tasks. We will examine several typical use cases, including question-answering (RAG), and explore in detail how evaluation can be conducted for them. Additionally, we will discuss metric selection, considering scenarios where we have labels/references and those where we do not. We will also explore the ways in which we can generate datasets using LLMs. false https://program.berlinbuzzwords.de/bbuzz24/talk/VEQGF8/ Kesselhaus Short Talk 2024-06-11T09:30:00+02:00 09:30 00:20 GenAI is transforming organizations rapidly. But how can we scale individual experimentation to enterprise-grade business products? This talk will cover learnings from hundreds of use cases in pharma about user intentions, information access patterns and quality metrics. bbuzz24-45790-learning-to-apply-generative-ai-at-enterprise-level Sebastian Arnold en GenAI is the latest buzzword promising to bring transformative capabilities to enterprise users at ready stage. With the current wave of commodity GenAI models, such as ChatGPT, organizations are tempted to translate all their business needs into prompts and deploy them as conversational chatbots. However, there are manifold challenges blocking us to unlock the full potential of GenAI, especially from the perspective of expert users with knowledge-based goals. This talk will cover learnings and techniques our pharma organization has developed from translating hundreds of use cases into GenAI prototypes. We introduce six common groups of user goals that we have identified in a study of 85 proposals and evaluated in dozens of production use cases. We break down their most critical challenges and propose reusable architecture patterns for speeding up prototyping, accessing corporate information and measuring result quality. false https://program.berlinbuzzwords.de/bbuzz24/talk/BFNXFY/ Kesselhaus Talk 2024-06-11T10:00:00+02:00 10:00 00:40 Curious about where unexpectedly Deep Learning is being used these days? Because not everything is ChatGPT, AI for search or to detect fraud in e-commerce, this talk will review, and build together state of the art DL architectures as the ones used by the Bundesliga. Help your local team with AI! bbuzz24-45677-deep-learning-plays-handball Pere Urbon Bayes en The use of machine learning in sports is nowadays a reality, not only in e-sports, but as well in professional sports, specially in the ones with financial power. Handball is however, the underrepresented small brother, just now taking the attention of the academic and industry. If you’re curious about how LSTMs, RNNs, Autoencoders or Transformers are used to help teams analyze their opponents, but as well to take real time decisions or even to score players performance, this is your talk. Learn how the profis take advantage of AI to help their results. Not to forget, lots of times, a traditional RandomForest could perform quite well on the tasks!. Let’s build together a model, using only the player coordinates, and observe the power that simple architectures have in this space. After this talk you will come out with a new, and probably unexpected use of an AI, and maybe with some tools to help you win more at FIFA or to help your local team! Certainly a fresh use case, outside of the more traditional business or health applications!. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/QQHZRA/ Kesselhaus Talk 2024-06-11T11:10:00+02:00 11:10 00:40 Celebrating Apache Lucene's 22nd anniversary, this conference explores its pivotal role in search and data technologies, from powering platforms like Solr, Elasticsearch and OpenSearch to recent AI synergies with vector indexing/search. Discover Lucene's evolution and future horizons. bbuzz24-45813-apache-lucene-from-text-indexing-to-artificial-intelligence Lucian Precup en Apache Lucene celebrated its twenty-second anniversary last September, a journey that continues to profoundly impact the world of Search and Data technologies. Lucene is the engine behind giants such as Elasticsearch, OpenSearch, Apache Solr, and the recent Atlas Search from MongoDB. Its integration into numerous other Open Source projects, such as Apache Nutch - the pioneering web crawler and precursor to Hadoop, and Apache Cassandra - the most scalable NoSQL database, attests to its widespread influence. Used in thousands of enterprise projects, including by leaders like LinkedIn and Twitter, Lucene enjoys a solid and diverse user base. The conference will dive into Lucene's evolution, from its essential inverted indexing for text processing to recent innovations that reflect continuous technological advancement. To conclude, we will discuss Lucene's latest features: vector indexing and vector search, which create a powerful synergy with generative artificial intelligence, opening new horizons for the future of search. false https://program.berlinbuzzwords.de/bbuzz24/talk/JMFLB7/ Kesselhaus Talk 2024-06-11T12:00:00+02:00 12:00 00:40 This talk aims to challenge the usual "code-centric" narrative around open source projects. That impact doesn't solely rest on lines of code but thrives within a diverse ecosystem of non-technical contributions. bbuzz24-44082-recognizing-the-driving-forces-of-impactful-os-projects Maryblessing Okolie en Come with me to discuss the driving wheels of open source projects. Six out of ten open source enthusiasts I interacted with see the open source space as code-centric. When they often think about contributors in open source, they think of code whether it is fixing bugs, adding features or developing something new. However, times are changing. Advocates like myself are championing the cause of no-code contributions in the open source community. The question that arises is “What happens when the focus shifts towards utilizing open source contributions as a gateway to employment?” Because adding new features and fixing bugs sound more glamorous and tick the green points on GitHub than documentation and providing support, This talk aims to challenge the usual "code-centric" narrative around open source projects. That impact doesn't solely rest on lines of code but thrives within a diverse ecosystem of non-technical contributions. By shedding light on the often-undervalued roles like documentation, outreach, and community management, the talk not only challenges the narrow definition of "contribution" but also shares ways to recognize the invaluable efforts of no-code contributors. In this talk, the insights and analysis shared are from Maryblessing's experience in the CHAOSS Project as the Tour Guide Initiative lead, member of the community managers team, member of the DEI working group and member of the CHAOSS Code of Conduct team. The CHAOSS Project perspective on no-code contributions and its proven approach to recognition. Participants in this talk who are open source project founders will walk away with a new perspective on no-code contributions and insights on how to better recognise them. Also, participants in this talk who are contributors will find reassurance in understanding the importance of their work and the motivation to keep at the pace and be their best advocate. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/UTCCPN/ Kesselhaus Talk 2024-06-11T14:00:00+02:00 14:00 00:40 Key practical insights across dataset construction, custom metrics, model building, and model de-biasing techniques behind the success of our real-time AI search ranking system which drives significant business impact and now serves as a centerpiece of Mercari’s Search & Discovery platform. bbuzz24-45470-robust-ai-search-ranking-for-radical-c2c-marketplace-growth Teo Narboneta ZosaChingis Oinar en In today's competitive e-commerce landscape, effective search ranking systems are no longer a luxury, but a necessity. At Mercari, Japan’s largest C2C e-commerce marketplace, AI search ranking is the flagship feature in our commitment to continuously integrate AI into search, driving significant engagement and GMV uplift in our pursuit to provide the best search experience for our millions of users. In this talk, we'll delve into key components of Mercari's search ranking system, specifically: * Dataset Construction: Demonstrating how we build a rich and diverse dataset incorporating user behavior, item attributes, and marketplace dynamics. We will show how we deal with implicit feedback specific to two-sided marketplaces with unique items. * Model Tracking & Monitoring: Showing how we approach tracking and monitoring our ML ranking models, including key custom metrics we’ve developed for robust evaluation. We will show how our custom metrics make our model debugging simpler and ensure our features have a measurable business impact in online testing. * Model Building: Sharing the technical details of our model building process, exploring what learning-to-rank is as well as our goals for optimal performance. * De-biasing through Historical Data: Outlining the potential problems inherent in implicit feedback and how we deal with potential biases in our ranking models through counterfactual learning. We will show how biased behavior affects the model learning process, what factors can trigger specific user behaviors, and how we mitigate these biases in a way that maintains and enhances model performance over time. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/7WXEJZ/ Kesselhaus Talk 2024-06-11T14:50:00+02:00 14:50 00:40 Everyone is unique, and that is especially true for what they eat. At Albert Heijn, we are moving away from popularity-based search for the broader customer base, in favor of tailoring search to our unique customers' tastes. We will tell you all about this transition. bbuzz24-45577-the-power-of-the-bonus-card-road-to-personalised-search Luuk KaandorpVincent Peijnenburg en Albert Heijn, the largest supermarket chain of the Netherlands, has a loyalty program, called the Bonus Card, allowing us to tie all purchased products to a customer whenever they scan the card, either in the store or online. This creates a huge potential for personalisation, which had previously not been utilised within product search. We will present about our journey going from popularity-based search for the broader customer base, to a tailored search experience for all of our unique customers using the information we gather from the Bonus Card. Specifically, we will focus on Learning-to-Rank (LTR). This transition was definitely not without it's challenges, on which we would love to share our experience: * Handling large quantities of data. Going from aggregated popularity to single user relevancy meant a million-fold increase in the quantities of data that we were handling. * Handling large amounts of point-in-time accurate features in offline feature stores. * Using distributed computing to train a model on this large quantity of data. * Redefining the concept of relevancy. How can we incorporate profitability? * Handling position bias in our data. * Using Kafka to facilitate the quick transfer of offline features to online features during inference. This presentation is relevant for anyone who is struggling to go from legacy popularity-based search to personalised search for big customer bases. You will learn how to face the challenges of moving to large quantities of personalized data, distributing a model to learn on this exploded quantity of data, and redefining the concept of relevancy. false https://program.berlinbuzzwords.de/bbuzz24/talk/AH7WKD/ Kesselhaus Talk 2024-06-11T16:00:00+02:00 16:00 00:40 Log-structured merge-tree databases rely on a combination of in-memory buffers and hierarchies of immutable on-disk structures, which are two cases where a trie data structure is most efficient. Join us for a glimpse at the ways in Apache Cassandra applies and benefits from this combination. bbuzz24-45446-applications-of-tries-in-apache-cassandra Branimir Lambov en Tries (also called Prefix or Radix Trees) are efficient data structures for representing maps. They offer a few advantages over more traditional comparison-based structures such as B-Trees, including compact representation and theoretically optimal lookup complexity. Despite their efficiency, tries have historically seen limited application in database technologies, in part due to their less favorable performance as mutable on-disk structures. A log-structure merge-tree (LSM tree) is a data structure tailored for storing an indexed map within a database. It leverages immutable on-disk structures commonly referred to as SSTables (Sorted String Table), organized in a hierarchy, coupled with an in-memory buffer, the memtable, which sorts and indexes as much data as possible in memory until it fills up and is written out to disk. Reads from an LSM tree merge data from all components; to keep read complexity in control, SSTables are reorganized from time to time in a process called compaction. This design allows LSM trees to exploit the difference between serial and random disk writes, to offer tunable trade-offs between read and write performance, and crucially for this talk, to avoid mutable on-disk structures. Apache Cassandra is a NoSQL database built on an LSM tree architecture. Recently we have started to take advantage of trie data structures in the implementations of the databases's memtables and SSTables to achieve significant improvements in many aspects of the database's performance. This talk will describe how we were able to replace comparison-based structures in Cassandra with tries, explore the reasons why tries are efficient at the task of organizing ordered data, and dive into the optimizations applied to suit the structure the specific applications, as well as the hardware and software environment that they target. We aim to demonstrate the practical applicability of tries in database systems, but also discuss the journey of turning a theoretical data structure into its useful practical implementation in a real-world computing environment. false https://program.berlinbuzzwords.de/bbuzz24/talk/EYULPE/ Kesselhaus Short Talk 2024-06-11T16:50:00+02:00 16:50 00:20 Have you unlocked the geospatial powers of PostgreSQL? This talk will show 3 neat features of how to use postgresql in effective ways to handle geospatial timeseries in a new domain. The data in subject is live marine traffic from the Norwegian territory. bbuzz24-44818-a-journey-in-geospatial-timeseries Nils Larsgård en Have you unlocked the powers of PostgreSQL yet? This talk will show 3 neat features of how to use postgresql in a somewhat unusual way to handle geospatial timeseries in an unknown domain. The talk will give an introduction to how to use json-types in postgresql, the postgresql-extensions postgis to handle geospatial data and the postgresql-extension timescaledb to handle the never-ending stream of geospatial data. The story of the talk is marine traffic in Norwegian waters with open data from the Norwegian Coastal Administration, presented in an application made by the autor. false https://program.berlinbuzzwords.de/bbuzz24/talk/UPCXA8/ Kesselhaus #BBuzz 2024-06-11T17:15:00+02:00 17:15 00:05 Join us as we wrap up Berlin Buzzwords 2024! bbuzz24-48819-closing-session en false https://program.berlinbuzzwords.de/bbuzz24/talk/XEHPK8/ Maschinenhaus Short Talk 2024-06-11T09:30:00+02:00 09:30 00:20 Introducing an open source library in Python: Quix Streams. It solves all the complexities of stream processing in a cloud native package with a familiar Pandas DataFrame API interface. This library lets you work with streaming data similarly to static data in your Jupyter Notebook! bbuzz24-45810-streaming-dataframes-a-new-way-to-process-streaming-data Tomáš Neubauer en Introducing an open source library in Python: Quix Streams. It solves all the complexities of stream processing in a cloud native package with a familiar Pandas DataFrame API interface. This library lets you work with streaming data similarly to static data in your Jupyter Notebook without any hassle associated with streaming technologies. Our mission is to bring masses of Python developers into streaming and make the journey as smooth as possible so real-time applications using ML are not so difficult. I will demonstrate this live on stage with examples of stateless operations and stateful operations like rolling windows and joins. Join me to learn how you can start working with streaming data today. false https://program.berlinbuzzwords.de/bbuzz24/talk/3NGGWM/ Maschinenhaus Talk 2024-06-11T10:00:00+02:00 10:00 00:40 Exploring and enhancing lexical search and semantic search in practical scenarios, we assess various optimization methods and their varied effects on metrics. The focus is on the integration of semantic search into an established lexical search system, addressing potential challenges and pitfalls. bbuzz24-44881-a-practical-approach-to-semantic-search Kentaro Takiguchi en The AI wave has hit the search community hard, making semantic search a hot topic. As a search engineer, you've probably worked on it too, developing PoCs or maybe even running online A/B tests in production. However, it's not uncommon to conclude that "Semantic search doesn't quite fit our domain," leading us back to the familiar territory of lexical search with its established synonym dictionary and knowledge graph. But why does semantic search often underperform both offline and online? This talk explores these critical questions, sharing insights from years of trial and error. We'll start with a detailed comparison of lexical and semantic search from various perspectives, including the validity of their evaluation metrics. A quantitative analysis will then highlight their strengths and weaknesses, providing a balanced view of both systems. A key part of our discussion will be how to integrate semantic search into existing lexical search systems. Most importantly, we'll address a crucial question: Is investing in semantic search integration really worth it? Attendees will leave with a clearer understanding of the semantic search landscape and the knowledge to make informed decisions about its implementation in their projects. false https://program.berlinbuzzwords.de/bbuzz24/talk/FBPVCX/ Maschinenhaus Talk 2024-06-11T11:10:00+02:00 11:10 00:40 This is a speedrun through the past ten years of R&D on approximate nearest-neighbor search algorithms and vector databases, covering the major advances, the current state of the art, and possible future directions. This will be informed by Joel's experience contributing to JVector, a Java embedded vector search library. bbuzz24-42756-under-the-hood-of-vector-search-with-jvector Joel Knighton en - ANN (approximate nearest neighbor) vector search is totally different from the problems that we're used to solving with databases. You can't just throw a B-tree at it and call it a day. - It's a super young field with arguably the most important breakthrough (HNSW) created only eight years ago. The best commercial products have started to ship designs based on DiskANN, which is five years old. - Nobody really knows the best way to solve several important problems (how to build indexes larger than memory? how to partition across machines? How to combine indexes to "garbage collect" obsolete data without a full rebuild?) - The importance of ANN to RAG (retrieval augmented generation) in generative AI is supercharging interest in the field and we should expect the state of the art to advance quickly. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/WHFU7W/ Maschinenhaus Talk 2024-06-11T12:00:00+02:00 12:00 00:40 Curious about NLP beyond the startup hype? Join us to explore NLP in a 'traditional' setting. Tackle challenges like data scarcity and domain specificity using e.g. data augmentation and zero-shot classification, and learn some tips and tricks to address concrete and relatable NLP problems bbuzz24-45652-hardcoding-airpods-and-other-stories-from-nlp-in-insurance Murhaf FaresEmanuele Lapponi en Tired of listening to NLP talks from people working at cool startups, FAANG, or contributing to sexy open source projects? Curious about NLP in a more traditional setting, where data is scarce, domain knowledge is king, and the stakes are high? Then you have come to the right place! In this talk, we present our experience with developing and deploying NLP systems at Fremtind, one of Norway's largest insurance corporations (we know, right? Insurance!). We will discuss challenges such as data scarcity, domain specificity and ad-hoc evaluation, and how we addressed them using different NLP techniques. We walk you through some of our projects, such as data augmentation using weak supervision, customer feedback classification and ranking, and zero-shot classification for dynamically changing label-sets. Finally, we will reflect on the nuances of real-life system evaluation, and how some of our solutions did or didn't change with the increasing ubiquity of LLMs. This talk should peak the interest of bbuzz participants who work in more traditional industries (we exist!), by providing tips and tricks to address concrete and highly relatable NLP challenges. It will also be highly relevant to tool and model developers, as it will provide insights on the actual challenges we face in a setting that is traditionally underrepresented in "cool" tech conferences. false https://program.berlinbuzzwords.de/bbuzz24/talk/9K7LGN/ Maschinenhaus Talk 2024-06-11T14:00:00+02:00 14:00 00:40 You've heard the wisdom of choosing boring technology, but how do you balance this with new usage patterns and ever-increasing scale? Learn how Slack developed a petabyte-scale log search engine with a team of just three engineers, supporting over one million queries per day. bbuzz24-45594-standing-on-the-shoulders-of-giants Bryan BurkholderVarun Thacker en In early 2021, and after yet another incident caused by a large spike in log volume, it was clear that something had to change. Facing a near-constant 50% year-over-year volume growth and rapidly mounting operational overhead, the Slack Observability team made the decision to build a new open-source log search engine. Building a new log search engine is a large undertaking, and there are many features that engineering teams require before replacing an existing solution. Many of these are solved problems however, with projects like Apache Lucene and Opensearch. At Slack our focus became how we could build on these existing, boring technologies to solve our unique scaling challenges better. This enabled us to support features specific to log search, and ease our team’s operational challenges by building a solution that was tailored for Slack infrastructure. In this talk we will discuss how to identify areas where using perceived boring technology can reduce risk and accelerate development. Learn how to do more with fewer people, and how to see further, standing on the shoulders of giants. false https://program.berlinbuzzwords.de/bbuzz24/talk/PFNRQV/ Maschinenhaus Talk 2024-06-11T14:50:00+02:00 14:50 00:40 Most of the foundational tools we use in this industry are open source. But CI/CD is a domain where open source has stagnated and most innovation is happening in proprietary code. This talk discusses why this is, why it's dangerous, and what we can do about it. bbuzz24-45814-the-c-in-ci-cd-is-not-for-closed Josh Reed en The CI/CD landscape is increasingly dominated by closed source offerings. This is unlike many other tools developers use. If I need an OS to host my application, I have Linux. If I need a database, Postgres. If I need to stream data, why not Kafka? But if I need CI/CD? There are open source options, but high performing teams increasingly choose closed source SaaS products. Even worse, closed source CI systems are the default option for most open source projects these days, leading to a situation where a staggering chunk of open source software is built by closed source tools. Paradoxically, you’re far more likely to encounter an OSS system building your organization’s private code than you are to see it building the OSS libraries you use. It wasn’t always this way. This talk will answer questions about this: How did we get there? What trends drove this change? How can we fix this? Should we even care? I argue that we can beat this by focusing on better build and observability tools, over integrated CI/CD solutions. I call on this community to change how we think about our projects, so that we don’t end up with private control over community builds. false https://program.berlinbuzzwords.de/bbuzz24/talk/CXCJAB/ Maschinenhaus Talk 2024-06-11T16:00:00+02:00 16:00 00:40 We present the lessons learnt from improving the search of our global online marketplace with 20 million products sold per year. We successfully moved from a traditional word-match based approach (BM25) to a modern hybrid solution by adding a semantic vector model which we fine-tuned to our domain. bbuzz24-45562-from-text-to-context-how-we-introduced-hybrid-search Ansgar GrueneDharin ShahAlexander Butenko en With numerous references to current literature, we will explain how we designed our new system and solved the multiple challenges we encountered on both the ML and engineering side (data pipeline encoding documents, live service encoding queries, integration with search engine) as well as sharing insights from analyzing the impact. Our system is based on OpenSearch, the lessons can be applied to other search engines as well. To be more specific, the presentation will cover: - Status and Short-Comings of our old Search - Introduction of Hybrid Search - general setup - recommendations from literature - Machine Learning - model decision (quality vs. latency) - fine-tuning and offline evaluation (in particular: using Paid Search / SEM data if you have few historic own search performance data) - Architecture and Implementation: (with special consideration of latency) - pipeline for encoding documents and indexing the resulting vectors (PySpark) - service for live-encoding of queries (Python) - implementing hybrid search within OpenSearch (including important filter value extraction from query and ranking scores) - Learnings and Next Steps: - observations from our A-B test - challenge of cut-off decisions - realistic training / evaluation data - filter value extraction from query vs. semantic search - combining search with auto-complete - impact of the call to action in the search bar - for which other use cases we successfully apply such semantic vector approaches false https://program.berlinbuzzwords.de/bbuzz24/talk/EBM7XK/ Maschinenhaus Short Talk 2024-06-11T16:50:00+02:00 16:50 00:20 Explore RAG systems: Understand evaluation criteria, tools, and frameworks. Discover essential components and compare them with existing frameworks. Gain insights for enhanced control in your path toward RAG optimization. bbuzz24-45766-cracking-the-code-deciphering-evaluation-essentials-for-rag Atita Arora en Join us for an enlightening exploration into the world of Retrieval Augmented Generation (RAG) systems, where we uncover the intricacies of evaluation criteria and demystify the role of existing tools and frameworks. In this presentation, we'll dissect the essential components necessary for robust RAG evaluation, contrasting them with the offerings of current frameworks. From semantic understanding to content coherence, we'll highlight what truly matters in the evaluation process, empowering you to make informed decisions in the ever-evolving landscape of RAG systems. false https://program.berlinbuzzwords.de/bbuzz24/talk/YSJLBF/ Palais Atelier Short Talk 2024-06-11T09:30:00+02:00 09:30 00:20 We all know there are several vector databases, and once Andrej Karpathy said that even an array can do the same job, true but not true. Vector database provide you the infra to store the embeddings but how those embeddings are made are the most innovative part of all. bbuzz24-43031-the-unsung-hero-of-vector-database-metric-learning Sonam Pankaj en The unsung Hero of vector database. And I am talking about a machine learning concept not one of the companies (because there are so many) 1. Metric learning: What is metric learning? 2. Problem without metric learning, mostly with an example of negation. Cosine similarity doesn't work with negation. 3. How to train metric learning embeddings. 4. Data, Model, and the loss function 5. Data, what is anchor, positive and negative 6. Model: Siamese networks, since we deal with different data, which needs different architecture. 7. Loss: the loss function is big, triple loss, and contrastive loss. 5. Demo of how it improved the overall experience of working with negations. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/UVMCE3/ Palais Atelier Talk 2024-06-11T10:00:00+02:00 10:00 00:40 This talk introduces OpenSearch's Flow Framework and RAG (Retrieval-Augmented Generation) Tool, enabling developers to create AI-augmented agent, queries, and ingestion flows, integrate ML models, and streamline app development through a no-code/low-code builder bbuzz24-51163-elevating-ai-applications-with-opensearch-s-flow-framework-and-rag-tool Mingshi LiuOwais Kazi en This talk introduces OpenSearch's Flow Framework and RAG (Retrieval-Augmented Generation) Tool, enabling developers to create AI-augmented agent, queries, and ingestion flows, integrate ML models, and streamline app development through a no-code/low-code builder. The Flow Framework offers a template for composing AI-augmented flows and integrating ML models. The RAG Tool combines large language models with relevant data from your indexes, enabling question-answering that truly understands and reasons over your proprietary data. Attendees will gain insights into building chatbots, question-and-answer search systems, and other AI-driven applications by leveraging the capabilities of OpenSearch's Flow Framework and RAG Tool. <br> --- <br> This session is sponsored by <a href="https://opensearch.org">OpenSearch</a> false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/SRSS7M/ Palais Atelier Talk 2024-06-11T11:10:00+02:00 11:10 00:40 This talk will focus on the exploration of serverless Kafka and real-time data streaming with Node.js microservices. In this talk, I will delve into Upstash's serverless Kafka architecture and how it can be leveraged to build scalable and cost-effective solutions for microservices. bbuzz24-44133-kafka-on-the-fly-a-serverless-approach-to-data-streaming Desmond Obisi en Description: In this session, I will introduce the audience to the concept of serverless computing and its benefits when combined with Kafka for real-time data streaming. We will dive into the specifics of using Upstash, a cutting-edge serverless data streaming platform, to create a truly scalable and event-driven architecture with Kafka. Throughout the session, I will cover the following key points: 1. Understanding the fundamentals of serverless computing and its advantages 2. Exploring the integration of Kafka with serverless architecture 3. Introducing Upstash - a powerful open-source serverless Data Stream Platform 4. Demonstrating the setup and configuration of Kafka with Upstash 5. Showcasing real-life use cases of Kafka on the Fly with Upstash By the end of this talk, attendees will gain a comprehensive understanding of how serverless Kafka with Upstash can revolutionize real-time data streaming, simplifying deployment, reducing infrastructure costs, and enabling seamless scalability. Key Takeaways: - Insight into the benefits of serverless computing for real-time data streaming with Kafka - Understanding the integration and setup of Kafka with Upstash - Knowledge of best practices and use cases for serverless Kafka architecture - Inspiration to leverage serverless technologies for scalable and cost-effective solutions. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/QLNHDA/ Palais Atelier Talk 2024-06-11T12:00:00+02:00 12:00 00:40 I'll talk about all the mistakes we made and all the lessons we learned developing a data platform on top of a dozen or so Kubernetes operators so you don't have to go through the same ordeal. bbuzz24-45026-lessons-learned-writing-10-kubernetes-operators Lars FranckeJannik Heyl en We have written more than 10 operators for Kubernetes. When we started our journey in 2020, we thought lots of people surely must have done this already. We could not have been more wrong. As it turns out, most people write a single operator: For their product. All of them slightly different. Everyone has their own way of specifying Pod overrides, storage, TLS or LDAP support, log aggregation, monitoring, and so on. This is before we have entered the land of horror that is Kerberos, multi-homing and various other fun topics like certifications. We knew that we would be writing more than one we made sure to move as much functionality into a common framework as possible. That is also the reason we did not pick Go as our programming language of choice but are using Rust instead. Its support for generic (which Go lacked at that point) made it much easier to extract common functionality. In this talk we will tell you about some of the things we learned while writing our operators, pitfalls, open questions, and a plea to collaborate on some standards in the operator world. We – as a community – can do better than today's state, I am sure of it. But you will not only hear the developer’s side, but you will also hear the side of a user and partner of ours who appreciates the ease of use across multiple operators and utilize the Stackable platform to build on prem data platforms for their customers. Would we do it again? Yes. Would we do it differently? Also, yes. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/L8JAWX/ Palais Atelier Talk 2024-06-11T14:00:00+02:00 14:00 00:40 Embeddings transform text into numerical vectors, capturing semantic relationships. This talk explores the data preparation, training processes, evaluation, and a demo of Jina Embeddings V2 in a hybrid search pipeline, showcasing its practical applications. bbuzz24-45519-jina-embeddings-v2-from-raw-data-to-bilingual-hybrid-search Bo WangIsabelle Mohr en In this talk, we explore the sophisticated design, training, and application of bilingual Jina Embeddings V2, the state-of-the-art German-English embedding model crafted here in Berlin. Acknowledging the inherent shortcomings of traditional exact match and term-based retrieval methods, we dive into the application of this bilingual model in a hybrid search setup. By combining vector-based search with conventional BM25 search, we harness the strengths of both approaches, leading to a marked enhancement in search results. This discussion is therefore highly relevant to anyone in the search field. Participants gain insights into the training processes of embedding models, the methodologies for sourcing and preparing data for these models, and the straightforward integration of our open-source German-English bilingual model into a search pipeline to enhance results. This talk is aimed at those keen on the latest in search and retrieval technologies, offering practical knowledge on improving search systems through the use of embeddings. false https://program.berlinbuzzwords.de/bbuzz24/talk/NRS3ZU/ Palais Atelier Talk 2024-06-11T14:50:00+02:00 14:50 00:40 This talk presents an experimental serverless Solr indexer where the records from multiple database tables are merged using the map-reduce approach. The merge is implemented in AWS using Step Function Distributed Map to orchestrate many lambda functions. bbuzz24-45564-blazing-fast-serverless-mapreduce-indexer-for-apache-solr Daniele Antuzi en Indexing data from databases to Apache Solr has always been an open problem: for a while, the data import handler was used even if it was not recommended for production environments. Traditional indexing processes often encounter scalability challenges, especially with large datasets. In this talk, we explore the architecture and implementation of a serverless MapReduce indexer designed for Apache Solr but extendable to any search engine. By embracing a serverless approach, we can take advantage of the elasticity and scalability offered by cloud services like AWS Lambda, enabling efficient indexing without needing to manage infrastructure. We dig into the principles of MapReduce, a programming model for processing large datasets, and discuss how it can be adapted for indexing documents into Apache Solr. Using AWS Step Functions to orchestrate multiple Lambdas, we demonstrate how to distribute indexing tasks across multiple resources, achieving parallel processing and significantly reducing indexing times. Through practical examples, we address key considerations such as data partitioning, fault tolerance, concurrency, and cost. We also cover integration points with other AWS services such as Amazon S3 for data storage and retrieval, as well as DynamoDB for distributed lock between the lambda instances. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/A9E3XU/ Palais Atelier Talk 2024-06-11T16:00:00+02:00 16:00 00:40 Modifying SolrCloud to separate compute and storage presents challenges but is cost effective and improves scalability and availability. Nodes become stateless and durability is delegated to the shared storage implementation. Salesforce runs such clusters at scale. The code is open source. bbuzz24-44962-search-in-the-cloud-separation-of-compute-and-storage Ilan Ginzburg en The traditional architecture of a search cluster, with nodes providing compute and persistent storage, ends up being more expensive to run than it should, especially in the context of public cloud and when dealing with unpredictable traffic patterns/load. This presentation is about modifying the implementation of SolrCloud so a single copy of each index is stored on shared storage (for example S3) and accessed by all nodes for reads and updates. The nodes are then stateless, which means any one (or all) of them can be turned off at any time without data loss. New nodes added to the cluster get their data from the shared storage and not from other nodes. Note that many aspects of this new architecture apply to other search clusters, and no prior knowledge of SolrCloud is assumed. Such an approach is a good match for running Search on a public cloud infrastructure, as it allows for compute and storage to be scaled independently and decreases the overhead of bringing a new node up in the cluster. The challenges faced in the process of separating compute and storage will be detailed. The hard problems concern updates (indexing), and protecting the shared storage from conflicting concurrent updates, preventing data loss or overwrites yet allowing concurrent updates. Tradeoffs for integrating such a large conceptual change into an existing system will also be discussed. The existing code has limitations that will be presented, as well as a high level design and plans for lifting them. The current code is available (see SIP-20 https://cwiki.apache.org/confluence/display/SOLR/SIP-20%3A+Separation+of+Compute+and+Storage+in+SolrCloud for code and documentation). The aim is to eventually merge these changes into the main Solr branch. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/8Q7RU9/ Palais Atelier Short Talk 2024-06-11T16:50:00+02:00 16:50 00:20 The Open Web Search Initiative (OWS.eu) is a European initiative providing an open platform to foster innovation in search and AI applications. Join us for an introduction to OWS, its public datasets and an intro to two of the open source projects that underpin it - StormCrawler and URLFrontier. bbuzz24-45456-open-web-search-a-platform-for-a-free-european-web-index Michael Dinzinger en In recent years, the Internet and the digital world have become extremely important in our everyday life as almost every aspect of our daily routine is tied to digital resources. Those resources are currently curated by a small number of non-European gatekeepers. In the matter of Web Search, this results in limited access to information, provided by companies that are primarily focused on commercial success rather than taking the individual needs and societal values into consideration. The dominance of a few can lead to polarisation and disinformation. Consequently, the concept of information as a public good, with freely accessible and transparent content is no longer under public control. The objective of the Open Web Search Initiative (OWS.eu) is to create an Open Web Index (OWI) – open in contrast to the proprietary, closed web indices of the large commercial providers. The forthcoming web index is planned as a public infrastructure. The underlying software will be shared as open-source code and its content undergoes public and transparent moderation. The Open Web Index is specifically aimed at companies, NGOs, scientists and other organisations in Europe to develop new search applications and AI applications. In our session, we will first introduce the project, its objectives, and the open data it generates. To bring the project idea closer to the audience in a practical way, we plan to conduct a live demonstration of the functionalities of the public dashboard. This web service enables users to configure custom indices and download these at regular intervals to their systems. User-customised indices can encompass the entire Open Web Index or focus only on specific parts of the web, allowing for countless use cases. For instance, researchers interested in structured web data can offload documents containing JSON-LD metadata, whereas companies may want to create topic-specific search engines within the scope of their particular domain. In this way, the data we collect becomes intuitive and easily usable without significant technical overhead. With the live demo, we want to foster a hands-on understanding of the Open Web Index's potential. After presenting the “what we are doing”, we will delve into the technical details and the “how we are doing it”. Our focus lies on the data collection using the OWLer (Open Web Crawler), a distributed crawling system implemented with the help of the open-source projects StormCrawler and URLFrontier. StormCrawler is a SDK that allows developers to construct their own efficient web crawlers on top of an Apache Storm cluster. This results in distributed crawlers that are both scalable and high-performing. Moreover, they integrate seamlessly with the URLFrontier framework, which offers a crawler-agnostic API and abstracts the web frontier data structure - the queue for the next URLs to crawl. Both software projects, StormCrawler and URLFrontier, are the backbone of the technical setup of our Open Web Crawler. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/DUDQWG/ Frannz Salon Short Talk 2024-06-11T09:30:00+02:00 09:30 00:20 Spare Cores, a Python-based open-source ecosystem, offers standardized inventory and performance evaluations for compute resources of public cloud and server providers, helping with selecting and starting the optimal instance type for containerized tasks, such as ML model training or ETL processes. bbuzz24-45349-harnessing-spare-cores-to-breeze-through-cloud-compute Gergely Daroczi en Spare Cores, an innovative Python-based open-source ecosystem, provides a comprehensive and standardized inventory, along with performance evaluations of available compute resources across public cloud and server providers. In this talk, we demonstrate how Spare Cores can help identify the optimal instance types across various vendors and datacenters, ensuring efficiency for containerized tasks (such as training machine learning models using TPUs, rendering videos on GPUs, executing ETL processes requiring a lot of RAM, or running computationally heavy microservices), using either our public or self-hosted web application, Swagger/OpenAPI-documented APIs, or the open-source SDKs for several programming languages, accompanied by user-friendly CLI helpers to launch instances within your existing environment effortlessly. We also briefly showcase a streamlined SaaS solution for those seeking simplicity and/or unwilling to manage their cloud infrastructure. The managed Spare Cores environment covers the entire life cycle of batch jobs and microservices, eliminating the need for direct vendor engagement. Spare Cores is a new project started in Q4 2023, part of the NGI Search Open Call #3 https://www.ngisearch.eu/view/Events/OC3Searchers false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/ZDLK3M/ Frannz Salon Talk 2024-06-11T10:00:00+02:00 10:00 00:40 Nearly half of data scientists find it difficult to adopt streaming technologies. The audience will learn about the technological barrier and the operational burden that inspired us to look for better solutions, and how we built a stream-batch unified Python dataframe API. bbuzz24-45771-streaming-doesn-t-have-to-be-hard Chloe He en Machine learning is moving towards real-time. In online prediction, real-time observability, and continual learning, streaming technologies play an indispensable role. Hence, data scientists are increasingly exposed to streaming technologies. However, our surveys show that close to half of data scientists shy away from streaming technologies because “streaming is hard, both technically and operationally”. Unlike batch systems, which operate on bounded data, streaming systems are designed with unbounded data in mind and operate with unique concepts such as “event time”, “processing time”, “watermark”, etc. Furthermore, making batch and streaming systems work together is a difficult challenge. For a large percentage of data scientists and companies, despite a high potential of return on investment, the barrier to adopt streaming technologies into existing batch workloads is too high. We faced the same challenges and set out to find better solutions. Since batch is just a special case of streaming, we believe in a world where data scientists should be able to define feature logic without worrying about whether the workload will be deployed in batch or streaming. Inspired by Flink SQL, our solution is a stream-batch unified Python dataframe API that is decoupled from backend execution. This allows us to write the transformation logic once, parse it into a standard query plan, and execute it everywhere, across streaming and batch engines (even across different languages and systems). We don’t need to worry about differences in dialects, query plans, and how computations are carried out underneath the hood. Introducing a single, unified API lowers the barrier of entry for data scientists who are new to streaming and companies who are looking to get started with streaming. It protects data scientists and platform engineers from elusive bugs that often arise from intricate discrepancies across engines. It also reduces vendor lock-in and allows us to choose the best engine for each workload without ridiculously high switching costs. Through this work, we aspire to close the gap between batch and streaming and work with the community to push for better unification standards. This talk is for data scientists who are new to streaming and companies that are interested in learning more about and adopting streaming technologies. The audience will learn about basic streaming concepts, how we created an open-source library that unifies batch and streaming, and how they can use this to adopt streaming technologies into existing batch workloads with minimal code changes. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/XB78T3/ Frannz Salon Talk 2024-06-11T11:10:00+02:00 11:10 00:40 I will discuss advanced retrieval augmented generation(RAG) techniques such as hybrid search, re-ranking, query generation, and semantic text chunking. bbuzz24-44149-advanced-retrieval-augmented-generation-techniques Zain Hasan en Chatbots are becoming increasingly popular for interacting with users, providing information, entertainment, and assistance. However, building chatbots that can handle diverse and complex user queries is still a challenging task. One of the main difficulties is finding relevant and reliable information from large and noisy data sources. In this talk, I will present some of the latest advances in retrieval-augmented generation(RAG) techniques, which combine the strengths of both retrieval-based and generative approaches for chatbot development. Retrieval-based methods can leverage existing text documents to provide informative and coherent responses, while generative methods can produce novel and engaging conversations personalized to the user. I will cover the following topics: 1. Hybrid search with vector databases: How to use both keyword-based and semantic-based search methods to retrieve relevant documents from large-scale vector databases. 2. Query generation using LLMs: How to use large language models to generate natural and effective queries for document retrieval, based on the user input and the dialogue history. 3. Automatically excluding irrelevant search results: How to use various filtering and ranking techniques based on vector distance to exclude irrelevant search results. 4. Re-ranking: How to dynamically re-rank retrieved documents to further improve context relevance. 5. Chunking Techniques: How to use text segmentation and summarization methods to chunk long documents into shorter and more relevant passages. I will demonstrate the effectiveness of these advanced techniques in the RAG workflow. I will also discuss the challenges and limitations of these techniques and the future directions for research and development. false https://program.berlinbuzzwords.de/bbuzz24/talk/EZ3EZZ/ Frannz Salon Talk 2024-06-11T12:00:00+02:00 12:00 00:40 Explore how leveraging a knowledge graph combined with vector indices can enhance semantic search and retrieval-augmented generation (RAG), enabling global insights beyond document boundaries bbuzz24-45818-enhancing-rag-with-neo4j-knowledge-graph Djordje Benn-Maksimovic en In this talk, we'll examine the synergy between knowledge graphs, semantic search and retrieval-augmented generation. We will investigate how integrating an automatically generated knowledge graph derived from a corpus of documents enhances RAG capabilities. Specifically, we will look at how this integration can overcome common limitations of standard RAG approaches, enabling the extraction of "global" trends and insights or those that stem from combining pieces of information from several documents (i.e. ones that are impossible to answer by simply injecting information from a number of unconnected retrieved documents). As tools we will use Neo4j graph database as the knowledge graph combined with vector indices, Langchain for business logic flow. false https://program.berlinbuzzwords.de/bbuzz24/talk/KS3PAC/ Frannz Salon Talk 2024-06-11T14:00:00+02:00 14:00 00:40 We explore the evolving role of product management in open-source generative AI, focusing particularly on Retrieval Augmented Generation's (RAG) industry impact. We cover community-commercial balance, innovative AI product discovery, and sustainable monetization strategies. bbuzz24-45515-open-source-generative-ai-a-product-manager-s-blueprint Saahil Ognawala en In this talk, we dissect the complex role of product management within open-source generative AI, an area brimming with innovation yet fraught with unique challenges. - We start by setting the stage with the current surge of generative AI, and the proliferation of vector search and RAG use-cases. - We highlight the distinctions between product and model offerings from commercial AI startups and their open-source counterparts, and emphasize the critical role open-source plays in innovation. - We delve into the nuances of product management pivots required in this space, including strategic community engagement, navigating rapid competitive shifts, and developing sustainable business models. - Using examples from leading open-source projects, from past and present, we illustrate how these challenges can be turned into opportunities for growth and innovation. - Attendees will leave with a comprehensive understanding of the open-source AI landscape, equipped with strategies to leverage community resources effectively, adapt product offerings in line with evolving tech trends, and craft monetization approaches that align with the open-source ethos. This session aims to empower leaders with the knowledge and tools to lead their projects to success in the competitive world of generative AI. false Slides https://program.berlinbuzzwords.de/bbuzz24/talk/JREKNJ/ Frannz Salon Talk 2024-06-11T14:50:00+02:00 14:50 00:40 Learn how sequential testing has cut our experiment durations by nearly 40%. We'll delve into the core principles, tackle real-world implementation issues, and unveil Basemath, our Python-driven open-source tool. Gain insights on refining your experimental analysis for smarter feature deployment. bbuzz24-45655-sequential-testing-simplified-with-basemath Alexander WeissKonrad Richter en At GetYourGuide, we've achieved a 40% reduction in experiment timelines by adopting sequential testing, a method that aligns intuitive decision-making with statistical accuracy. This approach allows for dynamic stopping rules based on real-time data, ensuring the validity of statistical conclusions. Sequential testing is increasingly popular in the tech industry for its ability to streamline the traditional A/B testing process, which typically requires a fixed sample size and can lead to unnecessary delays when experimenters have an early indication of an experiment's outcome. In our upcoming presentation, we will delve into the foundational concepts of sequential testing and share practical experiences from our own implementation, including the challenges we faced. We'll introduce Basemath, our own approach to sequential testing that eliminates the need for repeated numerical integration, enhancing efficiency and accessibility. Basemath's Python-based implementation speeds up the evaluation of multiple concurrent experiments and is more user-friendly than traditional C++ code. As an open-source project, Basemath is available to the broader community, offering a valuable tool for anyone seeking to optimize their experimental analysis. Join us to learn how sequential testing can transform your A/B testing strategy and help you make more informed decisions about feature rollouts. false [Preprint] Basemath’s Test: Group Sequential Testing Without Recurrent Numerical Integration https://program.berlinbuzzwords.de/bbuzz24/talk/8KBZUL/ Frannz Salon Talk 2024-06-11T16:00:00+02:00 16:00 00:40 This talk equips developers with everything they need to understand and optimize the cost of Kafka workloads in the era of cloud computing. bbuzz24-45774-taming-the-cost-of-kafka-workloads-in-the-cloud Stefan Sprenger en Cloud computing has changed how we operate and think about software: At any time, we have instant access to a seemingly unlimited capacity of compute resources and can scale our applications elastically on demand. We no longer have to procure physical machines and pay for them upfront but are billed for only those resources that we have actually used. While the usage-based pricing sounds great at first glance, it makes cost planning a complicated task, often leading to huge surprises when receiving invoices for cloud services, as revealed in numerous blog posts of authors waking up to invoice amounts that are multiple orders of magnitude higher than expected. This talk explores the main cost factors of Kafka workloads in the cloud: compute, network, and storage. We identify their contribution to the overall cost and walk through different techniques for reducing their cost footprint by, for instance, using Kafka's follower fetching to reduce cross-AZ traffic, compression to reduce overall traffic, or scaling streaming applications "to zero" in the absence of incoming events to save compute resources. We focus on setups where both Kafka and associated workloads, such as Kafka Streams or Flink applications, are operated on cloud platforms. The goal of this talk is to enable developers to benefit from the advantages of cloud computing in the context of Kafka workloads while taming the associated costs. false https://program.berlinbuzzwords.de/bbuzz24/talk/WG3WZV/ Frannz Salon Short Talk 2024-06-11T16:50:00+02:00 16:50 00:20 We've recently seen a boom of specialized vector databases. At the same time, almost all popular database projects have added support for vectors. So a lot of people are asking themselves if and when do they really need a specialized vector databases, and when could use an already deployed tool. bbuzz24-44407-comparing-vector-implementations-in-generic-databases Tudor Golubenco en We're going to look in particular at (at least) two vector search implementation in popular tools that a lot of people already use: * pgvector for PostgreSQL * Lucene vector implementation for Elasticsearch and OpenSearch We recently had to evaluate the two for a particular use case and the comparison is quite interesting, there are pros to each, for example: * pgvector means less infra and cost, and is always strongly consistent * Elasticsearch/Opensearch can do automatic sharding * in postgres you can shard by tenant easier by using schemas or partitioned indexes * Lucene can combine functionality with full-text search We'll go through the above and also discuss when going for a dedicated vector DB makes sense. false https://program.berlinbuzzwords.de/bbuzz24/talk/BVJPUX/