Berlin Buzzwords 2025

Why Chatbots Still Fail: The Hidden Pitfalls of RAG
2025-06-17 , Palais Atelier

Many knowledge chatbots and search engines use RAG. Despite their popularity, these chatbots are often worse than ChatGPT and frustrate users by failing to answer even the simplest questions. In my talk, I reveal how ineffective chunking strategies are a key culprit and demonstrate how to refine chunking to build more reliable RAG systems.


Large Language Models (LLMs) have experienced a significant boom. Among the most popular use cases are intelligent knowledge search engines, or put simply - chatbots. Whether on an airline's website, facing customers, or as the new search tool in your company's intranet, chatbots are everywhere. However many applications fall short of expectations. The system used behind many knowledge applications is called Retrieval Augmented Generation (RAG), in which a specific database is connected with an LLM. Many strategies have emerged to enhance RAG performance, but the core is often overlooked—the data itself. In my presentation I will explain RAG as the foundation of intelligent knowledge applications, its pitfalls and caveats. Using a RAG's first step - chunking - as an example, I show what is necessary to improve the reliability and robustness of RAG systems and what you absolutely have to do before you can trust your own chatbot.


Tags:

Data Science, Stories

Level:

Intermediate

Entrepreneur in the AI space, ex-McKinsey

Entrepreneur in the AI Space, ex-McKinsey, Medical Doctor