2025-06-17 –, Kesselhaus
ColPali is revolutionary—here’s why: it combines document retrieval with a vision-based large language model, allowing you to search directly within images without needing to extract text. However, running the full model on personal hardware can be challenging due to its computational demands. And thus we’ve released a quantized version of ColPali.
ColPali is a late interaction model, that is the context remain intact. And it's finetuned on vision LLM, Pali Gemma to be able to perform text search on images. But what we did was to be able to bring it more towards consumer by quantizing the model, so you can perform search locally on your laptop.
The talk will cover:
What is ColPali?
What is Late-Interaction?
How can you deploy it locally?
Search, Data Science
Level:Advanced
Sonam is a GenerativeAI Evangelist. She is also the author of embedanything, which is an opensource ingestion, inference and indexing solution in rust with more than 200k+ downloads and 500+ stars in past 9 months. She has previously worked in generative AI and conversational AI. She is also building StarlightSearch, a local and on-premise solution for search and agents in rust.