Berlin Buzzwords 2025

Text Search on Images with Quantized ColPali
2025-06-17 , Kesselhaus

ColPali is revolutionary—here’s why: it combines document retrieval with a vision-based large language model, allowing you to search directly within images without needing to extract text. However, running the full model on personal hardware can be challenging due to its computational demands. And thus we’ve released a quantized version of ColPali.


ColPali is a late interaction model, that is the context remain intact. And it's finetuned on vision LLM, Pali Gemma to be able to perform text search on images. But what we did was to be able to bring it more towards consumer by quantizing the model, so you can perform search locally on your laptop.

The talk will cover:
What is ColPali?
What is Late-Interaction?
How can you deploy it locally?


Tags:

Search, Data Science

Level:

Advanced

Sonam is a GenerativeAI Evangelist. She is also the author of embedanything, which is an opensource ingestion, inference and indexing solution in rust with more than 200k+ downloads and 500+ stars in past 9 months. She has previously worked in generative AI and conversational AI. She is also building StarlightSearch, a local and on-premise solution for search and agents in rust.