Berlin Buzzwords 2024

cuVS and Lucene: GPU-based Vector Search
2024-06-10 , Kesselhaus

This talk focuses on how we leveraged Nvidia’s open source cuVS library to accelerate Apache Lucene’s vector search capabilities on the GPU. cuVS contains several algorithms for approximate nearest neighbors and clustering on the GPU and we show how it pairs well with Lucene, which has long been the


By 2025, it is believed that 80% of all data will be unstructured. Meanwhile, modern advances in large language models and generative AI have acted as a catalyst for the use of vector embeddings, which are impacting nearly every industry. Vector search is growing in popularity and, thanks to these advances, the need for efficient and scalable semantic search is becoming more evident. GPUs have become synonymous with AI over the past decade, but even more intriguing is the advancement of software capabilities that can leverage GPUs to accelerate more general-purpose data processing workloads like vector search.

cuVS, from Nvidia [1], is a CUDA-based library containing state of the art implementations of several algorithms for approximate nearest neighbors and clustering on the GPU. Apache Lucene [2] is an open-source search library that is at the core of popular search engines like MongoDB, Elasticsearch, OpenSearch and Apache Solr.

This talk will have two main parts, 1) Introduction to Nvidia’s cuVS library, its history, approximate nearest neighbor search algorithm types, their implementations and comparison, the novel graph-based CAGRA algorithm, and the cuVS future roadmap. 2) Integration of cuVS into Apache Lucene to power GPU accelerated vector search, the motivations, challenges, the roadmap associated with this integration, and potential future directions for turbo-charging Lucene on the GPU.

We will provide benchmarks and lessons we've learned along the way, and hope that this will spark a new trend where GPUs can be used to accelerate other compute-heavy search, analytics and database tasks in the future.

[1] - cuVS, https://github.com/rapidsai/cuvs, was formerly part of RAFT, https://github.com/rapidsai/raft
[2] - https://github.com/apache/lucene

Corey is a principal engineer on the RAPIDS ML team at NVIDIA, where he builds machine learning algorithms that support extreme data loads at light speed. Prior to joining NVIDIA 5 years ago, Corey spent over a decade building massive-scale exploratory data science & real-time analytics platforms for big-data and HPC environments in the defense industry. Corey holds Bs. & Ms. degrees in Computer Science. He is also finishing up his Ph.D. in the same discipline, focused on the acceleration of algorithms at the intersection of graph and machine learning. Corey has a passion for using data to make better sense of the world.

Vivek Narang is a software engineer at SearchScale and is currently working on building search-related tools and products.