Radu Gheorghe
Radu has been in the search space for many years, mainly on Elasticsearch, Solr, OpenSearch, and, more recently, Vespa.ai. Helps users with both the relevance and the operations side of retrieval. Enjoys education in all its forms (training, blog posts, books, conferences...) and got the chance to be involved in all of them.
Session
06-16
10:40
20min
Which GPU for Local LLMs?
Radu Gheorghe, Rafał Kuć
You’re using local LLMs. For example, to power RAG. You want to deploy them in production, but you don’t know where: which type of GPU? How large should it be? Should you use a larger model but quantize more aggressively?
Our benchmark results and their interpretation will give you some answers.
Kesselhaus