2025-06-16 –, Frannz Salon
Lucene 10 came out last year in October. One of the changes was about the minimum version requirement of Java 21 – this looks like it allows to introduce new features like native access to file system cache and therefore better memory mapping. Is it as easy as it sounds?
This talk will discuss the new features of Java 21 and how they can be used in Lucene 10. But it will also show the challenges that come from the fact that the many of the Lucene-relevant APIs are preview-only in that version.
Uwe will introduce the mechanisms used to provide access to the native layer of Java 21 but also the limitations. New features implemented using native APIs are madvise/fadvise kernel hints for the file system, but also preloading of pages required during search.
The limitations of the current system make it impossible to integrate modern Java APIs into the public API of Lucene, so Lucene 10 still has the same limitations like previous versions with regards to Java preview features. Discussions are ongoing to change the main branch to align to the "tip & tail" model of development done in the OpenJDK community and start to integrate types like MemorySegment as first class citizen to Lucene's public API in the main (development) branch and release new versions with new minimum requirements more often. One example for this is a PR to offload calculations to a graphics card which was submitted to the Lucene project at the time of writing this proposal.
Search, Scale
Level:Advanced
Uwe is committer and PMC member of Apache Lucene and Apache Solr. His main focus is on development of Lucene Core. He implemented fast numerical search and is maintaining the new attribute-based text analysis API. He studied Physics at the University of Erlangen-Nuremberg and works as managing director for SD DataSolutions GmbH in Bremen, Germany, a company that provides consulting and support for Apache Lucene, Elasticsearch, and Apache Solr. He also works for “PANGAEA – Publishing Network for Geoscientific & Environmental Data” where he implemented the portal's geo-spatial retrieval functions with Lucene Java. Uwe had talks about Lucene at various international conferences like the previous Berlin Buzzwords, ApacheCon EU/US, Lucene Revolution, Lucene Eurocon, and various local meetups.