From EU project to EU search engine | NKS DIT "Success Stories“ Yearbook - Open Web Search – Promoting Europe‘s Independence in Web Search

The National Contact Point Digital and Industrial Technologies (NKS DIT) of the German Federal Ministry of Education and Research introduces the OpenWebSearch.EU project in their yearbook, which features “success stories“ from the Horizon Europe EU program.

With the purpose of presenting stories of particularly successful EU projects to the German Federal Ministry of Education and Research (BMBF) and the German public, this year’s volume features 12 flagship projects that have reached important milestones.
OpenWebSearch.eu is portrayed as an important undertaking to foster European digital sovereignty in alignment with ethical requirements, as the following statement from the entry underlines: “The collaborative, decentralized and interdisciplinary European approach makes a major contribution to a fair, open, diverse and free web.“
“This is a great recognition of our work to date,” says a delighted Prof Dr Michael Granitzer, holder of the Chair of Data Science and scientific coordinator of the EU project. “I also see it as a signal of how important open and transparent digital technologies are for the future of Europe. With our web index, we are creating a foundation on which new, promising AI applications can be built.”

2.7 billion URLs indexed in 185 languages

Over the past two years, the interdisciplinary research team coordinated by Professor Granitzer has developed an Open Web Index prototype. It is currently hosted via an infrastructure of four European supercomputing centers and is modelled with an open future so that new promising technologies such as AI applications can be taken into account. Moreover, different types of search engines could be set up, including vertical search engines that focus on special interest topics or argument search engines, which can help reduce confirmation bias in search results. The index prototype has so far “crawled” 2.7 billion URLs in 185 languages and comprises 400 tebibits, which is roughly equivalent to 191,000 pictures.

AI-driven web search in the focus of the coming project year

The project continues until end of August 2025. Next year, the team wants to develop ethical guidelines for the curation and establishment of the hosting infrastructure and explore options for its long-term funding. The focus will in addition be on the preparation of data for the use of AI-driven web searches. The research team is also aiming to further scale the index with the aim of covering a relevant part of the web and making the data usable in application scenarios for web search, AI and web data analysis.
“Our mission is to contribute to a fair, open, diverse and free web. We make the European economy and society less dependent on global digital players by enabling transparent and open access to web data – for independent search engines as well as for the analysis of web data and the use of AI,” explains Professor Granitzer.

Find the full project portrait via NKS DIT here:

https://www.nks-dit.de/aktuelles/news/erfolgsbroschuere_2024_nksdit_digitale_schluesseltechnologien