Project NordLink – A Northern Europe Federated Open Web Search Data Center
The NordLink project aims to establish an independent and transparent internet search index, comparable to Google, for services such as LLM training in AI applications.
The Primary objective of the project NordLink will be to connect our storage and HPC resources with the OWS infrastructure and participate in web crawling and indexing activities.
The aim is to demonstrate that we are technically able to participate in the crawling and further processing of the index data as a local data center. The challenge lies in the current primary phase which is not so much the provision of technical resources, but rather the technical connection of storage space and computing resources (HPC) to the distributed OWS infrastructure via specialized interfaces.
Objectives in the primary phase:
- Integration of our data center’s storage and HPC resources with the OWS infrastructure.
- Ensuring seamless connectivity and data flow between local systems and the OWS network.
- Implementing and maintaining interfaces for data exchange and processing.
- Conducting web crawling and indexing in the next phase to support the OWS search index.
Why is it relevant for OpenWebSearch.eu?
By participating in OpenWebSearch.eu, we aim to enhance our capabilities and contribute to the development of a significant and impactful web search infrastructure. We are committed to ensuring the success of the OWS project through dedicated efforts and collaboration with our partners.
This Third-party call will provide local connectivity of our storage and HPC resources to the OWS infrastructure. We are also committed under this Third-party call to conduct web crawling and indexing to support the OWS Search Index.
This will also allow us to have a local availability of the index as a basis for ML applications especially with regard to solving practical problems.