Welcome to the Open Web Search Project!
We’re excited to have you here. Most likely our web crawler has recently visited your site as part of our research project. We crawl the Web to create an open web index and to bootstrap a more open internet search ecosystem, which also includes that webmasters and content producers have more control over how and for what their precious content is used. We’ve outlined more details of what we are doing and why we are crawling the Web (and thus your site) below, including technical information on how to exclude your site from being crawled by us. It would be great if you would allow us to crawl your site, but it is your choice!
Our goal is to build a comprehensive open web index to benefit all its users. A web index forms the core for every search engine and can be seen as a map of the internet, which relates words and search terms to websites where they occur in. A web index allows users to search and rank websites according to their defined queries. Our index will be openly available to everyone, so that everyone can build their own web search engine.
We hope that by enabling more web search engines, that users get the choice of where to search. Comparable to how you choose your newspaper, you should be able to choose your search engine.