A Slovenian team set out to build open-source tools that help small retailers go digital easily, by importing product descriptions from a spreadsheet into an online shop – with AI-enhanced descriptions and images, in just a few clicks

For small to medium sized Brick-and-Mortar retailers, the move from physical shops to e-commerce is a long and cost intensive process. These businesses typically have an accounting system with a list of products, perhaps a supplier’s website with technical specifications, and neither the time nor the budget to manually write product descriptions, source images, and populate an online shop for hundreds or thousands of items. The result is that many small retailers either delay their digital transition or end up with online catalogues that are sparse, poorly described, and unappealing to customers.

The main challenge is not a lack of products but a lack of digital product content. A physical shop’s inventory usually exists as a list of names, SKU (stock keeping units) codes, and prices in accounting software. An online shop needs other specifications : well-crafted product descriptions, high-quality images, metadata, and engaging presentations. Creating this content  quickly becomes a substantial undertaking.

The DTCommerce project, carried out by the Slovenian company ZenLab under the European OpenWebSearch.eu project funding, set out to solve this problem with an automated solution. The idea is simple: take the product list a shop already has, find the corresponding product information on the web, enhance the descriptions using AI, and deliver the result as a ready-to-use online shop – with minimal manual effort.

The Approach: Automated Extraction and AI Enhancement

The DTCommerce system operates in two stages. The first stage is a web crawling process that, given a list of product URLs from supplier or manufacturer websites, automatically extracts the key product information, such as: title, description, imagery, price, and technical specifications. The crawler is built on Scrapy, a well-established open-source web scraping framework, and includes support for structured data formats (JSON-LD) as well as domain-specific extractors for particular target sites.

The second stage is where AI comes in. The raw product descriptions extracted from supplier websites are often technical, dry, and written for a trade audience rather than end consumers. DTCommerce feeds these descriptions to an AI language model (Perplexity AI’s sonar-pro), which rephrases them into clearer, more engaging copy while preserving every technical detail – dimensions, model numbers, and specifications. The original description is retained alongside the enhanced version, so nothing is lost. The result is a set of enriched product records in a standardised format, ready to be imported into an e-commerce system.

From Pipeline to Plugin: A Few Clicks to a Full Shop

To make the pipeline usable for non-technical shop owners, the ZenLab team built a WordPress/WooCommerce plugin that wraps the entire workflow into a simple administrative interface. The process works as follows: the shop owner exports a product list from their accounting software as an Excel file and uploads it to the plugin. The plugin creates basic product entries in WooCommerce, sends them to the enrichment service, and automatically populates each product page with enhanced descriptions and images – all without requiring the shop owner to edit a single product manually.

An Honest Detour: When the Open Web Index Didn’t Have What Was Needed

DTCommerce was originally designed to use the Open Web Index (OWI) as its primary data source for finding product information across the web. The vision was that a shop owner could provide a product name or SKU code, and the system would search the OWI to find matching products on supplier and manufacturer websites, automatically retrieving descriptions and images.

In practice, the specific e-commerce sites that the project’s use cases required were not present in the OWI at the time of development. This is not surprising: the OWI is still being built, and its coverage of niche commercial sites – particularly smaller B2B suppliers – is not yet comprehensive. The team adapted by switching to direct web scraping of predefined supplier URLs, which allowed the project to deliver its core functionality on schedule.

Why the project matters

DTCommerce addresses a real and widespread problem. Across Europe, millions of small retailers face pressure to establish an online presence but lack the resources to do so effectively. By automating the most labour-intensive part of the process – creating digital product content – the project lowers the barrier to entry in a meaningful way. The fact that the tools are open source and built on widely used platforms (WordPress, WooCommerce, Scrapy) means they are accessible to a broad audience and can be adapted to different markets and product domains.

The project also illustrates a type of application that open web search infrastructure is well suited to support. The ability to search an open web index for product information – matching a local shop’s inventory against the broader web – is precisely the kind of use case that depends on open, non-proprietary access to web data. As the OWI matures, tools like DTCommerce stand to benefit directly. The project overall also demonstrates both the potential of the OWI-based approach and its current practical limits.

Final Outlook

The CIFFIL Service project shows that open web index standards can help small municipalities improve their search quality by accessing results from larger ones

Search engines work best when they have a lot of data to learn from. The more documents in a collection, the better the system can distinguish between common words and genuinely informative ones – and therefore the better it can identify what is relevant to a query. This is a well-known principle in information retrieval, and it creates an obvious problem for anyone who needs to search a small collection of documents: the search results are simply not as good as they could be.

The CIFFIL Service project, funded under the European OpenWebSearch.EU project, tackled exactly this problem – in a setting with direct consequences for citizens. Spinque, a Dutch search technology company, builds search systems for municipalities that allow council members and residents to search through publicly available government documents. Some of these municipal collections are small, containing fewer than 10,000 documents, and are full of domain-specific jargon. The result is that search quality suffers. The CIFFIL project set out to fix this by allowing municipalities to share their search index data with one another through an open standard.

The problem: Small collections, unreliable statistics

Most search engines use some variant of a ranking algorithm called BM25. At its core, BM25 judges the relevance of a document to a query by looking at how often the query terms appear in the document and how rare those terms are across the collection as a whole. Terms that do not appear in a lot of documents signal relevance.

This is where small collections often fall short. When a collection has only a few thousand documents, the estimates of how common or rare a term is very unreliable. The ranking algorithm, relying on these skewed statistics, makes poor decisions about what is relevant. The result for the user is a search experience that feels hit-or-miss.

The solution: Sharing index data through an open solution

The CIFFIL team’s approach is simple. If a small municipality’s search system suffers from unreliable statistics because its collection is too small, why not supplement those statistics with data from a larger municipality that deals with similar types of documents? After all, Dutch municipal documents share a common vocabulary of administrative, legal, and policy language.

The technical mechanism for this sharing is the Common Index File Format, or CIFF – an open standard developed in the information retrieval research community for exchanging inverted index data between systems. An inverted index is the core data structure behind a search engine: it maps every term in a collection to the documents in which that term appears, along with statistics such as how often it appears and in how many documents.

Spinque integrated CIFF support into its search platform, Spinque Desk. This involved building a CIFF reader (to import index data), a CIFF writer (to export index data), and – critically – a modified BM25 ranking component that can combine the statistics from a local collection with those from an external CIFF index. When a small municipality’s search system uses this combined approach, it effectively “borrows” the larger municipality’s understanding of which terms are common and which are rare, while still searching its own documents.

Proof of concept

The team implemented tests for the functionalities implemented for this project. Specifically, they did manual testing by doing experiments using CIFF exports to see if they could replicate effectiveness results on open datasets. Additionally, they implemented unit tests to ensure the parser and writer were producing indexes according to the CIFF specifications.

The results were clear. The small collection performed substantially worse than the baseline, confirming that skewed statistics degrade search quality. But when the small collection borrowed statistics from the larger one, performance not only recovered but actually slightly exceeded the baseline – because the small collection, now ranked with accurate statistics, contained a higher concentration of relevant documents.

In practice

The project created CIFF indices for four major Dutch municipalities: Amsterdam, Utrecht, Nijmegen, and Almere. A live deployment was initiated for the municipality of Nieuwegein, a smaller city near Utrecht, using the Utrecht index as the background collection. Evaluation of the real-world impact on user experience is ongoing.

All of the CIFF tools developed during the project have been released as open-source software, and the export service ensures that published indices are automatically updated when the underlying data changes.

Why it matters

The CIFFIL project illustrates a principle that is central to the OpenWebSearch.eu core idea: that open, interoperable standards can enable forms of cooperation that proprietary systems cannot. By sharing index statistics through CIFF, municipalities can improve their search quality without sharing their actual documents, without depending on a single commercial provider, and without each needing to build a large collection of their own. It is a form of search infrastructure as a public good.

The approach is also notable for its simplicity. It does not require neural models, large language models, or expensive computational resources. It works by making better use of data that already exists, through a well-understood ranking algorithm and an open file format.

What’s next

The immediate priorities are completing the open publication of all four municipal indices, conducting user-experience evaluations in the live deployments, and publishing the experimental findings as a research paper. Longer-term, the approach could be extended to additional municipalities and to other domains where small document collections need better search – such as cultural heritage institutions, local archives, or specialised libraries. The underlying principle – that sharing standardised index data can improve search quality without centralising control – has broad applicability wherever open, cooperative search infrastructure is valued.

Find the full project report here: https://zenodo.org/records/17750643

The CIFFIL Service project was funded under the OpenWebSearch.EU project (Horizon Europe, Grant Agreement 101070014, Call #2).

The TILDE project builds a health search system that doesn’t just find answers – it checks them for bias, explains the underlying reasoning, and let’s users explore the evidence visually

Search for a health question online and you will get plenty of results. But how much can you trust what you find? Are the top results there because they are the most accurate, or because they happen to be the most popular? Are they showing you the full picture, or a skewed one – biased toward certain demographics, viewpoints, or types of sources?

The TILDE project – Trustworthy Access to Knowledge from the Indexed Web – funded under the European OpenWebSearch.eu project and carried out by Know Center Research GmbH in Austria, tackles this issue directly. It builds a health domain search system on top of the Open Web Index that goes beyond finding relevant documents to actively examining search results for bias, ensuring viewpoint diversity, and providing  visual tools to help users explore the evidence for themselves.

The Problem: Bias in health search

Health information is one of the most searched-for categories on the web, and also one of the most consequential. A search for COVID-19 treatment options, for example, should ideally return results that are medically accurate, drawn from credible sources, and representative of different perspectives – official health guidance, clinical research, patient experiences. In practice, standard search systems optimise for another kind of relevance, which is often approximated by popularity and click behaviour. This can systematically favour certain types of content while marginalising others.
The problem is compounded when large language models are involved. LLM-based systems, including RAG (retrieval-augmented generation) pipelines, inherit and can amplify biases present in both their training data and the documents they retrieve.
A search result list that is geographically skewed, lacks viewpoint diversity, or reinforces stereotypes about particular demographic groups is not just an academic concern – it can directly affect how people understand their health options.

The Approach: Three modules for trustworthy search

TILDE addresses this through three integrated modules, each tackling a different dimension of the problem.The first module extracts medical knowledge from the Open Web Index. Starting from approximately 200,000 health-related websites identified in the OWI, the team extracted medical entities – diseases, symptoms, drugs, procedures – using a named entity recognition model, then standardised these entities against the UMLS clinical ontology (a comprehensive medical terminology system). This creates a structured knowledge layer on top of the raw web content. The extracted entities and their relationships form a medical knowledge graph that links websites to each other and to clinical concepts. A hybrid search engine combines entity-based retrieval (finding pages that mention specific medical concepts) with semantic similarity search (finding pages whose content is meaningfully related to the query), fusing the results to balance precision and recall.

The second module checks search results for fairness and trustworthiness. This is TILDE’s most distinctive contribution. Built on DSPy, a Stanford framework for programmatic LLM pipelines, the trustworthiness module processes search results through three stages. First, each candidate document is enriched with fairness-related attributes: its viewpoint (official guidance, patient narrative, investigative journalism), its source credibility (from high-authority institutional sources down to user-generated content), whether its content is factual or anecdotal, and a gender neutrality score. Second, an intelligent re-ranker uses these attributes to reorder results according to a strict hierarchy: maximise fairness first, then filter for credibility, then ensure viewpoint diversity. The system uses chain-of-thought reasoning, meaning it explains its re-ranking decisions step by step. Third, a stereotype audit inspired by established bias benchmarks checks both the system’s internal reasoning and its user-facing output for harmful stereotypes – a safety net against the system itself introducing bias.

The third module provides visual aids to help users understand the evidence. Rather than presenting search results as a flat list of links, the visual web interface allows users to explore medical information through multiple lenses: highlighted medical concepts within document text, faceted search by entity type, tag clouds and bar charts showing the frequency of different symptoms or drugs across results, co-occurrence matrices revealing relationships between medical concepts, and an interactive knowledge graph that can be expanded and filtered.

Why It Matters: Making fairness operational

There is no shortage of academic research on bias in search systems. What is less common is work that takes established fairness metrics – like the NFaiRR measure of retrieval fairness – and turns them into actionable components within a working search pipeline. TILDE does exactly this. The re-ranking module does not merely measure bias after the fact; it actively uses fairness criteria in real time to reorder results, while maintaining credibility and diversity as additional constraints. The chain-of-thought reasoning makes the process transparent: users and auditors can see why results were ranked the way they were.

The health domain is the testbed, but the approach is not limited to it. The same pipeline – entity extraction, hybrid retrieval, fairness-aware re-ranking with transparent reasoning, and visual analytics – could be applied to any domain where search results carry real-world consequences: legal information, financial advice, educational content, public policy. The fact that it is built on the Open Web Index, rather than on a proprietary search engine, means the underlying data is open and the approach is reproducible.

What’s Next

The immediate next step is completing the integration of the hybrid search across all components of the visual interface. Longer-term priorities include optimising the trustworthiness pipeline for real-time performance, extending the approach to additional health sub-domains, and conducting user studies to understand how fairness-aware re-ranking and visual analytics actually affect the way people seek and evaluate health information.

To read the full technical report, go here: https://zenodo.org/records/17542369

The TILDE project was funded under the OpenWebSearch.eu initiative (Horizon Europe, Grant Agreement 101070014, Call #2).

How a European research team built a browser-based fact-checking assistant powered by the #OpenWebIndex

In an information environment where misleading claims can spread in real time, the ability to verify what you read online is a necessity. The VERITAS project, funded under the European OpenWebSearch.EU project and conducted by DEXAI, set out to build a practical tool for exactly this purpose: an AI-powered assistant that sits in your browser, answers your questions with sourced evidence, and draws its knowledge not from a proprietary index controlled by a single corporation, but from an open, European web search infrastructure.

The Problem: Misinformation and the Limits of Conventional Search

The War in Ukraine has been accompanied by an unprecedented volume of online misinformation – from fabricated reports and manipulated imagery to subtly misleading narratives. For journalists trying to verify claims, researchers analysing media coverage, and ordinary citizens attempting to understand what is actually happening, conventional search engines offer limited help. They return ranked lists of links, but they do not assess the credibility of sources, provide citations for specific claims, or explain the basis for their answers. The burden of verification falls entirely on the user.

At the same time, the emergence of AI chatbots has introduced a new set of problems. Large language models can produce fluent, confident-sounding answers that are entirely fabricated – a phenomenon known as hallucination. Without mechanisms to ground their outputs in verifiable evidence, these systems risk becoming part of the misinformation problem rather than the solution.

The Approach: Retrieval-Augmented Generation via the Open Web Index

The DEXAI/VERITAS team adopted an approach known as RAG (Retrieval-Augmented Generation). The core idea: instead of prompting an AI model to generate answers from whatever it absorbed during training, you first retrieve relevant documents from a trusted knowledge base and then ask the model to compose its answer based specifically on those documents. Every claim in the response can thus be traced back to an identifiable source.

Rather than relying on a commercial search engine or a static dataset, the system draws its evidence from the Open Web Index (OWI).

In practice, the system works as follows. The VERITAS pipeline fetches the latest crawled web pages from the OWI (pulling the latest 30 days of crawled content). The dataset is then indexed using a semantic embedding model, which converts text passages into numerical vectors that capture their meaning. When a user poses a question, the system converts it into a similar vector, finds the most relevant passages in the index, and passes them – together with the question – to a language model (LLaMA 3.1), which generates a grounded response.

The chatbot supports multiple audiences, offering background information with explicit citations for journalists, metadata-rich summaries for researchers, and concise, jargon-free explanations for the general public.

What They Built: A Fact-Checking Assistant in Your Browser

The finished product is a Chrome browser extension. Once installed, it provides a small popup where users can type questions in natural language and receive answers accompanied by source references.

The system is designed to serve different types of users in different ways. Journalists receive background information with explicit citations. Researchers get metadata-rich summaries. Members of the general public are given concise, jargon-free explanations. In the current prototype, the system is focused specifically on the War in Ukraine – a deliberate scoping decision that allowed the team to develop and validate the approach within a well-defined domain.

Why It Matters: Open Infrastructure for Trustworthy Information

VERITAS is significant not only for what it does, but for how it does it. By building on the Open Web Index rather than a proprietary data source, the project demonstrates that open search infrastructure can serve as the foundation for practical applications.

The RAG approach itself addresses one of the most persistent criticisms of AI-generated text: the lack of verifiability. By requiring the model to base its answers on retrieved documents and by presenting those documents to the user, VERITAS moves away from the “trust me” paradigm of conventional chatbots towards a “check for yourself” model of AI-assisted information access.

What’s next?

Future development could also introduce user feedback mechanisms, allowing the quality of responses to be improved over time, as well as streaming responses for a more interactive user experience. Perhaps most importantly, the VERITAS approach could be applied to other domains where information verification is critical – from public health to climate science to electoral integrity.
An outstanding challenge overall lies in the growing complexity of multi-model ecosystems. As retrieval systems, ranking components, embedding models, and large language models are increasingly combined—often across organisational and infrastructural boundaries—the integrity of the final answer depends on the entire chain. Outputs generated by one model may be ingested, summarised, or re-ranked by another, creating feedback loops that are difficult to detect and audit. In such environments, disinformation cascades can emerge when misleading or low-quality content propagates across interconnected systems, gaining credibility through repetition and algorithmic reinforcement. Ensuring traceability, cross-model accountability, and robust provenance mechanisms will be essential to prevent systemic amplification of false or manipulated claims.

To read the full technical report, go here: https://zenodo.org/records/17588890

The VERITAS project was funded under the OpenWebSearch.EU project (Horizon Europe, Grant Agreement 101070014, Call #2).