The Policy Context Behind the Study: AI as a Grand Challenge
This AI patent prior art search study was commissioned because the UK Government identified artificial intelligence as one of its Industrial Strategy's "Grand Challenges," and wanted to test whether it could genuinely improve how patent examiners search for prior art. The study sits within the UK Government's Better Regulation agenda, which has actively encouraged regulators like the Intellectual Property Office (IPO) to explore innovation-friendly frameworks. It was funded through the £10 million Regulators' Pioneer Fund (RPF), launched in 2018 specifically to drive this innovation agenda forward and help unlock long-term economic opportunities identified in the Government's modern Industrial Strategy. Regulators across different sectors were asked to look at new and emerging issues where productive collaboration with others might help businesses bring innovative products to market faster.
The IPO had already established an internal working group specifically tasked with investigating how the organisation could embrace AI technologies, both to modernise its own internal operations and to improve the application process for customers seeking intellectual property rights, a category that includes patents, trade marks, and designs. Patents present a particularly interesting challenge here: the highly technical nature of patent specifications, combined with the dense legal wording used throughout patent applications, raises a distinctive set of difficulties for any AI system trying to work with this kind of text.
Why the IPO Commissioned This Research
It was within this context that the IPO commissioned Cardiff University to investigate the feasibility, technical complexity, and effectiveness of AI solutions that might benefit IPO customers during the filing and prosecution of patent applications, with funding coming directly from the £10m Regulators' Pioneer Fund. More specifically, the IPO's interest centred on building a proof-of-concept for an AI-powered prior art search or due-diligence check that could eventually become part of both the online patent filing process and the examiner's own prior art searching workflow. The thinking here is genuinely practical: if a system could flag the most relevant prior art to an applicant before they even file, it could warn them early about material that might hinder their application from being granted. And crucially, the results of that same search could then be handed on to the examiner actually prosecuting the application, potentially saving them significant time, and by extension, cost, in their own search process.
The underlying aim of the whole project, then, was to reduce the time and cost involved in patent prior art searches and due-diligence checks, and in doing so, improve the overall quality of the patent examination process. If this worked, customers applying for intellectual property rights could benefit from noticeably faster handling of their applications. This vision lines up with a broader ambition, echoed in earlier work by Andlauer (2018), of building automated search tools that complement, rather than compete with, the deep knowledge and expertise that patent examiners already bring to the table. The research was also intended to shed light on some genuinely open technological questions facing the wider intellectual property community, particularly around whether AI is even well-suited to patent searching given how technical patent specifications are and how specialised their terminology can be. As Krishna et al. (2016) had already pointed out, fully automated prior art retrieval systems run into real trouble because of both the highly technical content of patents themselves and the subtleties involved in interpreting patent law, subtleties that are themselves shaped by evolving court decisions.
The Study's Three Objectives
Given all this, the study's objectives were kept deliberately focused. There were three: first, to evaluate how viable various AI technologies actually are for patent prior art searching; second, to test a range of different approaches in order to pin down which algorithms perform best; and third, to fully test and evaluate whatever solution emerged as the strongest candidate. These three objectives gave the research team a clear, structured path from broad exploration through to a rigorously tested outcome.
What Earlier Research Says About Selecting Search Queries
Before diving into their own experiments, the researchers grounded their work in the existing literature on patent search, and what they found paints a useful picture of just how hard this problem really is. A successful prior art search, prior researchers have shown, depends heavily on selecting the right search queries (Bashir and Rauber, 2010), and a critical part of getting this right is the process of translating a human query, essentially, what the examiner actually wants to find, into a formal query representation the search system can act on (Crestani, 2003). This translation process is shaped by all sorts of human factors: the examiner's own background and experience in a given technical field, their communication and presentation skills, the trust and credibility they've built up over time, and even how they approach working with colleagues (Adams, 2018).
In practice, the terms used for prior art queries are typically pulled from the "claims" section of the patent being examined. But choosing the right terms turns out to be genuinely difficult, precisely because patents have such complex technical structures and often contain terms that are mismatched or frustratingly vague, meaning examiners frequently need to do additional research into the relevant technical domain just to select sensible search terms in the first place.
Why Patent Text Is Uniquely Difficult for AI
Part of what makes this so hard is that patents, as a category of text, are unusually difficult documents, arguably even less accessible than typical scientific literature. Researchers such as Anderson et al. (2017) have documented several defining characteristics of patents as a text genre: huge variation in document length, a strictly formalised structure at both the semantic and syntactic level, and heavy use of both standard and non-standard acronyms alongside dense domain-specific terminology. On top of this, patent drafters often deliberately choose unusual word combinations, not just synonyms, but genuine paraphrasing, a practice documented by Atkinson (2008). Applicants frequently invent their own vocabulary to describe the specifics of their invention, or deliberately use abstract and generic language in order to maximise the scope of legal protection they're seeking. To make matters more complex still, patents often contain entirely different types of content beyond plain text, drawings, mathematical formulas, biological sequence listings, or chemical structures, each of which demands its own specialised search and analysis techniques.
Beyond the core text of a patent, there's also the question of metadata. Patent offices typically assign standard metadata such as title, abstract, publication date, applicants, and inventors, and on top of this, they assign classification codes intended to help manage examination workload and support searching. The trouble is that these classification codes aren't consistently applied across different patent offices worldwide (Alberts et al., 2017), which introduces yet another layer of inconsistency for any search system to contend with. There's also a "diachronic" dimension to patent language, meaning that terminology genuinely shifts and evolves over time. A given term might refer to one technical concept during a particular period, and later come to mean something else entirely (Anderson et al., 2017; Harris et al., 2017). Combined with the sheer diversity of vocabulary across the patent corpus, this makes it considerably harder to sample representative training data for any text-mining application built to work with patents (Oostdijk et al., 2017).
Full-Text Search vs Structured Indexing: A Long-Standing Debate
There's also long-running debate within the patent profession about the relative merits of full-text search versus controlled, structured indexing (Adams, 2018). This debate really comes down to a fundamental trade-off in search quality: does full-text search generate too much irrelevant material, dragging down precision, or is it actually more prone to missing genuinely relevant documents because of unexpected variation in how source documents use terminology, dragging down recall? Currently, most patent examiners lean heavily on keyword searches combined with Boolean logic when hunting for prior art. Yet the broader research consensus, in both the information retrieval field and the patent domain specifically, is that keyword-based searching, even when carried out with the utmost professional care, often produces results that fall short of optimal (Helmers et al., 2019). This matters more in the patent world than it might elsewhere, because the consequences of getting it wrong aren't symmetrical. A false positive simply creates extra work, since the examiner has to manually exclude irrelevant documents from their report. But a false negative, missing a genuinely relevant piece of prior art, can lead to a patent being granted in error, with potentially serious legal and financial consequences down the line (Trippe et al., 2017).
Existing User-Centred Search Tools: Lessons from Sigma and Others
Encouragingly, several recent studies have pushed for the development of user-centred information retrieval systems, tools specifically designed to help expert patent examiners identify relevant literature and make sound decisions about prior art, rather than trying to replace their judgement outright. Such systems tend to emphasise interactivity and transparency, both of which turn out to be critical for earning users' trust. One real-world example is Sigma, a system currently being piloted at the United States Patent and Trademark Office (Krishna et al., 2016). Sigma goes beyond simple keyword search, allowing examiners to build custom search strategies tailored to the specific application in front of them. Another relevant study, by Showkatramani et al. (2018), explored the use of word embeddings for this purpose and reached a fairly humbling conclusion: no model on its own was sophisticated enough to match the quality of keyword expansion that a skilled human expert could produce.
Cardiff University's Design Philosophy and the Study's Scope
All of this research and background thinking fed directly into the study's own design philosophy. Cardiff University's approach followed what's known as user-centred design, meaning the needs, preferences, and genuine limitations of the people who would actually use the system, both patent applicants and examiners, were kept front and centre throughout the entire process. This translates into a "human-in-the-loop" model: the goal is to maximise overall performance by combining artificial intelligence with human judgement, with AI explicitly designed to supplement expert knowledge rather than replace it. Under this approach, AI algorithms help users navigate huge volumes of patent data by suggesting the most plausible search terms and by sorting patents into topics that are easy for a human to interpret at a glance. The human user always remains the key decision-maker; the AI's job is simply to provide intelligent decision support along the way.
Given all this, the scope of the feasibility study was defined quite specifically. It covers a wide range of state-of-the-art supervised and unsupervised machine learning approaches, intended to support five core tasks: feature extraction, query expansion, document classification, document clustering, and topic modelling. And to be absolutely clear about ambition versus reality: the intention was never to build a fully functional, production-ready information retrieval system. Rather, the goal was to develop a genuine proof-of-concept, one robust enough to allow meaningful experimental comparisons between different AI approaches, laying the groundwork for whatever comes next.
Frequently Asked Questions
Why did the UK IPO study AI for patent prior art searching?
The study was commissioned as part of the UK Government's Better Regulation agenda and funded by the £10 million Regulators' Pioneer Fund, which was set up to help regulators explore innovation-friendly approaches. The IPO wanted to understand whether AI could improve prior art search as part of both the online filing process and the examiner's own workflow. You can read more on our prior art search hub.
Why is patent text so difficult for AI systems to process?
Patents combine highly technical language with dense legal wording, inconsistent terminology, deliberately vague or invented vocabulary, and non-text content like drawings, formulas, and chemical structures. Classification codes also vary between patent offices, and terminology itself shifts over time, all of which make patents unusually hard documents for any AI system to work with.
What's the risk of relying only on keyword-based patent search?
Keyword and Boolean search remain the standard approach for most examiners, but research shows this often falls short of optimal results. The risk isn't symmetrical either: a false positive just means extra manual filtering, but a false negative, missing real prior art, can lead to a patent being granted in error, with real legal and financial consequences. Want to see how an AI-driven approach compares? You can try WOIPS's free novelty search on your own invention.
Are there other AI-powered patent search tools like this one?
Yes. One example is Sigma, a system piloted at the US Patent and Trademark Office that lets examiners build custom search strategies rather than relying on simple keyword search. Other research has tested word embeddings for query expansion, though results so far haven't matched the quality of expansion a skilled human expert can produce.
What is the human-in-the-loop approach used in this research?
It's a design philosophy where AI supports, rather than replaces, the examiner. The system suggests search terms and organises patents into interpretable topics, but the examiner always remains the decision-maker, using AI as an intelligent support tool rather than an autonomous one.
This article is adapted from: UK Intellectual Property Office, AI-assisted patent prior art searching – feasibility study, April 2020, ISBN 978-1-910790-80-9, © Crown Copyright 2020, licensed under the Open Government Licence v3.0.