Mapping AI Techniques to the Five Technical Requirements
With a clear set of technical requirements in hand, drawn directly from real examiner interviews, the research team turned to the actual business of choosing which AI technologies might realistically meet those needs. This feasibility study cast a genuinely wide net, investigating a combination of three broad technology families: natural language processing (NLP), machine learning (ML), and semantic technologies. Rather than picking one approach and running with it, the researchers systematically considered how well different algorithms might address each of the five technical requirements defined earlier, namely query expansion, document classification, document similarity, ranking, and topic modelling.
To make this comparison concrete, the researchers built a table mapping different categories of AI and NLP algorithms against the five technical requirements they were meant to support. Natural language processing techniques, including text segmentation, normalisation, lemmatisation, stemming, detecting co-occurrences between words, and extracting multi-word terms, turned out to be relevant across the board, touching all five requirements in some way. This makes sense, since these are foundational techniques that essentially prepare raw text so that more sophisticated algorithms can work with it effectively.
Unsupervised machine learning techniques centred on word embeddings and distributional semantics also proved broadly useful, supporting query expansion, document classification, document similarity, and topic modelling, everything in fact except ranking. Supervised machine learning techniques, by contrast, support vector machines, naive Bayesian learning, decision tree induction, and random forests, were found to be relevant specifically to document classification, and less useful for the other four requirements. A second category of unsupervised learning, covering neural networks and deep learning more broadly, was likewise mapped specifically to document classification. Meanwhile, dedicated similarity measures, including Jaccard similarity, Euclidean distance, and cosine similarity, were mapped specifically to the task of document similarity, which is essentially their core purpose. Finally, semantic technologies, including the use of lexico-semantic knowledge resources and latent Dirichlet allocation, turned out to be broadly useful once again, supporting query expansion, document similarity, ranking, and topic modelling.
For anyone trying to picture the landscape at a glance, the mapping breaks down roughly like this:
- NLP techniques (segmentation, normalisation, stemming, co-occurrence detection): relevant to all five requirements
- Unsupervised ML (word embeddings, distributional semantics): relevant to query expansion, classification, similarity, and topic modelling
- Supervised ML (SVMs, naive Bayes, decision trees, random forests): relevant mainly to classification
- Neural networks and deep learning: relevant mainly to classification
- Similarity measures (Jaccard, Euclidean, cosine): relevant to document similarity
- Semantic technologies (lexico-semantic resources, latent Dirichlet allocation): relevant to query expansion, similarity, ranking, and topic modelling
What the Technology Mapping Revealed
This mapping exercise gave the researchers a clear eyed view of the AI landscape before they built anything. No single family of techniques covered every requirement, but different combinations of techniques, layered together, could plausibly cover the full range of what examiners actually needed. That finding matters for anyone evaluating AI patent search tools today, since it explains why a genuinely useful system tends to combine several AI approaches rather than relying on one algorithm to do everything.
Designing the Concept Model for Prior Art Search
With this landscape mapped out, the next step was to design an overarching concept model, essentially a blueprint describing how a prior art search and the filtering of patent information should actually flow from start to finish, and where different AI techniques would slot in along the way. This concept model was designed specifically as a methodological tool, something the researchers could use to systematically experiment with different algorithms in a structured, comparable way, rather than testing them in isolation with no shared framework.
The proposed model rests on a specific sequence of assumptions about how the whole process should unfold. It begins with the examiner reading through the application and defining both a search statement and a corresponding search query, echoing exactly the process observed during the examiner interviews described earlier. From there, the system takes over the first automated step, classifying the application into one or more relevant classes. Next, the system extracts the most relevant keywords from the application, including multi-word technical terms that a simple keyword search might otherwise miss. Building on these extracted keywords, the system then suggests expanding the query with other related words, synonyms, broader terms, narrower terms, and so on. At this point, control returns to the human: the examiner curates the search query, reviewing and refining the system's suggestions based on their own expert judgement.
From Query Curation to Colour-Coded Results
Once the query has been finalised by the examiner, the system launches an actual search to retrieve documents from the relevant classes identified earlier. Rather than simply dumping a long, undifferentiated list of results on the examiner, the system then sorts the retrieved documents into distinct topics, with each topic described by its own set of representative keywords. This is where the examiner steps back in for a second time, selecting whichever topic or topics seem most relevant to the application at hand. From there, documents within the selected topic or topics are ranked according to how similar they are to the original application. And as a final touch, the content of each document is colour-coded to visually highlight just how relevant it is to the application, making it easier for the examiner to scan through results quickly and intuitively.
This entire workflow is captured in what the researchers refer to as Figure 1, a conceptual diagram illustrating the concept model of a prior art search and the filtering of patent information. What's worth emphasising here is how closely this model mirrors the human in the loop philosophy discussed earlier, since at no point does the AI simply take over the whole process. Instead, it alternates cleanly between automated steps (classification, keyword extraction, query expansion suggestions, retrieval, topic sorting, ranking, and colour-coding) and moments where the human examiner exercises judgement (curating the query and selecting relevant topics). This deliberate back and forth is precisely what allows the system to support the examiner's expertise rather than attempting to replace it. This is the same balance WOIPS aims for in practice, pairing automated retrieval and ranking with a clear, reviewable output, and you can see it for yourself by trying a free novelty search on your own invention.
Testing the Model: Building and Validating the System
To actually test this concept model in practice, rather than leaving it as a purely theoretical framework, the research team implemented a working system based on this design using the Python programming language. The specific external software libraries this system depended on are documented separately in an appendix to the full report, for anyone wanting to dig into the technical implementation details.
For the purposes of validating this system experimentally, the researchers didn't try to cover every possible technology area at once. Instead, they deliberately chose three specific domains to focus on throughout development: civil engineering, computer technology, and transport. These weren't arbitrary picks. The researchers chose them precisely because they represent the top three technology fields, ranked by number of patent filings at the IPO over the past ten years, based on the standard World Intellectual Property Organisation (WIPO) technology field classification system, which covers 35 distinct fields in total. Focusing on the busiest, most heavily filed domains meant the validation work would be grounded in realistic, high volume conditions rather than a narrow or unusual niche.
Each of these three domains was then formally and precisely defined, not just informally, but as the union of specific invention areas identified by their corresponding codes within the International Patent Classification (IPC) scheme, the internationally recognised system, maintained by the World Intellectual Property Organisation, for categorising patents by technical subject matter. The specific IPC codes chosen to represent each of the three domains are listed in a dedicated appendix within the full report.
Once the three domains were formally defined, the researchers built corresponding validation datasets by retrieving actual patents falling under these IPC classes and subclasses, drawing on data sources identified by the IPO itself. The original data arrived formatted in XML, following a specific schema also documented in an appendix to the report. To make this data easy to work with and query by metadata, things like publication date, applicant, or classification code, the researchers stored it all in a dedicated XML database, giving the whole experimental pipeline a solid, well organised technical foundation to build on for the rest of the study. The structured, classification-driven approach behind this pipeline is also what makes systematic patent information search possible at scale, well beyond a single manual query.
Frequently Asked Questions
What AI techniques were considered for automated patent prior art search?
The study evaluated natural language processing, machine learning (both supervised and unsupervised), and semantic technologies, mapping each category against five technical requirements: query expansion, classification, document similarity, ranking, and topic modelling.
What is the concept model in AI-assisted patent search?
The concept model is a blueprint describing how an AI-assisted prior art search should flow from start to finish, alternating between automated steps like classification and ranking, and human steps where the examiner curates the query and selects relevant topics.
Which AI technique is best for document classification in patent search?
The study found several options relevant to classification, including supervised machine learning methods such as support vector machines and random forests, as well as neural networks and unsupervised word-embedding techniques.
Why did the researchers test the model on civil engineering, computer technology, and transport?
These three fields were the most heavily filed technology areas at the IPO over the previous decade, based on WIPO's technology field classification, making them realistic, high-volume domains for validating the system.
Does AI replace the patent examiner in this model?
No. The concept model keeps the examiner in the loop at key decision points, curating the search query and selecting relevant topics, while the AI handles classification, keyword extraction, retrieval, ranking, and visual highlighting.
This article is adapted from: UK Intellectual Property Office, AI-assisted patent prior art searching – feasibility study, April 2020, ISBN 978-1-910790-80-9, © Crown Copyright 2020, licensed under the Open Government Licence v3.0.