Masterarbeit

Evaluation of Artificial Intelligence tools for supporting Audits and Inspections

Laura Cordoba Duran (2026)

Language: English

This master thesis investigated the potential of AI tools to support audits and inspections in GMP environments. The research focuses on the use of AI from the perspective of end-users responsible for GMP compliance, without IT knowledge and rely on standard AI solutions to support their daily activities.

After weighing the advantages and disadvantages of the AI tools developed at Boehringer Ingelheim, the research focused on the licensed version of Microsoft Copilot, which enables the creation of domain-specific agents that securely integrate internal (the validated VQD) and external sources (websites of relevant HAs) and that can be tailored by providing specific instructions. The custom Microsoft Copilot agent “Inspection Response Assistant” was designed to support SMEs and A&IMs in analyzing audit observations, identifying related GMP risks, referencing applicable regulatory guidelines, and providing recommendations how to address observations and proposing CAPAs. 

Initial results demonstrated significant limitations in Copilot’s ability to access and correctly cite regulatory information contained in downloadable PDFs from official websites, often resulting in hallucinated or incorrect citations. Consultation with IT experts, review of specialized papers and analysis of Copilot system behavior revealed technical constraints in search engine indexing, PDF parsing, and website access restrictions. These findings underscore that accurate regulatory citations require either direct file upload or provision of structured knowledge sources. The agent's functionality was enhanced by constructing a curated internal GMP Library containing official regulatory documents.

Following the implementation of the curated repository, several well-documented audit observations were examined. The quality of the agent’s responses demonstrated a marked improvement, which was confirmed by the assessment of SMEs based on criteria such as credibility, CAPA relevance, SOP reference accuracy, and consistency with real audit responses. Overall, the agent achieved high acceptability, with performance significantly strengthened when supported by curated regulatory sources.

At the end, the thesis contextualized the use of AI tools in GxP environments within the evolving regulatory landscape: the EU AI Act, the upcoming EU GMP Annex 22 for AI systems and some interfaces with other regulations. 

A key aspect is the alignment between the EU AI Act and the revised EU GMP Annex 11. Although developed independently, both frameworks share closely related concepts such as lifecycle management of computerized systems, documentation and traceability requirements, data governance, oversight responsibilities, and the expectation of controlled, explainable system behavior.  The thesis also critically reviews the upcoming EU GMP Annex 22, which is the first annex focused on AI and ML in GMP. Although Annex 22 provides needed validation structure, it takes a cautious approach: it excludes dynamic learning systems from critical use, lacks references to the AI Act, and leaves lifecycle challenges unresolved. These gaps reflect the current state of regulation, indicating that more harmonization and refinement will be needed as AI advances.

On the other hand, leveraging the insights of the recent multistakeholder Workshop on AI organized by the EMA and the HMA, it was highlighted what GMP inspectors are likely to focus on during AI related GMP inspections. Important for the pharmaceutical industry is to identify which AI Systems fall under the scope of the upcoming Annex 22, considering impact on patient or product, type of used model, and grade of human oversight. Under this premise, the Inspection Response Assistant is not considered a high risk or GMP critical AI system, as it is a generative LLM based tool used only for supportive, non critical activities.

Overall, this thesis shows that while AI already offers meaningful support for GMP related analyses, its full potential will only be realized when robust regulatory foundations and technically reliable system architectures are in place. To overcome the challenges highlighted in this research, the proposed agent could be further enhanced by utilizing the advanced AI platforms offered at Boehringer Ingelheim.

Pages: 72

Annexes: 7, Pages: 27