Masterarbeit

Regulatory Evaluation of AI-Generated Dossier Parts with the Focus on Module 2.3 ***

Jan-Oliver Kahl (2025)

Summary

Language: English

The automated generation of Module 2.3 (Quality Overall Summary) across five marketing authorisation dossiers currently approved in the EU, covering a wide range a medicinal product types, was evaluated using a high-performing, easily accessible LLM. Following a stepwise approach (from extracting individual types of information to producing the complete summary in one session), both the strengths and limitations were identified: While plain text and tables could be reproduced accurately with minimal hallucinations, complex graphical elements were frequently misrepresented or ignored. Shorter subsections generally met regulatory requirements and closely resembled those written by experts. However, longer and more complex sections were often over-summarised and lacked key information. Attempts to mitigate these issues through providing the LLM smaller fragments of Module 3 did not improve the results. As a result, current available LLMs are not suitable for generating regulatory-compliant documents automatically. Improving output quality will require structured data inputs, domain-specific fine-tuning, and a robust human-in-the-loop review process.

Pages: 62

Annexes: 5 Pages: 8