Karolinska researchers set real-world testing as next step for medical AI update framework

Karolinska Institutet researchers say their next step is to test AEGIS, a framework for overseeing medical AI updates, in real-world use with clinical partners and authorities. In an October 23 update, the team outlined that direction after proof-of-concept demonstrations that checked its decision rules—not whether the medical models were safe for patients.

The framework, published October 13 in npj Digital Medicine, addresses a problem that begins after approval. Changes in patients, equipment and clinical workflows can gradually erode a model’s performance. Most approved medical AI devices do not learn continuously; improvements require retraining and validation, corresponding author Farhad Abtahi says.

Two decisions, not one approval

AEGIS divides the work among modules for incorporating data and retraining, monitoring models, and applying decision rules. Each candidate receives one of four outcomes: APPROVE, CONDITIONAL APPROVAL, CLINICAL REVIEW or REJECT. New versions are assessed against a fixed performance reference, using acceptance criteria configured for the clinical application.

An independent ALARM signal evaluates the previously released model. That permits a REJECT + ALARM result: no acceptable replacement is available, while the existing model is assessed as at risk. Rejecting an update therefore does not automatically give the current version a clean bill of health.

The authors designed AEGIS to support FDA predetermined change-control plans and EU medical-device requirements. They identify an implementation gap: regulations describe manufacturers’ duties without fully specifying the operating rules. Under the FDA framework, manufacturers define acceptable modifications and implementation criteria in advance; changes within an approved plan’s boundaries need no additional submissions.

Each clinical application requires its own validation and regulatory assessment.

Farhad Abtahi, corresponding author of the AEGIS study

What the demonstrations actually tested

The researchers used deliberately conventional medical models to exercise the governance layer. The two demonstrations tested different kinds of change:

• Sepsis prediction used PhysioNet data with deliberately introduced changes. This checked whether engineered conditions triggered the intended decisions, not detection performance on unexpected real-world changes.
• Brain-tumor outlining used BraTS imaging data with genuine differences across imaging cohorts. It applied the same modules and decision hierarchy, with rules configured for that application.

A monitoring gap remains: AEGIS detects changes in the distribution of incoming data, called covariate shift. It does not directly detect concept drift—a change in how those inputs relate to outcomes. The authors leave that capability for future work.