Then there is the risk of bias. AI models may be trained on datasets that might not reflect the populations they will be applied to, exacerbating health inequalities based on things like gender or ethnicity.
Therefore, regulation is key. It needs to keep patients safe and protect their personal data, whilst at the same time increasing capacity to keep up with developments and allow AI to evolve and learn on the job.
AI-powered medical devices are tightly regulated by the Medicines and Healthcare products Regulatory Agency (MHRA).
The Health Foundation think tank recently published a six-point national strategy, external to ensure AI tools are rolled out fairly and regulation is updated.
Nell Thornton, improvement fellow at the Health Foundation, says: “There are so many of these models coming through the system that it’s difficult to assess them quickly enough.
“That’s where we need support around the capacity of the system to regulate these things and we also need some clarity on some of the challenges that will come from the quirkiness of generative AI systems and what additional regulation they might need.”
Dr Paul Campbell, MHRA Head of Software and AI, says: “As a regulator, we must balance appropriate oversight to protect patient safety with the agility needed to respond to the particular challenges presented by these products to ensure we continue to be an enabler for innovation.”
The Department of Health and Social Care says the new Labour government will “harness the power of AI” by purchasing new AI-enabled scanners to diagnose patients earlier and treat them faster.
While few can deny the transformative effect AI is having within healthcare, there are challenges to overcome, not least that NHS staff need the confidence to use it and patients must be able to trust it.