Direct-to-Consumer AI Health Services: Precision Healthcare Needs Precision Consent
-
The University of Melbourne
Australia -
RMIT University Australia
-
The University of Melbourne Australia
-
University of Oxford United Kingdom
Abstract
Artificial Intelligence (AI) technology has the potential to replace a visit to the doctor, with AI companies offering health services directly to the public that, until recently, could only be performed by humans. AI apps that claim to detect disease, such as skin cancer, envisage a hopeful future where everyone can access expert medical care on their phone, at a fraction of the cost of traditional healthcare. However, the advent of AI in medical contexts also raises anxiety that AI may not be as reliable as claimed, and that laypersons are ill-equipped to make these decisions by themselves.
AI companies have a duty of care to provide their consumers with adequate notice of the risks and limitations of the AI system. But given the complex and technical nature of this information, how can such notice be made salient for consumers? The main argument of this article is that, if AI companies effectively offer a health service traditionally performed by doctors, then they should be guided by the values that govern the doctor-patient relationship regarding the information provided to consumers about the limitations of the AI system.
We propose Precision Consent, an interdisciplinary framework that draws connections between the legal and ethical duties of doctors towards their patients, and capabilities in the field of AI. We discuss how values that guide doctors, such as respect for patient autonomy, non-maleficence and personalised warnings, can be incorporated into how notice is provided to consumers regarding the accuracy of AI health services.
Downloads
Author Biographies
Hui Chia is a PhD Candidate at Melbourne Law School. Her thesis examines the question of legal responsibility for AI systems. Prior related publications include "Autonomous AI" published in Law, Innovation and Technology (2023), and a book chapter on product liability for emerging technology, in Evil Corporations (2024).
Daniel Beck is a Senior Lecturer in Machine Learning at RMIT, with a research focus on evaluation methods for machine learning and modelling uncertainty of AI systems.
Jeannie Marie Paterson is a Professor of Law at the University of Melbourne, and co-director of the Centre for AI and Digital Ethics. Jeannie’s expertise lies in the areas of consumer protection and the law of digital technologies.
Julian Savulescu has held the Uehiro Chair in Practical Ethics at the University of Oxford from 2002 to 2022, and is currently the Chen Su Lan Centennial Professor of Medicine and Director of the Centre of Biomedical Ethics at the National University of Singapore. His research focuses on the ethics of new and emerging biotechnologies.
