Free beta access. 50 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

Automation supercharges risk homeostasis

Dr Cosima Gretton ’17. June 24, “The dangers of AI in health care: risk homeostasis and automation bias,” https://towardsdatascience.com/the-dangers-of-ai-in-health-care-risk-homeostasis-and-automation-bias-148477a9080f

The performance of a system when applied to messy, real-world clinical practice will no doubt have an accuracy of less than 100%, so human operators must know when to trust, and when not to trust the system. How we design the interaction between the human and machine becomes of utmost importance to prevent new biases and errors being introduced.

Risk homeostasis would suggest that over-automating aspects of clinical practice might lead to complacency and an increase in errors and accidents. Studies from other fields have shown that humans do indeed suffer from automation bias and a reduction in personal accountability when their tasks are taken over by machines.

Sign in to recut, update, or save this card.