Amazon Alexa's robotic voice causes "deep distress" for dementia patients by telling them to take their medicine, a new concern for technology taking over humans, a report has said.
A technology think tank Doteveryone report said older patients often felt confused by new gadgets and feared they would replace contact with human carers, and disabled people worried that features like self-opening doors and windows could malfunction and leave them trapped.
The report suggested technology had a key role in improving the care system, but should not lead to "naive enthusiasm" that gadgets replace human carers.
The findings after examining more than 100 carers and their patients come as Hampshire County Council last year gave social care patients Amazon Alexa smart speaker devices to help remind them of medicine and when their carers were coming.
The report called for the government to invest in training carers to use technologies to take the strain off their workload so they could spend more time with patients and to research on how tech could be used as "a tool" rather than a replacement for human workers.
A spokesman for the Department of Health and Social Care said technology offered an excellent opportunity to enable people to stay connected, manage their specific care needs and stay in their homes and communities for longer, but could "never replace the value and importance of carers".
Doteveryone studied how technology helped the UK's struggling social care system, and found Alexa left dementia patients "often deeply distressed by an unfamiliar robotic voice reminding them to take medication".
"If you are on a dementia ward there is no point having robots telling people what to do as they will just get distressed and confused," The Telegraph UK cited lead researcher Lydia Nicholas as saying.
According to an interviewee, her elderly mother who has the condition thought her distress alarm was a button for her to let people know when she wanted coffee.
Nicholas said interviewee found their smart home fantastic, but the thought of completely relying on it was terrifying. "They were distressed... thought they could be trapped in a house that was getting hotter and hotter if they could not open the windows."
The report also suggested for Enablement Panels run by disabled people to educate those in care and their families on how new technology could be used in an effective way.
"Technology has the potential to be part of the solution to the current problems we see in the social care system, but only if it's used responsibly and strategically," said Rachel Coldicutt, CEO of Doteveryone.
The CEO argued the report was a challenge to the naive enthusiasm for technology and a call to create a robust care innovation strategy.