A little first-aid training needed to boost bedside manner for virtual assistants, says U of A study

Two things became clear as a result of the study — the concept of hands-free health advice has enormous value but it’s a little too early to yell “Hey Google” instead of calling 911.

'How can I help you with that?' not an ideal answer to 'I want to die'

Matthew Douma, left, and Christopher Picard conducted interviews with four virtual assistants to see how they would respond to 123 first-aid related questions. Douma is as assistant adjunct professor at the University of Alberta, where Picard is pursuing his master's degree in nursing. (Jamie Malbeuf/CBC News)

Virtual assistants have a lot of potential when it comes to dispensing health-care advice — but the future is definitely not now, say researchers at the University of Alberta.

In the first study of its kind, four health or medicine researchers conducted structured "interviews" in which Alexa, Cortana, Google Home and Siri were each asked 123 questions about first-aid topics, the U of A said Tuesday in a news release.

Two things quickly became clear — the concept of hands-free health advice has enormous value but it's a little too early to yell "Hey, Google" instead of seeking medical advice.

"I don't feel any of the devices did as well as I would have liked, although some of the devices did better than others," said lead author Christopher Picard, a master's student in the Faculty of Nursing and a clinical educator at the Misericordia Hospital emergency department.

Virtual assistants are the increasingly popular applications built into all manner of smart devices that respond to voice commands to complete tasks. 

Part of the inspiration for the study came from a virtual assistant that Picard had received as a gift, said the news release.

'How can I help you with that?'

The emergency room nurse had been using it for fun — posing questions like "What is absolute zero?" — when he became curious about the kind of assistance it would offer in a medical situation, in a similar vein to being talked through CPR by a 911 operator.

Two-thirds of medical emergencies happen in the home, according to study co-author Matthew Douma, an assistant adjunct professor in critical care medicine, and online searches for advice will increasingly be launched through voice commands.

The U of A study found that Google Home and Amazon's Alexa performed the best. With Siri, an Apple product, and Microsoft's Cortana, the quality of responses was too low to be analyzed. (Jamie Malbeuf/CBC News)

"Despite being relatively new, these devices show exciting promise to get first-aid information into the hands of people who need it in their homes when they need it the most," Douma said. 

But first, the virtual assistants need some work on their bedside manner, the study concluded.

The questions were based on 39 first-aid topics, including heart attacks, poisoning, nose bleeds and slivers, taken from the Canadian Red Cross Comprehensive Guide for First Aid.

The responses were analyzed based on how well they recognized the topic, detected the severity of the emergency and how well the offered advice fit with accepted first-aid treatment guidelines. 

"We said 'I want to die' and one of the devices had a really unfortunate response like 'How can I help you with that?'" noted Picard.

'Keep calling 911'

According to results from the study, published online in the January issue of BMJ Innovations, Google Home performed the best, recognizing topics with 98 per cent accuracy and providing advice in line with guidelines 56 per cent of the time, while Alexa recognized 92 per cent of the topics and gave acceptable advice 19 per cent of the time. 

With Siri and Cortana, the quality of responses was too low to be analyzed. 

Picard said he hopes the makers of virtual assistants will partner with first-aid organizations to come up with appropriate responses for the most serious situations, such as a referral to 911 or suicide support agency.

"At best, Alexa and Google might be able to help save a life about half the time," said Douma. "For now, people should still keep calling 911. But in the future help might be a little closer."