AI Chatbots Often Downplay Sleep Apnea Risks, Study Finds

Study Finds AI Chatbots Frequently Misguide Sleep Apnea Patients

A study presented at the European Respiratory Society Congress in Barcelona found that in a third of cases, AI chatbots wrongly reassured patients with symptoms of obstructive sleep apnea (OSA) that their symptoms were not serious. This often discouraged patients from seeking specialist referral.

The research, led by Dr. Deeban Ratneswaran of Guy's and St Thomas' NHS Foundation Trust, tested how five widely used free AI chatbots responded to simulated patient conversations about OSA symptoms.

How the Study Was Conducted and Who It Applies To

Researchers created seven realistic patient scenarios, each meeting the criteria for referral to a sleep study. They ran 700 conversations with ChatGPT, Google Gemini, Claude, DeepSeek, and Grok, using both cooperative and resistant patient attitudes.

When patients were open about their symptoms, chatbots advised referral every time. However, when patients downplayed symptoms, chatbots gave correct advice only 64% of the time. In severe cases, correct advice dropped as low as 22%.

Implications for Those Managing Metabolic Health

Obstructive sleep apnea is linked to high blood pressure, stroke, heart disease, and type 2 diabetes. Many people with OSA are undiagnosed, and early detection is important for metabolic health.

The study found that chatbots sometimes offered lifestyle tips instead of recommending referral, potentially delaying diagnosis and treatment for those at risk.

Limitations and Cautions for AI Use in Health Queries

The research highlights that chatbot responses may depend on how patients describe their symptoms, with models often providing reassurance when patients resist referral.

Dr. Io Hui of the European Respiratory Society, who was not involved in the study, emphasized the need for realistic testing of chatbots and cautioned that their advice may not always be reliable, especially for patients who minimize their symptoms.