Comparing physician and artificial intelligence ChatGPT-4 responses to common patient questions regarding hidradenitis suppurativa: a single-blind study

Introduction:
Artificial intelligence in medicine has increased enormously in recent years.

Aim:
To evaluate the effectiveness of ChatGPT-4 in answering common patient queries about hidradenitis suppurativa.

Material and methods:
A group of 31 physicians and 30 patients suffering from hidradenitis suppurativa independently evaluated and compared the quality, empathy, and satisfaction of the ChatGPT-4’s and the dermatologist’s responses to patient queries, in a single-blind study, without any information about answers’ sources beforehand to minimize bias. Additionally, on the last page of the questionnaire, without the possibility of returning to assess the answers, the question was asked “Given that AI can answer your questions more accurately and empathetically than a doctor, who would you rather receive an answer from, the doctor or the AI”.

Results:
For each variable, patients as well as medical doctors rated the answers generated by ChatGPT-4 significantly better than the answers provided by the dermatologist (p < 0.00001 for each variable). Moreover, when asked which of the two answers patients would prefer to receive, they chose the answers generated by ChatGPT-4 out of an average of 88.33% of the questions. Among 30 patients, in the last question 21 people answered that, despite the lower quality and empathy, they would prefer to get an answer from a medical doctor.

Conclusions:
Our study revealed that despite ChatGPT-4’s responses being rated better than the physician’s, patients still prefer medical doctors as a source of information. Thus, AI will not replace physicians soon, although we should undoubtedly learn to live and cooperate with it.