Reprod Biomed Online. 2025 Aug 14;52(4):105221. doi: 10.1016/j.rbmo.2025.105221. Online ahead of print.
ABSTRACT
RESEARCH QUESTION: Is the quality, relevance and empathy of the answers provided by large language models (LLMs) in response to the most frequently asked patient questions in reproductive medicine comparable to those provided by human specialists?
DESIGN: This monocentric, double blind, prospective study involved two clinicians and two embryologists who answered 13 frequently asked questions in their respective field. The same questions were asked to a free online LLM, with the same constraint of text length as practitioners. All answers were blindly evaluated by four assessors (two gynaecologists and two embryologists depending on the topic) for quality and accuracy. A psychologist also evaluated empathy.
RESULTS: The mean number of words per answer was significantly higher (P < 0.001) for LLM than for humans. The average quality of answers was not statistically different between LLM and professionals. No answer provided by LLM was evaluated as completely aberrant, and only a minority contained false or inappropriate information or was scored as being very poor by assessors. Answers provided by embryologists, but not clinicians, ranked significantly higher (P = 0.02) than LLM. The psychologist chose LLM answers as most empathetic, clear, or both, in 14 out of 26 questions.
CONCLUSIONS: LLMs could be used as an educational tool within assisted reproductive technology centres to answer frequently asked patient questions. Although the potential applications of LLMs’ capabilities in answering medical questions are numerous, this should be carefully evaluated and regulated to prevent the dissemination of inaccurate information to patients.
PMID:41771212 | DOI:10.1016/j.rbmo.2025.105221
