Evaluating search engines and large language models for answering health questions

Fernández Pichel, MarcosPichel Campos, Juan CarlosLosada Carril, David Enrique2025-12-172025-12-172025-03-10Fernández-Pichel, M., Pichel, J.C. & Losada, D.E. Evaluating search engines and large language models for answering health questions. npj Digit. Med. 8, 153 (2025). https://doi.org/10.1038/s41746-025-01546-whttps://hdl.handle.net/10347/44538Search engines (SEs) have traditionally been primary tools for information seeking, but the new large language models (LLMs) are emerging as powerful alternatives, particularly for question-answering tasks. This study compares the performance of four popular SEs, seven LLMs, and retrieval-augmented (RAG) variants in answering 150 health-related questions from the TREC Health Misinformation (HM) Track. Results reveal SEs correctly answer 50–70% of questions, often hindered by many retrieval results not responding to the health question. LLMs deliver higher accuracy, correctly answering about 80% of questions, though their performance is sensitive to input prompts. RAG methods significantly enhance smaller LLMs’ effectiveness, improving accuracy by up to 30% by integrating retrieval evidence.engOpen Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/Evaluating search engines and large language models for answering health questionsjournal article10.1038/s41746-025-01546-w2398-6352open access