Web search or conversation with an artificial intelligence? Analysis of misinformation and relevance in the case of radon gas

Abstract

Health-related information plays a crucial role in public health management, empowering individuals to make informed decisions and adopt behaviours that mitigate the effects of potential risks. The internet and the emergence of new technologies, such as conversational models equipped with Artificial Intelligence, present opportunities, and challenges in this field. This research focuses specifically on the risk of radon, a natural radioactive gas recognised worldwide as one of the leading causes of lung cancer and a persistent threat over time. The aim of this study is to analyse the information provided for this specific risk by two key information access tools: web search engines and AI-based conversational agents (ChatGPT). To carry out this interdisciplinary research (journalism-communication-computer science) we employ a mixed methodological design (quantitative and qualitative) and apply methods from the areas of Information Retrieval (IR), Big Data, and Artificial Intelligence (AI). The results of this study show that information on the internet about the risk of radon often lacks relevance and does not meet the information needs of users. We also found that some websites provide a significant amount of good quality information but there are often some misleading contents. ChatGPT proves to be more accurate in providing relevant and good quality information but contains a higher proportion of misinformation. Consequently, this raises concerns about the integrity of the information provided and emphasises the need to monitor and improve the accuracy of these computational tools