Confalonieri, RobertoAlonso Moral, José María2024-02-052024-02-052023-11R. Confalonieri, J. M. Alonso-Moral, "An operational framework for guiding human evaluation in Explainable and Trustworthy AI", IEEE Intelligent Systems, 2023, https://doi.org/10.1109/MIS.2023.33346391541-1672http://hdl.handle.net/10347/32315The assessment of explanations by humans presents a significant challenge within the context of Explainable and Trustworthy AI. This is attributed not only to the absence of universal metrics and standardized evaluation methods, but also to complexities tied to devising user studies that assess the perceived human comprehensibility of these explanations. To address this gap, we introduce a survey-based methodology for guiding the human evaluation of explanations. This approach amalgamates leading practices from existing literature and is implemented as an operational framework. This framework assists researchers throughout the evaluation process, encompassing hypothesis formulation, online user study implementation and deployment, and analysis and interpretation of collected data. The application of this framework is exemplified through two practical user studies.engThis work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.http://creativecommons.org/licenses/by-nc-nd/4.0/Artificial IntelligenceTaxonomyPlanningIntelligent SystemsCognitive ScienceEthicsTask Analysisjournal article10.1109/MIS.2023.33346391941-1294open access