RT Journal Article
T1 Clever domain adaptation strategies for BERT in the task of hostile-language detection
A1 Villa Cueva, Emilio
A1 Aragón Saenzpardo, Mario Ezra
A1 López Monroy, Adrián
A1 Sánchez Vega, Fernando
K1 Hostile language
K1 Domain adaptation
K1 Social media
K1 Text classification
AB Cyberbullying has experienced a surge in recent years, mainly due to the widespread adoption of social media platforms. This trend manifests in multiple ways, with hostile language being one of the most common. The latter underscores the urgent need for robust detection methods to address this issue effectively. To address this problem, we propose a novel pipeline to enhance hostile language detection in social media. Our approach consists of a combination of two ideas: First, we propose conducting a Domain Adaptation procedure to specialize the knowledge of a pre-trained BERT, making it more specialized in the domain of social media. For this adaptation, we modify the traditional random Masked Language Modeling technique and propose three novel strategies for selecting the subset of tokens to mask out cleverly. Second, we tailor an Adversarial Regularizer when fine-tuning the adapted BERT for specific hostile-language datasets. We evaluate the performance of our method for detecting hate speech, aggressiveness, offensiveness, and sexism. Our results show that the Domain Adaptation procedure significantly outperforms vanilla BERT, and the Adversarial Regularizer can lead to more robust fine-tuning, thereby enhancing performance. Moreover, we demonstrate that these methods can be used together to achieve an even more significant performance boost.
PB Springer
YR 2026
FD 2026-03-31
LK https://hdl.handle.net/10347/46880
UL https://hdl.handle.net/10347/46880
LA eng
NO Villa-Cueva, E., Aragón, M.E., López-Monroy, A.P., & Sánchez-Vega, F. (2026) Clever domain adaptation strategies for BERT in the task of hostile-language detection.  Multimedia Tools and Applications, 85(323). https://doi.org/10.1007/s11042-026-21521-1
NO Villa-Cueva (CVU 1019520) thanks CONAHCYT for the support through the master’s degree scholarship at CIMAT. This work was supported by the project CBF-2025-I-4384, “Grandes Modelos de Lenguaje Especializados para Detectar Ciberacoso y Violencia Digital”, approved under the Ciencia Básica y de Frontera 2025 call of SECIHTI, Mexico. Mario Ezra Aragón thanks the support obtained from MICIU/AEI/10.13039/501100011033 (PID2022-137061OB-C22, supported by ERDF), Xunta de Galicia-Consellería de Cultura, Educación, Formación Profesional e Universidades (ED431G 2023/04, ED431C 2022/19, supported by ERDF), and the support obtained from the Juan de la Cierva Grant (JDC2023-052296-I), funded by MCIN/AEI/10.13039/501100011033 and by the FSE+. Sanchez-Vega acknowledges CONAHCYT / SECIHTI’s support through the program “Investigadoras e Investigadores por México” (Project ID 11989, No. 1311).
NO Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This research was funded by Xunta de Galicia, Ministerio de Ciencia e Innovación (Spain), and CONAHCYT (Mexico).
DS Minerva
RD 6 may 2026