RT Journal Article
T1 A fine-tuning approach based on spatio-temporal features for few-shot video object detection
A1 Cores Costa, Daniel
A1 Seidenari, Lorenzo
A1 Bimbo, Alberto del
A1 Brea Sánchez, Víctor Manuel
A1 Mucientes Molina, Manuel
K1 Few-shot object detection
K1 Video object detection
K1 Few-shot learning
AB This paper describes a new Fine-Tuning approach for Few-Shot object detection in Videos that exploits spatio-temporal information to boost detection precision. Despite the progress made in the single image domain in recent years, the few-shot video object detection problem remains almost unexplored. A few-shot detector must quickly adapt to a new domain with a limited number of annotations per category. Therefore, it is not possible to include videos in the training set, hindering the spatio-temporal learning process. We propose augmenting each training image with synthetic frames to train the spatio-temporal module of our method. This module employs attention mechanisms to mine relationships between proposals across frames, effectively leveraging spatio-temporal information. A spatio-temporal double head then localizes objects in the current frame while classifying them using both context from nearby frames and information from the current frame. Finally, the predicted scores are fed into a long-term object-linking method that generates object tubes across the video. By optimizing the classification score based on these tubes, our approach ensures spatio-temporal consistency. Classification is the primary challenge in few-shot object detection. Our results show that spatio-temporal information helps to mitigate this issue, paving the way for future research in this direction. FTFSVid achieves 41.9 AP50 on the Few-Shot Video Object Detection (FSVOD-500) and 42.9 AP50 on the Few-Shot YouTube Video (FSYTV-40) dataset, surpassing our spatial baseline by 4.3 and 2.5 points. Additionally, FTFSVid outperforms previous few-shot video object detectors by 3.2 points on FSVOD-500 and 14.5 points on FSYTV-40, setting a new state-of-the-art.
PB Elsevier
SN 0952-1976
YR 2025
FD 2025-04-15
LK https://hdl.handle.net/10347/43086
UL https://hdl.handle.net/10347/43086
LA eng
NO Cores, D., Seidenari, L., Bimbo, A. D., Brea, V. M., & Mucientes, M. (2025). A fine-tuning approach based on spatio-temporal features for few-shot video object detection. Engineering Applications of Artificial Intelligence, 146, 110198. 10.1016/j.engappai.2025.110198
NO This research was partially funded by the Spanish Ministerio de Ciencia e Innovación (grant number PID2020-112623GB-I00), and the Galician Consellería de Cultura, Educación e Universidade (grant numbers ED431C 2018/29, ED431C 2021/048, ED431G 2019/04). These grants are co-funded by the European Regional Development Fund (ERDF).
DS Minerva
RD 22 abr 2026