RT Journal Article
T1 Deep learning with simulated laser scanning data for 3D point cloud classification
A1 Esmorís Pena, Alberto Manuel
A1 Weiser, Hannah
A1 Winiwarter, Lukas
A1 Cabaleiro Domínguez, José Carlos
A1 Höfle, Bernhard
K1 Virtual laser scanning
K1 LiDAR simulation
K1 Point clouds
K1 Machine learning
K1 Point-wise classification
K1 Leaf-wood segmentation
AB Laser scanning is an active remote sensing technique applied in many disciplines to acquire state-of-the-art spatial measurements. Semantic labeling is often necessary to extract information from the raw point cloud. Deep learning methods constitute a data-hungry solution for the semantic segmentation of point clouds. In this work, we investigate the use of simulated laser scanning for training deep learning models, which are applied to real data subsequently. We show that training a deep learning model purely on virtual laser scanning data can produce results comparable to models trained on real data when evaluated on real data. For leaf-wood segmentation of trees, using the KPConv model trained with virtual data achieves 93.7% overall accuracy, while the model trained on real data reaches 94.7% overall accuracy. In urban contexts, a KPConv model trained on virtual data achieves 74.1% overall accuracy on real validation data, while the model trained on real data achieves 82.4%. Our models outperform the state-of-the-art model FSCT in terms of generalization to unseen real data as well as a baseline model trained on points randomly sampled from the tree mesh surface. From our results, we conclude that the combination of laser scanning simulation and deep learning is a cost-effective alternative to real data acquisition and manual labeling in the domain of geospatial point cloud analysis. The strengths of this approach are that (a) a large amount of diverse laser scanning training data can be generated quickly and without the need for expensive equipment, (b) the simulation configurations can be adapted so that the virtual training data have similar characteristics to the targeted real data, and (c) the whole workflow can be automated through procedural scene generation.
PB Elsevier
YR 2024
FD 2024-07-13
LK https://hdl.handle.net/10347/38723
UL https://hdl.handle.net/10347/38723
LA eng
NO ISPRS Journal of Photogrammetry and Remote Sensing Volume 215, September 2024, Pages 192-213
NO This research was funded by the Deutsche Forschungsgemeinschaft (DFG), German Research Foundation, by the projects SYSSIFOSS (Grant Number: 411263134) and VirtuaLearn3D (Grant Number: 496418931). It was furthermore supported by the BMBF (Federal Ministry for Education and Research, Germany) in the frame of the AImon5.0 project (Funding code: 02WDG1696). This work has also received financial support from the Consellería de Cultura, Educación e Ordenación Universitaria (accreditation ED431C 2022/16 and accreditation ED431G-2019/04) and the European Regional Development Fund (ERDF), which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System, and the Ministry of Economy and Competitiveness, Government of Spain (Grant Number PID2019-104834GB-I00 and PID2022-141623NB-I00). The many deep learning experiments computed on the FinisTerrae-III supercomputer were possible thanks to the CESGA (Galician supercomputing center). Diverse experiments were also possible thanks to the data curators of the Hessigheim and Wytham Woods datasets and the manually labeled leaf-wood datasets. The authors gratefully acknowledge support by the state of Baden-Württemberg through bwHPC and the German Research Foundation (DFG) through grant INST 35/1597-1 FUGG. Furthermore, the authors acknowledge the data storage service SDS@hd supported by the Ministry of Science, Research and the Arts Baden-Württemberg (MWK) and the German Research Foundation (DFG) through grant INST 35/1503-1 FUGG.
DS Minerva
RD 3 may 2026