Fully automated 3D-data interpretation by deep learning
With the availability of a multitude of increasingly sensitive environmental sensors that produce streams of raw data, scaling the interpretation of said data is a pressing issue. Deep learning offers a set of tools to help make sense of huge amounts of data without requiring manual analysis of every data point. We describe a pipeline that produces a semantic interpretation of a georeferenced point cloud produced from 3D laser scanning with the help of complementary 2D RGB image data. A fully convolutional neural network trained on a large amount of custom collected and annotated data performs a semantic segmentation of the images; we then fuse segmented images and point cloud to extract 3D object instances, which serve as input for different applications.