Skip to main content
Auf einen Blick

The joint research project Smart3D with the IFM group is developing innovative AI-based sensor fusion and object classification. Autonomous driving uses for example PMD cameras which generate three-dimensional photos by using the so called “Time of Flight”-methods. Unfortunately, the resolution of those cameras in outdoor use is too low for a reliable object classification.

The Smart3D project aims to improve such a low-resolution 3D image by the information of a high-resolution 2D color image and matching the color information with the corresponding information of the 3D image on a pixel-based level. Unfortunately, the use of simple interpolation methods is nor successful, especially regarding the transition of objects in combination with a far-away background.  A new research approach is based on the machine learning methods by training a neural network with different data sources. This approach is transcribed to PMD data within the scope of Smart3D.

Schematische Darstellung der Fusion eines PMD-Tiefenbildes mit einer 2D-Farbbild

Project partners

IFM electronic IFM electronic