ToF camera
This page describes the integration of PMD Flexx2 ToF camera with NavQPlus
Last updated
This page describes the integration of PMD Flexx2 ToF camera with NavQPlus
Last updated
A Time-of-Flight camera (ToF) is a range imaging sensor that produces a depth image, each pixel of which encodes the distance to the corresponding point in the scene [1].
In addition to the camera that produces 2D images, the ToF sensor includes an IR transmitter. By measuring the phase difference between the radiated (red) and reflected (blue) IR waves, we can calculate the distance to the object as depicted in the figure below.
The PMD Flexx2 camera intended use are: augmented Reality / Virtual Reality and SLAM. It has the following specifications:
Depth Measurements from 10 cm to 7 m
VCSEL with 940 nm Wavelength
38,000 3D Pixels with up to 60 FPS
Independent of External Light Source
224 x 172 Resolution
Multiple built-in User Modes
56 x 44 Degree Field-of-View
USB 3.0 Type-C Port
71.9 x 19.2 x 10.6 mm compact size
Software Suite "Royale" and API
Download the royale SDK from the official website: https://pmdtec.com/en/download-sdk/
You need to fill in the form then you will get an email with a download link available for 24 hours. Use the password that comes with the camera.
You need to download two .zip files: one for your PC where you will display the camera data connected to the PC or the NavQP, and one for the NavQP.
The recent SDK for Flexx2 camera does not have the ROS2 driver. But the previous SDK does. Then, as recommendation, download the zip file libroyale-4.24.0.1201-LINUX-arm-64Bit-ub2004.zip for NavQP.
Linux PC: libroyale-<ROYALE-SDK-VERSION>-LINUX-x86-64Bit.zip
Windows PC: libroyale-<ROYALE-SDK-VERSION>-WINDOWS-x86-64Bit.exe.zip
NavQP: libroyale-4.24.0.1201-LINUX-arm-64Bit-ub2004.zip
Connect the camera to the PC using a USB-A to USB-C cable.
If you are using a Linux machine, download the zip file libroyale-<ROYALE-SDK-VERSION>-LINUX-x86-64Bit.zip from the download page, add the udev rules then run royaleviewer after reboot:
A GUI will start, click on start button on the bottom right corner to start the display:
You can use the TOOLS button to change some parameters. For example you can change the color range, or the type of data displayed (Gray image, depth image), etc
There is also a LOG button on the top right corner of the GUI where you can find useful debug or error logs.
If you are using a windows machine, install the zip file libroyale-<ROYALE-SDK-VERSION>-WINDOWS-x86-64Bit.exe. Unzip the file then run the executable to install the Royale Viewer.
Once installed you can find the executable with the name royaleviewer-royale_<ROYALE-SDK-VERSION> under C:\Users\<USER>\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\royale_<ROYALE-SDK-VERSION> (Win64)
Once the program is launched, the steps are similar to those described in Linux sub-section.
Connect the camera to the NavQPlus with an USB-C to USB-C cable (see picture below).
Connecting the camera to the NavQPlus uses a USB-C to USB-C cable. A USB-A to USB-C cable will not work and the camera will not seen by the board. The USB port is configured correctly as HOST.
Before using the camera, you need to add the udev rules available in the SDK zip file that you downloaded from PMD download link.
Under the bin/ directory, run the binary tcpserver:
The Royale SDK provides some sample codes available under samples/cpp/ directory. You should find a directory sampleROS2, which contains code for ROS2 driver.
You need to adjust this code to be able to compile in ROS2 Humble. First create a ROS2 workspace and copy the ROS2 driver there:
Then change the CMakeLists.txt file and the package.xml file to add the package pluginlib dependency:
Then compile the ROS2 workspace using the following command:
Change the path to Royale SDK depending on your environment, and make sure to reference to share/ directory in this command.
Make sure that mlan0 interface in listed in the list of interfaces in CycloneDDSConfig.xml file, then run RVIZ in a PC:
[1] Hansard, Miles, Seungkyu Lee, Ouk Choi, and Radu Horaud. Time-of-Flight Cameras: Principles, Methods and Applications. SpringerBriefs in Computer Science. London: Springer London, 2013. https://doi.org/10.1007/978-1-4471-4658-2.
[2] https://3d.pmdtec.com/en/3d-cameras/flexx2/
[3] SDK download link: https://pmdtec.com/en/download-sdk/
