About this document
This document describe the setup detail for Interfacing, Installing, programming (basis) and testing depth cameras with MX6QDL based boards using the Robotic Operating System (ROS). If you are not using ROS you can also install the proper drivers and compile, in your Ubuntu system as explained on document: https://community.freescale.com/docs/DOC-330278
Supported NXP HW boards:
Software: Gcc, Ubuntu 14.04v, OpenCV, Openni, Python, ROS.
For installation steps of ROS on iMX6 boards in your board, please follow up:
https://community.freescale.com/docs/DOC-3301478
Before you can use ROS, you will need to initialize rosdep. It enables you to easily install system dependencies for source you want to compile and is required to run some core components in ROS.
$ sudo rosdep init
$ rosdep update
There are many different libraries and tools in ROS - not all compile fully on ARM. In this case we already have installed the ROS Base, however any other packages need to be installed individually. First install the following dependencies, which will take some time and space (~1.4 GB):
$
sudo apt-get install --no-install-recommends freeglut3-dev
libfreenect-devlibusb-1.0-0-dev
libudev-dev ros-indigo-camera-info-manager ros-indigo-dynamic-reconfigure
ros-indigo-image-transport ros-indigo-image-proc ros-indigo-depth-image-proc ros-indigo-tf
ros-indigo-openni-launch
ros-indigo-freenect-* ros-indigo-depthimage-to-laserscanros-indigo-image-view ros-indigo-camera-info-manager ros-indigo-dynamic-reconfigure
libudev-dev
doxygen graphviz openjdk-6-jdk ros-indigo-openni2-cameraros-indigo-openni2-launch ros-indigo-rqt-common-plugins ros-indigo-rqt-graph
There is no any additional installation to run kinect with ROS, If you are using kinect you can pass to part 4. However the packages used to run the PrimeSense / Asus Xtion on the i.Mx6 are not available over apt yet, so they need to be compiled from source. To use OpenNI2 with ROS, we only need the shared OpenNI2 libraries and the Drivers.
Clone OpenNI2
$ git clone https://github.com/OpenNI/OpenNI2
$ cd OpenNI2
Edit ThirdParty/PSCommon/BuildSystem/Platform.Arm
$ nano ThirdParty/PSCommon/BuildSystem/Platform.Arm
and replace
CFLAGS += -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp #-mcpu=cortex-a8
with
CFLAGS += -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=hard
Add support for pthread library:
$ nano ThirdParty/PSCommon/BuildSystem/CommonCppMakefile
Search the line 95 and add the code between the two lines:
OUTPUT_NAME = $(EXE_NAME)
# We want the executables to look for the .so's locally first:
LDFLAGS += -Wl,-rpath ./
+ ifneq ("$(OSTYPE)","Darwin")
+ LDFLAGS += -lpthread
+ endif
OUTPUT_COMMAND = $(CXX) -o $(OUTPUT_FILE) $(OBJ_FILES) $(LDFLAGS)
endif
Save the file and exit
Then run make to compile the OpenNI2 drivers and libraries
$ PLATFORM=Arm make ALLOW_WARNINGS=1
Once the compilation is done, run the linux install script
$ cd Packaging/Linux
$ sudo ./install.sh
Copy libraries and includes to the system paths
$ cd ../../
$ sudo cp -r Include /usr/include/openni2
$ sudo cp -r Bin/Arm-Release/OpenNI2 /usr/lib/
$ sudo cp Bin/Arm-Release/libOpenNI2.* /usr/lib/
Create a package config file
$ sudo nano /usr/lib/pkgconfig/libopenni2.pc
and fill it with this:
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include/openni2
Name: OpenNI2
Description: A general purpose driver for all OpenNI cameras.
Version: 2.2.0.0
Cflags: -I${includedir}
Libs: -L${libdir} -lOpenNI2 -L${libdir}/OpenNI2/Drivers -lDummyDevice -lOniFile -lPS1080.so
This will enable ubuntu to find the location of the drivers, libraries and include files. To make sure it is correctly found, run
$ pkg-config --modversion libopenni2
Which should give the same version as defined in the file above (2.2.0.0). Now the Xtion is ready to be used. Plug it in (if it is already, unplug it first), then run the sample program
$ ./Bin/Arm-Release/SimpleRead
Then create a catkin workspace as described here, and check out the following packages in the src folder of the catkin workspace:
$ cd ~/catkin_ws/src
$ git clone https://github.com/ros-drivers/openni2_camera
$ git clone https://github.com/ros-drivers/openni2_launch
$ git clone https://github.com/ros-drivers/rgbd_launch
Now the ros packages checked out above to the catkin workspace can be compiled with catkin_make
$ cd ~/catkin_ws
$ catkin_make
Once the packages are compiled, the Xtion is ready for use with ROS with
Terminal 1: Run ROS
$ roscore
Terminal 2: launch the Freenect
$ roslaunch freenect_launch freenect.launch
Terminal 3: run the image capture
$ rosrun image_view image_view image:=camera/rgb/image_color
or:$ rosrun image_view image_view image:=camera/rgb/image_rect_mono
or:$ rosrun image_view disparity_view image:=camera/depth/disparity
It will open a new terminal with the rgb points, mono and depth images from the Kinect.
Xtion.
Terminal 1: Run ROS
$ roscore
Terminal 2: launch Openni2
$ roslaun openi2_launch openni2.launch
Terminal 3: you can use rqt or Rviz session to visualize the sensor e.g:
$ rqt
or
$ rosrun rqt_graph rqt_graph
In the “rqt” window select “Plugins” -> “Visualization” -> “Image View“
(optional) Install PySide, in any case you get an error with python rqt graph:
$ pip install PySide
$ cd ~/
Note: rqt and Rviz demand a lot of i.MX GPU work, so general graphic functionality will be affected. For this case is suggested to run rviz in a remote Network ROS session.
References:
- https://dobots.nl/2014/05/05/asus-xtion-using-openni2-and-ros-on-udoo/