View source for PCL/OpenNI tutorial 1: Installing and testing

Go to root: [[PhD-3D-Object-Tracking]]

----
----


[[Image:Kinect.jpg|thumb|right|200px|Microsoft Kinect device.]]

This series of tutorials will explain the usage of a depth camera like [https://en.wikipedia.org/wiki/Kinect Kinect] for "serious" researching purposes. As you may know, Kinect is in fact an affordable depth sensor, developed with technology from [https://en.wikipedia.org/wiki/PrimeSense PrimeSense], based on infrarred structured light method. It also has a common camera (which makes it a RGB-D device), a microphone and a motorized pivot. Its use is not limited to playing with a Xbox360 console, you can plug it to a computer and use it like any other sensor. Many open-source drivers and frameworks are available.

Since its release on November 2010, it has gained a lot of popularity, specially among the scientific community. Many researches have procured themselves one because, despite the low cost (about 100 $), it has proven to be a powerful solution for depth sensing projects. Current investigations focus on real-time surface mapping, object recognition and tracking, and localization. Impressive results (like the [http://research.microsoft.com/en-us/projects/surfacerecon/ KinectFusion] project from Microsoft) are already possible.

The new [https://en.wikipedia.org/wiki/Xbox_One Xbox One] ships with an upgraded version, [https://en.wikipedia.org/wiki/Kinect_for_Xbox_One Kinect v2], with enhanced resolution, that is able to detect your facial expression, measure your heart rate, and track every one of your fingers. A PC development-ready version ([https://en.wikipedia.org/wiki/Kinect_for_Xbox_One#Kinect_for_Windows Kinect for Windows v2]) was released in July 2014, but it could only be used with the official Windows SDK (open source support [https://github.com/OpenKinect/libfreenect2 exists] but is still young). In October 2014 an adapter that allows to plug the standard Kinect v2 to a PC was released, so the development version of the sensor was discontinued in April 2015. Now you can just buy the [http://www.microsoftstore.com/store/msusa/en_US/pdp/Kinect-for-Xbox-One/productID.307499400 standard one] for the console plus the [http://www.microsoftstore.com/store/msusa/en_US/pdp/Kinect-Adapter-for-Windows/productID.308803600 adapter].

I will explain the installation and usage of one of the "legacy" Kinect 1.0 devices with a common PC, and the possibilities it offers. I will do it in an easy to understand way, intended for students that have just acquired it and want to start from scratch.

Keep in mind that the software that we are going to use (Point Cloud Library and OpenNI drivers) will also let you use any other device like the [https://www.asus.com/3D-Sensor/Xtion_PRO/ Xtion PRO] or [https://www.asus.com/3D-Sensor/Xtion_PRO_LIVE/ Xtion PRO LIVE] from ASUS (the PRO only has a depth sensor, the PRO LIVE is a RGB-D camera) without changing a line of code.

<span style="color:#606060">'''''NOTE: The tutorials are written for Linux platforms. Also, 64-bit versions seem to work better than 32-bit.'''''</span>

=Requirements=

You will need the following:

* A common Kinect device, out of the box. You can buy it in your local electronics shop, or online. It also includes a free copy of ''[https://en.wikipedia.org/wiki/Kinect_Adventures! Kinect Adventures!]'', which is useless if you do not own the console. Microsoft has released a ''Kinect for Windows'' device, which is a normal looking Kinect no longer compatible with Xbox360, that will only work with their official SDK, intended for developers only. Also, like I stated earlier, you can use an ASUS Xtion indistinctly.
* A computer running Linux (Debian or Ubuntu preferably).
* A medium-sized room. Kinect has some limitations for depth measurement: 40cm minimum, 8m maximum (make it 6).

<span style="color:#606060">'''''NOTE: ''Kinect for Windows'' may have problems working with open source drivers on Linux .'''''</span>

=Connecting everything=

Kinect does not work with a common USB port. Its power consumption is a bit higher because of the motor, so Microsoft came up with a connector that combines USB and power supply. Old Xbox 360 models needed a special adapter, new S model ones already have this new port. Luckily, Kinect comes with the official adapter out of the box (otherwise you will have to [http://www.amazon.com/Power-Supply-Cable-Kinect-Xbox-360/dp/B004S7GA46/ref=sr_1_1?ie=UTF8&qid=1378288327&sr=8-1 buy one]).

Just plug the adapter to any power socket, and the USB to your computer. Let's check typing this in a terminal:

<syntaxhighlight lang=Bash>lsusb</syntaxhighlight>

Output should list the following devices:

<syntaxhighlight lang=Bash>Bus 001 Device 005: ID 045e:02b0 Microsoft Corp. Xbox NUI Motor
Bus 001 Device 006: ID 045e:02ad Microsoft Corp. Xbox NUI Audio
Bus 001 Device 007: ID 045e:02ae Microsoft Corp. Xbox NUI Camera</syntaxhighlight>

If you are using a Xtion, you should see:

<syntaxhighlight lang=Bash>Bus 001 Device 004: ID 1d27:0601 ASUS</syntaxhighlight>

"0601" identifies the new Xtion model, "0600" the older one. Both should work the same. But try to avoid [[PCL/OpenNI_troubleshooting#ASUS_Xtion_PRO|USB 3.0 ports]]!

=Installing the software=

There is more than one way to get your Kinect working on your PC, and start developing applications for it:

* [https://www.microsoft.com/en-us/download/details.aspx?id=40278 Kinect for Windows SDK] and [https://www.microsoft.com/en-us/download/details.aspx?id=40276 Developer Toolkit]: released on June 16, 2011 as a non-commercial SDK intended for application development. Version 1.8, the last there will ever be now that Kinect v2 is out, was released on September 2013. Because it comes from Microsoft, it is obviously the easiest way to get everything working. Sadly, there is no Linux version.
* [https://github.com/OpenKinect/libfreenect libfreenect] library: from the [http://openkinect.org/wiki/Main_Page OpenKinect] project, it is intended to be a free and open source alternative to the official drivers. libfreenect is used by projects like [https://github.com/ofTheo/ofxKinect ofxKinect], an addon for the [http://www.openframeworks.cc/ openFrameworks] toolkit (and as of version 0.8, included in the core package) that runs on Linux and OS X. ofxKinect packs a nice example application to show the RGB and point cloud taken from Kinect.
* [https://github.com/PrimeSense/Sensor PrimeSense drivers]: they were released as open source after the the [https://en.wikipedia.org/wiki/OpenNI OpenNI] project was created, along with the motion tracking middleware, ''NITE'', and the SDK. NI stands for Natural Interaction, and the project tried to enforce a common standard for human input using Kinect-like sensors. These official drivers are used by [http://wiki.ros.org/openni_kinect ROS] (the Robot Operating System, a massive collection of libraries and tools for robotic researchers) and [http://pointclouds.org/ PCL] (the Point Cloud Library, with everything needed for 3D point cloud processing). Sadly, version 2.0 of the OpenNI SDK dropped support for Kinect on Linux due to licensing issues, so for now PCL relies on legacy 1.x versions. Also, Apple bought PrimeSense on November 2013, and on April 2014 OpenNI's webpage was closed. The source is now being maintained by [http://structure.io/openni a third party].
* [https://github.com/avin2/SensorKinect SensorKinect]: a modified version of the official PrimeSense drivers, used for example by [https://github.com/gameoverhack/ofxOpenNI ofxOpenNI] (another openFrameworks addon). Last updated on 2012.

For this tutorial, we are going to use PCL with the OpenNI drivers, so owners of a Xtion can also get it to work.

==Precompiled PCL for Ubuntu==

If you have a valid [http://wiki.ros.org/ROS/Installation installation of ROS] (through their repository), you do not have to install anything. Both OpenNI and PrimeSense drivers, as well as PCL, should be already installed. You can check it with:

<syntaxhighlight lang=Bash>sudo apt-get install libpcl-1.7-all libpcl-1.7-all-dev libopenni-dev libopenni-sensor-primesense-dev</syntaxhighlight>

The previous command should state that all packages are already installed (change the PCL version number as needed), install them if not. If you get an error about overwriting some file, check [[PCL/OpenNI_troubleshooting#Trying_to_overwrite_X.2C_which_is_also_in_package_Y|this]].

In case you do not want ROS, there is a [https://launchpad.net/~v-launchpad-jochen-sprickerhof-de/+archive/ubuntu/pcl PPA] (Personal Package Archive, a private repository) which has everything we need. Add it to your sources, and install everything:

<syntaxhighlight lang=Bash>sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl
sudo apt-get update
sudo apt-get install build-essential cmake libpcl1.7 libpcl-dev pcl-tools</syntaxhighlight>

Trying to mix ROS and PCL repositories and packages can cause some errors, so choose one of them and stick with it. Check the [[PCL/OpenNI troubleshooting]] page because your Kinect may not work by default in 32 bits.

==Compiling PCL from source==

For Linuxes without a precompiled version of PCL, you will need to compile it yourself. This has an advantage, actually: you can customize the build options and choose what you want. Also, the resulting binaries and libraries should be a bit faster. And you always get the latest version! The instructions are [http://pointclouds.org/documentation/tutorials/compiling_pcl_posix.php here], but the steps are easy so I will show them to you.

===Installing the dependencies===

Some of PCL dependencies can be installed via the package manager. Others will require additional work.

<syntaxhighlight lang=Bash enclose="div">sudo apt-get install build-essential cmake cmake-curses-gui libboost-all-dev libeigen3-dev libflann-dev libvtk5-dev libvtk5-qt4-dev libglew-dev libxmu-dev libsuitesparse-dev libqhull-dev libpcap-dev libxmu-dev libxi-dev libgtest-dev libqt4-opengl-dev</syntaxhighlight>

The trunk (1.8) version of PCL uses [https://en.wikipedia.org/wiki/VTK VTK] 6 and [https://en.wikipedia.org/wiki/Qt_%28software%29 Qt] 5, so if you intend to compile it, you must install the following packages (say yes if you are asked to remove VTK 5 and Qt 4 packages):

<syntaxhighlight lang=Bash enclose="div">sudo apt-get install libvtk6-dev libqt5opengl5-dev</syntaxhighlight>

====JDK====

OpenNI requires Sun's official JDK (Java Development Kit), which is no longer available on official apt repositories. You can use [http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html unofficial ones], or do it manually. For the last method, go to the [http://www.oracle.com/technetwork/java/javase/downloads/index.html Java SE downloads] page (SE means Standard Edition) and download the latest version (e.g., <span style="color:#1E90FF">''jdk-8u66-linux-x64.tar.gz''</span>). Extract it, then move the contents to <span style="color:#FF8C00">''/usr/lib/jvm/''</span> so it is available system-wide:

<syntaxhighlight lang=Bash>sudo mkdir -p /usr/lib/jvm/
sudo cp -r jdk1.8.0_66/ /usr/lib/jvm/</syntaxhighlight>

Then, make it the default choice to compile and run Java programs. Remember to change the version number as needed!

<syntaxhighlight lang=Bash>sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.8.0_66/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.8.0_66/bin/javac" 1
sudo update-alternatives --install "/usr/bin/jar" "jar" "/usr/lib/jvm/jdk1.8.0_66/bin/jar" 1</syntaxhighlight>

To be sure, use the following commands to manually select the correct option, in case there is more than one choice:

<syntaxhighlight lang=Bash>sudo update-alternatives --config java
sudo update-alternatives --config javac
sudo update-alternatives --config jar</syntaxhighlight>

If you are still not sure, run them and display the version, making sure it is the one you installed:

<syntaxhighlight lang=Bash>java -version
javac -version</syntaxhighlight>

Sun's JDK is now installed.

====OpenNI====

PCL uses OpenNI and the PrimeSense drivers to get data from devices like Kinect or Xtion. It is optional, but it would not make much sense not to install it, would it? If you are using Ubuntu, add the [[#Precompiled PCL for Ubuntu | PPA]] and install <span style="color:#228B22">''libopenni-dev''</span> and <span style="color:#228B22">''libopenni-sensor-primesense-dev''</span>, which should be done already. Otherwise, fetch the [https://github.com/OpenNI/OpenNI OpenNI] and [https://github.com/PrimeSense/Sensor PrimeSense Sensor] sources from GitHub (download them as .zip, the link is on the right). Extract them, and install the dependencies:

<syntaxhighlight lang=Bash>sudo apt-get install python libusb-1.0-0-dev freeglut3-dev doxygen graphviz</syntaxhighlight>

Go to the directory where you extracted OpenNI (<span style="color:#FF8C00">''OpenNI-master/''</span> for me), and open a terminal in the <span style="color:#FF8C00">''Platform/Linux/CreateRedist/''</span> subdirectory. Issue:

<syntaxhighlight lang=Bash>./RedistMaker</syntaxhighlight>

When it finishes, and if there are no errors (check the [[PCL/OpenNI troubleshooting]] page if you get any), go to <span style="color:#FF8C00">''Platform/Linux/Redist/OpenNI-Bin-Dev-Linux-x64-v1.5.7.10/''</span> (or your equivalent), and install (you must be root):

<syntaxhighlight lang=Bash>sudo ./install.sh</syntaxhighlight>

Now, go to the directory where you extracted the PrimeSense drivers (<span style="color:#FF8C00">''Sensor-master/''</span> for me), and repeat the exact same procedure (go to <span style="color:#FF8C00">''Platform/Linux/CreateRedist/''</span>, issue <span style="color:#228B22">''./RedistMaker''</span>, go to <span style="color:#FF8C00">''Platform/Linux/Redist/Sensor-Bin-Linux-x64-v5.1.6.6/''</span>, issue <span style="color:#228B22">''sudo ./install.sh''</span>). Congratulations, you have now installed OpenNI.

====CUDA====

Like OpenNI, [https://en.wikipedia.org/wiki/CUDA nVidia CUDA] is an optional dependency, that will allow PCL to use your GPU (Graphics Processing Unit, that is, your graphics card) for certain computations. This is mandatory for tools like KinFu (do not bother unless you have at least a series 400 card with 1.5 GB of VRAM).

Some distributions provide packages for CUDA in the repositories. For example, in Ubuntu:

<syntaxhighlight lang=Bash>sudo apt-get install nvidia-cuda-dev nvidia-cuda-toolkit</syntaxhighlight>

If you want to install it manually (which is incompatible with the previous method), go to the [https://developer.nvidia.com/cuda-downloads CUDA downloads] page, which is self-explanatory, and get the small .deb file or the huge .run toolkit/SDK installer file for your system (you should have installed the nVidia drivers already, but the installer will also do it for you if needed).

If you chose the .deb file, install it using the method of your choice, like Gdebi package manager, or through console:

<syntaxhighlight lang=Bash>sudo dpkg -i <package.deb></syntaxhighlight>

The .deb does not contain all CUDA stuff, it just adds their repository to your software lists. Now you must install everything:

<syntaxhighlight lang=Bash>sudo apt-get update
sudo apt-get install cuda</syntaxhighlight>

If, on the other hand, you downloaded the .run, give it execution permissions:

<syntaxhighlight lang=Bash>chmod +x cuda_7.0.28_linux.run</syntaxhighlight>

And install it. You can use the default options, although if you have a working nVidia graphics driver for your card, you may want to say "no" when the installer offers to install it for you:

<syntaxhighlight lang=Bash>sudo ./cuda_7.0.28_linux.run</syntaxhighlight>

The environment variables need to be changed so your system can find CUDA's libraries and binaries. Open <span style="color:#1E90FF">''/etc/ld.so.conf''</span>:

<syntaxhighlight lang=Bash>sudo nano /etc/ld.so.conf</syntaxhighlight>

And append one of these two lines:

<syntaxhighlight lang=Bash>/usr/local/cuda/lib64 # Add this on 64-bit only.
/usr/local/cuda/lib # Add this on 32-bit only.</syntaxhighlight>

Save with <span style="color:#228B22">'''Ctrl+O'''</span> and Enter, exit with <span style="color:#228B22">'''Ctrl+X'''</span>. Reload the cache of the dynamic linker with:

<syntaxhighlight lang=Bash>sudo ldconfig</syntaxhighlight>

Now, append CUDA's bin directory to your PATH. Do this by editing your local <span style="color:#1E90FF">''.bashrc''</span> file:

<syntaxhighlight lang=Bash>nano ~/.bashrc</syntaxhighlight>

And append this line:

<syntaxhighlight lang=Bash>export PATH=$PATH:/usr/local/cuda/bin</syntaxhighlight>

CUDA is now installed.

===Getting the source===

Every dependency is installed. Time to download PCL's source code. First, you must choose whether to install the stable or the experimental branch of PCL. The stable branch is the latest official release and it is guaranteed to work without problems. The experimental branch may give you a compiling error seldomly, but you can find some interesting features that stable users will have to wait some months for. Apart from that, both are built the same way.

To get the stable version, go to the [https://github.com/PointCloudLibrary/pcl/releases downloads] page, get <span style="color:#1E90FF">''pcl-pcl-1.7.2.tar.gz''</span> or whatever the latest release is, and extract it somewhere. For the experimental version, use Git:

<syntaxhighlight lang=Bash>sudo apt-get install git
git clone https://github.com/PointCloudLibrary/pcl PCL-trunk-Source</syntaxhighlight>

The compiled trunk version of PCL will take up more than 8 GB. So make sure you put the source folder in a partition with enough free space!

===Compiling===

Go the the PCL source directory (<span style="color:#FF8C00">''pcl-pcl-1.7.2/''</span> or <span style="color:#FF8C00">''PCL-trunk-Source/''</span>), and create a new subdirectory to keep the build files in:

<syntaxhighlight lang=Bash>mkdir build
cd build</syntaxhighlight>

Now it is time to configure the project using CMake. We will tell it to build in Release (fully optimized, no debug capabilities) mode now, and customize the rest of the options later:

<syntaxhighlight lang=Bash>cmake -DCMAKE_BUILD_TYPE=Release ..</syntaxhighlight>

CMake should be able to find every dependency, thus being able to build every subsystem except for the ones marked as <span style="color:#FF1493">''"Disabled by default"''</span>. If you are happy, you can build now, otherwise let's invoke CMake's curses interface to change a couple of things (mind the final dot):

<syntaxhighlight lang=Bash>ccmake .</syntaxhighlight>


[[Image:Ccmake_GUI_PCL.png|thumb|center|900px|Interface of ''ccmake''.]]


Here you can change the build options. The program usage can be found at the bottom of the screen. Try turning all functionality <span style="color:#FF1493">"ON"</span>. The most important thing, in case you want to use CUDA, is to enable it and give CMake the path to your SDK. Go to the <span style="color:#FF1493">"CUDA_SDK_ROOT_DIR"</span> option and enter the correct path (probably <span style="color:#FF8C00">''/usr/local/cuda/''</span> or similar).

When you are done, press <span style="color:#228B22">'''C'''</span> to configure and <span style="color:#228B22">'''G'''</span> to generate and exit the tool. Sometimes, the options you change can activate previously omitted parameters, or prompt some warning text. Just press <span style="color:#228B22">'''E'''</span> when you are finished reading the message, and keep pressing <span style="color:#228B22">'''C'''</span> until it lets you generate (new parameters will be marked with an asterisk, so you can check them and decide whether or not you want further customization).

If you are done configuring, it is time to build:

<syntaxhighlight lang=Bash>make</syntaxhighlight>

<span style="color:#606060">'''''NOTE: Additionally, you can append the parameter -jX to speed up the compilation, X being the number of cores or processors of your PC, plus one.'''''</span>

Remember that, at any time, you can manually force the project to be reconfigured and built from scratch by emptying the <span style="color:#FF8C00">''build/''</span> directory with:

<syntaxhighlight lang=Bash>rm -rf ./*</syntaxhighlight>

===Installing===

It will take some time to compile PCL (up to a few hours if your PC is not powerful enough). When it is finished, install it system-wide with:

<syntaxhighlight lang=Bash>sudo make install</syntaxhighlight>

And you should reboot and proceed to the next section, to see if your computer now recognizes (and uses) your Kinect device.

=Testing (OpenNI viewer)=

We are going to write a simple example program that will fetch data from the Kinect or Xtion and present it to the user, using the PCL library. It will also allow to save the current frame (as point cloud) to disk. If you feel lazy, you can download it [http://robotica.unileon.es/~victorm/PCL_openni_viewer.tar.gz here], and skip the next two sections. Otherwise, create a new directory anywhere in your hard disk and proceed.

==CMakeLists.txt==

Inside that directory, create a new text file named <span style="color:#1E90FF">''CMakeLists.txt''</span>. PCL-based programs use the CMake build system, too. Open it with any editor and paste the following content:

<syntaxhighlight lang=CMake>cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
 
project(PCL_openni_viewer)
 
find_package(PCL 1.7 REQUIRED)
 
include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})
 
set(PCL_BUILD_TYPE Release)
 
file(GLOB PCL_openni_viewer_SRC
    "src/*.h"
    "src/*.cpp"
)
add_executable(openniViewer ${PCL_openni_viewer_SRC})
 
target_link_libraries (openniViewer ${PCL_LIBRARIES})</syntaxhighlight>

CMake syntax is quite self-explanatory. We ask for a CMake version 2.8 installation, minimum. We declare a new project named <span style="color:#FF1493">"openni_PCL_viewer"</span>. We tell CMake to check for the presence of PCL library development files, version 1.7. If our system can not meet the CMake and PCL version requirement, the process will fail.

Next, we feed the compiler and linker the directories where PCL includes and libraries can be found, and the defined symbols. We tell CMake to use the <span style="color:#FF1493">"Release"</span> build type, which will activate certain optimizations depending on the compiler we use. Other build types are available, like <span style="color:#FF1493">"Debug"</span>, <span style="color:#FF1493">"MinSizeRel"</span>, and <span style="color:#FF1493">"RelWithDebInfo"</span>.

Finally, we create a variable, <span style="color:#FF1493">"opennipclviewer_SRC"</span>, that will store a list of files to be compiled (though we will only have one). We create a new binary to be compiled from these source files, and we link it with the PCL library.

Check the [https://cmake.org/Wiki/CMake_Useful_Variables CMake help] for more interesting options.

==main.cpp==

We told CMake it could find the source files in a <span style="color:#FF8C00">''src/''</span> subdirectory, so let's keep to out word and create it. Then, add a new <span style="color:#1E90FF">''main.cpp''</span> file inside and paste the following lines:

<syntaxhighlight lang=CPP>// Original code by Geoffrey Biggs, taken from the PCL tutorial in
// http://pointclouds.org/documentation/tutorials/pcl_visualizer.php

// Simple OpenNI viewer that also allows to write the current scene to a .pcd
// when pressing SPACE.

#include <pcl/io/openni_grabber.h>
#include <pcl/io/pcd_io.h>
#include <pcl/visualization/cloud_viewer.h>
#include <pcl/console/parse.h>

#include <iostream>

using namespace std;
using namespace pcl;

PointCloud<PointXYZRGBA>::Ptr cloudptr(new PointCloud<PointXYZRGBA>); // A cloud that will store color info.
PointCloud<PointXYZ>::Ptr fallbackCloud(new PointCloud<PointXYZ>);    // A fallback cloud with just depth data.
boost::shared_ptr<visualization::CloudViewer> viewer;                 // Point cloud viewer object.
Grabber* openniGrabber;                                               // OpenNI grabber that takes data from the device.
unsigned int filesSaved = 0;                                          // For the numbering of the clouds saved to disk.
bool saveCloud(false), noColor(false);                                // Program control.

void
printUsage(const char* programName)
{
	cout << "Usage: " << programName << " [options]"
		 << endl
		 << endl
		 << "Options:\n"
		 << endl
		 << "\t<none>     start capturing from an OpenNI device.\n"
		 << "\t-v FILE    visualize the given .pcd file.\n"
		 << "\t-h         shows this help.\n";
}

// This function is called every time the device has new data.
void
grabberCallback(const PointCloud<PointXYZRGBA>::ConstPtr& cloud)
{
	if (! viewer->wasStopped())
		viewer->showCloud(cloud);

	if (saveCloud)
	{
		stringstream stream;
		stream << "inputCloud" << filesSaved << ".pcd";
		string filename = stream.str();
		if (io::savePCDFile(filename, *cloud, true) == 0)
		{
			filesSaved++;
			cout << "Saved " << filename << "." << endl;
		}
		else PCL_ERROR("Problem saving %s.\n", filename.c_str());

		saveCloud = false;
	}
}

// For detecting when SPACE is pressed.
void
keyboardEventOccurred(const visualization::KeyboardEvent& event,
					  void* nothing)
{
	if (event.getKeySym() == "space" && event.keyDown())
		saveCloud = true;
}

// Creates, initializes and returns a new viewer.
boost::shared_ptr<visualization::CloudViewer>
createViewer()
{
	boost::shared_ptr<visualization::CloudViewer> v
	(new visualization::CloudViewer("OpenNI viewer"));
	v->registerKeyboardCallback(keyboardEventOccurred);

	return (v);
}

int
main(int argc, char** argv)
{
	if (console::find_argument(argc, argv, "-h") >= 0)
	{
		printUsage(argv[0]);
		return -1;
	}

	bool justVisualize(false);
	string filename;
	if (console::find_argument(argc, argv, "-v") >= 0)
	{
		if (argc != 3)
		{
			printUsage(argv[0]);
			return -1;
		}

		filename = argv[2];
		justVisualize = true;
	}
	else if (argc != 1)
	{
		printUsage(argv[0]);
		return -1;
	}

	// First mode, open and show a cloud from disk.
	if (justVisualize)
	{
		// Try with color information...
		try
		{
			io::loadPCDFile<PointXYZRGBA>(filename.c_str(), *cloudptr);
		}
		catch (PCLException e1)
		{
			try
			{
				// ...and if it fails, fall back to just depth.
				io::loadPCDFile<PointXYZ>(filename.c_str(), *fallbackCloud);
			}
			catch (PCLException e2)
			{
				return -1;
			}

			noColor = true;
		}

		cout << "Loaded " << filename << "." << endl;
		if (noColor)
			cout << "This cloud has no RGBA color information present." << endl;
		else cout << "This cloud has RGBA color information present." << endl;
	}
	// Second mode, start fetching and displaying frames from the device.
	else
	{
		openniGrabber = new OpenNIGrabber();
		if (openniGrabber == 0)
			return -1;
		boost::function<void (const PointCloud<PointXYZRGBA>::ConstPtr&)> f =
			boost::bind(&grabberCallback, _1);
		openniGrabber->registerCallback(f);
	}

	viewer = createViewer();

	if (justVisualize)
	{
		if (noColor)
			viewer->showCloud(fallbackCloud);
		else viewer->showCloud(cloudptr);
	}
	else openniGrabber->start();

	// Main loop.
	while (! viewer->wasStopped())
		boost::this_thread::sleep(boost::posix_time::seconds(1));

	if (! justVisualize)
		openniGrabber->stop();
}</syntaxhighlight>

Save and close.

==Compiling==

Follow the same steps you used to build PCL. That is, create a new <span style="color:#FF8C00">''build/''</span> subdirectory next to the <span style="color:#FF8C00">''src/''</span> one. Open a terminal there and issue:

<syntaxhighlight lang=Bash>cmake -DCMAKE_BUILD_TYPE=Release ..
make</syntaxhighlight>

==Executing==

Still from the same terminal, run the compiled example program:

<syntaxhighlight lang=Bash>./openniViewer</syntaxhighlight>

After some seconds, the main window will appear and the application will start grabbing frames from the device. You can inspect the current point cloud using the mouse, holding the left button to rotate, the right one (or the mouse wheel) to zoom, and the middle one to pan the camera around. At first, you may see only a black screen, or some big colored axes, but no cloud. Try zooming out, to see the whole scene. Another useful key is <span style="color:#228B22">'''R'''</span>, which will reset the camera parameters when pressed. Use it whenever you notice that zooming has gotten slow after some camera movement, or if you still can not see the cloud. See the [http://pointclouds.org/documentation/tutorials/pcl_visualizer.php PCLVisualizer] tutorial for additional controls and features.

Whenever you feel ready, press the <span style="color:#228B22">'''SPACE'''</span> key. The program will pause for a fraction of a second and the output <span style="color:#FF1493">''"Saved inputCloud0.pcd."''</span> will appear on the console. Check the current folder to see that file <span style="color:#1E90FF">''inputCloud0.pcd''</span> has indeed been written. You can now close the program with <span style="color:#228B22">'''Q'''</span> or <span style="color:#228B22">'''Alt+F4'''</span>.

Next, run it again giving the following parameter:

<syntaxhighlight lang=Bash>./openniViewer -v inputCloud0.pcd</syntaxhighlight>

This will tell the program not to take data from the device, but from the saved point cloud file instead. After it loads, you will realize that you are presented the same scene you saved to disk.

<span style="color:#606060">'''''NOTE: PCD data is saved relative to the sensor. No matter how much you have manipulated the view, it will reset to default when you load the file.'''''</span>

=Conclusion=

At this point, your Kinect device should be working and getting depth data for you. There is a collection of excellent [http://pointclouds.org/documentation/tutorials/ tutorials] for PCL in the official webpage. I encourage you to finish them all before proceeding your experiments with the Kinect sensor. You can also find a good introduction/tutorial at the PCL library [http://www.pointclouds.org/assets/uploads/PCL-IAS13_Introduction.pdf here].

If you use an ASUS Xtion PRO device instead, you should have gotten everything to work without problems or additional steps (except maybe for [[PCL/OpenNI_troubleshooting#ASUS_Xtion_PRO | this one]]).


----
----

Go to root: [[PhD-3D-Object-Tracking]]

Links to articles:

[[PCL/OpenNI tutorial 0: The very basics]]

[[PCL/OpenNI tutorial 1: Installing and testing]]

[[PCL/OpenNI tutorial 2: Cloud processing (basic)]]

[[PCL/OpenNI tutorial 3: Cloud processing (advanced)]]

[[PCL/OpenNI tutorial 4: 3D object recognition (descriptors)]]

[[PCL/OpenNI tutorial 5: 3D object recognition (pipeline)]]

[[PCL/OpenNI troubleshooting]]