PCL/OpenNI tutorial 4: 3D object recognition (descriptors) - Revision history

Victorm at 16:59, 5 November 2015

2015-11-05T16:59:49Z

Victorm at 16:42, 5 November 2015

2015-11-05T16:42:05Z

Victorm at 11:12, 2 November 2015

2015-11-02T11:12:00Z

Victorm at 11:08, 2 November 2015

2015-11-02T11:08:04Z

Victorm at 23:12, 1 November 2015

2015-11-01T23:12:44Z

Victorm at 19:50, 1 November 2015

2015-11-01T19:50:07Z

Victorm at 19:46, 1 November 2015

2015-11-01T19:46:32Z

Victorm at 19:39, 1 November 2015

2015-11-01T19:39:43Z

Victorm at 19:33, 1 November 2015

2015-11-01T19:33:25Z

Victorm at 08:38, 10 April 2015

2015-04-10T08:38:32Z

@@ Line 1,686: / Line 1,686: @@
 	// GRSD estimation object.
-	GRSDEstimation<PointXYZ, Normal, GRSDSignature21> grsd;
+	pcl::GRSDEstimation<pcl::PointXYZ, pcl::Normal, pcl::GRSDSignature21> grsd;
 	grsd.setInputCloud(cloud);
 	grsd.setInputNormals(normals);

@@ Line 348: / Line 348: @@
 	// RSD estimation object.
-	RSDEstimation<pcl::PointXYZ, pcl::Normal, pcl::PrincipalRadiiRSD> rsd;
+	pcl::RSDEstimation<pcl::PointXYZ, pcl::Normal, pcl::PrincipalRadiiRSD> rsd;
 	rsd.setInputCloud(cloud);
 	rsd.setInputNormals(normals);

@@ Line 1,264: / Line 1,264: @@
 	rops.compute(*descriptors);
 }</syntaxhighlight>
 <div style="background-color: #F8F8F8; border-style: dotted;">

@@ Line 1,264: / Line 1,264: @@
 	rops.compute(*descriptors);
 }</syntaxhighlight>
 <div style="background-color: #F8F8F8; border-style: dotted;">

@@ Line 27: / Line 27: @@
 There are many 3D descriptors implemented into PCL. Each one has its own method for computing unique values for a point. Some use the difference between the angles of the normals of the point and its neighbors, for example. Others use the distances between the points. Because of this, some are inherently better or worse for certain purposes. A given descriptor may be scale invariant, and another one may be better with occlusions and partial views of objects. Which one you choose depends on what you want to do.
-After calculating the necessary values, an additional step is performed to reduce the descriptor size: the result is binned into an [http://en.wikipedia.org/wiki/Histogram histogram]. To do this, the value range of each variable that makes up the descriptor is divided into ''n'' subdivisions, and the number of occurrences in each one is counted. Try to imagine a descriptor that computes a single variable, that ranges from 1 to 100. We choose to create 10 bins for it, so the first bin would gather all occurrences between 1 and 10, the second from 11 to 20, and so on. We look at the value of the variable for the first point-neighbor pair, and it is 27, so we increment the value of the third bin by 1. We keep doing this until we get a final histogram for that keypoint. The bin size must be carefully chosen depending on how descriptive that variable is (the variables do not have to share the same number of bins, and also the bins do not have to be of the same size; if for example most values from the previous example fell in the 50-100 range then it would be sensible to have more bins of smaller size in that range).
+After calculating the necessary values, an additional step is performed to reduce the descriptor size: the result is binned into an [https://en.wikipedia.org/wiki/Histogram histogram]. To do this, the value range of each variable that makes up the descriptor is divided into ''n'' subdivisions, and the number of occurrences in each one is counted. Try to imagine a descriptor that computes a single variable, that ranges from 1 to 100. We choose to create 10 bins for it, so the first bin would gather all occurrences between 1 and 10, the second from 11 to 20, and so on. We look at the value of the variable for the first point-neighbor pair, and it is 27, so we increment the value of the third bin by 1. We keep doing this until we get a final histogram for that keypoint. The bin size must be carefully chosen depending on how descriptive that variable is (the variables do not have to share the same number of bins, and also the bins do not have to be of the same size; if for example most values from the previous example fell in the 50-100 range then it would be sensible to have more bins of smaller size in that range).
 Descriptors can be classified in two main categories: global and local. The process for computing and using each one (recognition pipeline) is different, so each will be explained in its own section in this article.
@@ Line 37: / Line 37: @@
 ** [http://www.pointclouds.org/assets/icra2013/pcl_features_icra13.pdf How does a good feature look like?]
 * '''Publication''':
-** [http://www.inf.ethz.ch/personal/zeislb/publications/aldoma_2012jram_PCLTutorial.pdf Point Cloud Library: Three-Dimensional Object Recognition and 6 DoF Pose Estimation] (Aitor Aldoma et al., 2012)
+** [https://www.inf.ethz.ch/personal/zeislb/publications/aldoma_2012jram_PCLTutorial.pdf Point Cloud Library: Three-Dimensional Object Recognition and 6 DoF Pose Estimation] (Aitor Aldoma et al., 2012)
 </div>
@@ Line 158: / Line 158: @@
 PFH stands for Point Feature Histogram. It is one of the most important descriptors offered by PCL and the basis of others such as FPFH. The PFH tries to capture information of the geometry surrounding the point by analyzing the difference between the directions of the normals in the vicinity (and because of this, an imprecise normal estimation may produce low-quality descriptors).
-First, the algorithm pairs all points in the vicinity (not just the chosen keypoint with its neighbors, but also the neighbors with themselves). Then, for each pair, a [http://en.wikipedia.org/wiki/Darboux_frame fixed coordinate frame] is computed from their normals. With this frame, the difference between the normals can be encoded with 3 angular variables. These variables, together with the euclidean distance between the points, are saved, and then binned to an histogram when all pairs have been computed. The final descriptor is the concatenation of the histograms of each variable (4 in total).
+First, the algorithm pairs all points in the vicinity (not just the chosen keypoint with its neighbors, but also the neighbors with themselves). Then, for each pair, a [https://en.wikipedia.org/wiki/Darboux_frame fixed coordinate frame] is computed from their normals. With this frame, the difference between the normals can be encoded with 3 angular variables. These variables, together with the euclidean distance between the points, are saved, and then binned to an histogram when all pairs have been computed. The final descriptor is the concatenation of the histograms of each variable (4 in total).
@@ Line 232: / Line 232: @@
 ===FPFH===
-PFH gives accurate results, but it has a drawback: it is too computationally expensive to perform at real time. For a cloud of ''n'' keypoints with ''k'' neighbors considered, it has a [http://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations complexity] of ''O(nk<sup>2</sup>)''. Because of this, a derived descriptor was created, named FPFH (Fast Point Feature Histogram).
+PFH gives accurate results, but it has a drawback: it is too computationally expensive to perform at real time. For a cloud of ''n'' keypoints with ''k'' neighbors considered, it has a [https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations complexity] of ''O(nk<sup>2</sup>)''. Because of this, a derived descriptor was created, named FPFH (Fast Point Feature Histogram).
 The FPFH considers only the direct connections between the current keypoint and its neighbors, removing additional links between neighbors. This takes the complexity down to ''O(nk)''. Because of this, the resulting histogram is referred to as SPFH (Simplified Point Feature Histogram). The reference frame and the angular variables are computed as always.
@@ Line 285: / Line 285: @@
 }</syntaxhighlight>
-An additional implementation of the FPFH estimation that takes advantage of multithreaded optimizations (with the [http://en.wikipedia.org/wiki/OpenMP OpenMP API]) is available in the class <span style="color:#FF1493">"FPFHEstimationOMP"</span>. Its interface is identical to the standard unoptimized implementation. Using it will result in a big performance boost on multi-core systems, meaning faster computation times. Remember to include the header <span style="color:#FF1493">"fpfh_omp.h"</span> instead.
+An additional implementation of the FPFH estimation that takes advantage of multithreaded optimizations (with the [https://en.wikipedia.org/wiki/OpenMP OpenMP API]) is available in the class <span style="color:#FF1493">"FPFHEstimationOMP"</span>. Its interface is identical to the standard unoptimized implementation. Using it will result in a big performance boost on multi-core systems, meaning faster computation times. Remember to include the header <span style="color:#FF1493">"fpfh_omp.h"</span> instead.
 <div style="background-color: #F8F8F8; border-style: dotted;">
@@ Line 646: / Line 646: @@
 ==RIFT==
-The Rotation-Invariant Feature Transform, like the spin image, takes some concepts from 2D features, in this case from the Scale-Invariant Feature Transform ([http://en.wikipedia.org/wiki/Scale-invariant_feature_transform SIFT]). It is the only descriptor seen here that requires intensity information in order to compute it (it can be obtained from the RGB color values). This means, of course, that you will not be able to use RIFT with standard XYZ clouds, you also need the texture.
+The Rotation-Invariant Feature Transform, like the spin image, takes some concepts from 2D features, in this case from the Scale-Invariant Feature Transform ([https://en.wikipedia.org/wiki/Scale-invariant_feature_transform SIFT]). It is the only descriptor seen here that requires intensity information in order to compute it (it can be obtained from the RGB color values). This means, of course, that you will not be able to use RIFT with standard XYZ clouds, you also need the texture.
 In the first step, a circular patch (with the given radius) is fitted on the surface the point lies on. This patch is divided into concentric rings, according to the chosen distance bin size. Then, an histogram is populated with all the point's neighbors lying inside a sphere centered at that point and with the mentioned radius. The distance and the orientation of the intensity gradient at each point are considered. To make it rotation invariant, the angle between the gradient orientation and the vector pointing outward from the center of the patch is measured.
@@ Line 739: / Line 739: @@
 ==NARF==
-The Normal Aligned Radial Feature is the only descriptor here that does not take a point cloud as input. Instead, it works with range images. A range image is a common RGB image in which the distance to the point that corresponds to a certain pixel is encoded as a color value in the [http://en.wikipedia.org/wiki/Visible_spectrum visible light spectrum]: the points that are closer to the camera would be violet, while the points near the maximum sensor range would be red.
+The Normal Aligned Radial Feature is the only descriptor here that does not take a point cloud as input. Instead, it works with range images. A range image is a common RGB image in which the distance to the point that corresponds to a certain pixel is encoded as a color value in the [https://en.wikipedia.org/wiki/Visible_spectrum visible light spectrum]: the points that are closer to the camera would be violet, while the points near the maximum sensor range would be red.
 NARF also requires us to find suitable keypoints to compute the descriptor for. NARF keypoints are located near an object's corners, and this also requires to find the borders (transitions from foreground to background), which are trivial to find with a range image. Because of this lengthy pipeline, I will describe the whole process in different sections.
@@ Line 1,578: / Line 1,578: @@
 As you may have guessed, GFPFH stands for Global Fast Point Feature Histogram, the global version of the FPFH descriptor. GFPFH was designed for the task of helping a robot navigate its environment, having some context of the objects around it.
-The first step before being able to compute the descriptor is surface categorization. A set of logical primitives (the classes, or categories) is created, which depends on the type of objects we expect the robot to find on the scene. For example, if we know there will be a coffee mug, we create three: one for the handle, and the other two for the outer and inner faces. Then, FPFH descriptors are computed, and everything is fed to a [http://en.wikipedia.org/wiki/Conditional_random_field Conditional Random Field] (CRF) algorithm. The CRF will label each surface with one of the previous categories, so we end up with a cloud where each point has been classified depending of the type of object (or object's region) it belongs to.
+The first step before being able to compute the descriptor is surface categorization. A set of logical primitives (the classes, or categories) is created, which depends on the type of objects we expect the robot to find on the scene. For example, if we know there will be a coffee mug, we create three: one for the handle, and the other two for the outer and inner faces. Then, FPFH descriptors are computed, and everything is fed to a [https://en.wikipedia.org/wiki/Conditional_random_field Conditional Random Field] (CRF) algorithm. The CRF will label each surface with one of the previous categories, so we end up with a cloud where each point has been classified depending of the type of object (or object's region) it belongs to.

@@ Line 1,713: / Line 1,713: @@
 =Saving and loading=
-You can save a descriptor to a file just [[PCL/OpenNI_tutorial_2:_Cloud_processing_(basic)#Writing_to_file|like with any other cloud type]]. One caveat, though. If you are using a descriptor that has its own custom type, like <span style="color:#FF1493">"PFHSignature125"</span>, everything will be OK. But with descriptors that do not (where you have to use <span style="color:#FF1493">"pcl::Histogram<>"</span>), you will get this error: <span style="color:#FF1493">''"POINT_TYPE_NOT_PROPERLY_REGISTERED"''</span>. In order to save or load from a file, PCL's IO functions need to know about the number, type and size of the fields. To solve this, you will have to properly register a new point type for your descriptor. For example:
+You can save a descriptor to a file just [[PCL/OpenNI_tutorial_2:_Cloud_processing_(basic)#Writing_to_file|like with any other cloud type]]. One caveat, though. If you are using a descriptor that has its own custom type, like <span style="color:#FF1493">"PFHSignature125"</span>, everything will be OK. But with descriptors that do not (where you have to use <span style="color:#FF1493">"pcl::Histogram<>"</span>), you will get this error: <span style="color:#FF1493">''"POINT_TYPE_NOT_PROPERLY_REGISTERED"''</span>. In order to save or load from a file, PCL's IO functions need to know about the number, type and size of the fields. To solve this, you will have to properly register a new point type for your descriptor. For example, this will work for the [[PCL/OpenNI_tutorial_4:_3D_object_recognition_%28descriptors%29#RoPS|RoPS]] descriptor example we saw earlier:
 <syntaxhighlight lang=CPP>POINT_CLOUD_REGISTER_POINT_STRUCT(ROPS135,

@@ Line 137: / Line 137: @@
 † Values marked with an asterisk (*) indicate that the descriptor's size depends on some parameter(s), and the one given is for the default values.
-†† Descriptors without their own custom PointType use the generic <span style="color:#FF1493">"pcl::Histogram<>"</span> type.
+†† Descriptors without their own custom PointType use the generic <span style="color:#FF1493">"pcl::Histogram<>"</span> type. See [[PCL/OpenNI_tutorial_4:_3D_object_recognition_%28descriptors%29#Saving_and_loading|Saving and loading]].
 Optionally, you can download a document with a less simple version of the table. Page format is A4, landscape:
@@ Line 1,713: / Line 1,713: @@
 =Saving and loading=
-You can save a descriptor to a file just [[PCL/OpenNI_tutorial_2:_Cloud_processing_(basic)#Writing_to_file|like with any other cloud type]]. One caveat, though. If you are using a descriptor that has its own custom type, like <span style="color:#FF1493">"PFHSignature125"</span>, everything will be OK. But with descriptors that do not (where you have to use <span style="color:#FF1493">"pcl::Histogram<>"</span>), you will get this error: <span style="color:#FF1493">''"POINT_TYPE_NOT_PROPERLY_REGISTERED"''</span>.
+You can save a descriptor to a file just [[PCL/OpenNI_tutorial_2:_Cloud_processing_(basic)#Writing_to_file|like with any other cloud type]]. One caveat, though. If you are using a descriptor that has its own custom type, like <span style="color:#FF1493">"PFHSignature125"</span>, everything will be OK. But with descriptors that do not (where you have to use <span style="color:#FF1493">"pcl::Histogram<>"</span>), you will get this error: <span style="color:#FF1493">''"POINT_TYPE_NOT_PROPERLY_REGISTERED"''</span>. In order to save or load from a file, PCL's IO functions need to know about the number, type and size of the fields. To solve this, you will have to properly register a new point type for your descriptor. For example:
 =Visualization=

@@ Line 1,710: / Line 1,710: @@
 </div>
 =Visualization=
@@ Line 1,838: / Line 1,842: @@
 ==PCL Viewer==
-This program, included with PCL, will also let you open and visualize a saved descriptor. Internally, it uses [[PCL/OpenNI_tutorial_4:_3D_object_recognition_%28descriptors%29#PCLPlotter|PCLPlotter]]. Remember that you can save a descriptor to a file just [[PCL/OpenNI_tutorial_2:_Cloud_processing_(basic)#Writing_to_file|like with any other cloud type]]. Once done, invoke the viewer from the command line:
+This program, included with PCL, will also let you open and visualize a saved descriptor. Internally, it uses [[PCL/OpenNI_tutorial_4:_3D_object_recognition_%28descriptors%29#PCLPlotter|PCLPlotter]]. You can invoke the viewer from the command line like this, so it comes handy:
 <syntaxhighlight lang=Bash>pcl_viewer <descriptor_file></syntaxhighlight>

@@ Line 50: / Line 50: @@
 ! scope="col"| Name
 ! scope="col"| Type
-! scope="col"| Size
+! scope="col"| Size<sup>†</sup>
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#PFH|PFH]] (Point Feature Histogram)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 125
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#FPFH|FPFH]] (Fast Point Feature Histogram)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 33
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#RSD|RSD]] (Radius-Based Surface Descriptor)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 289
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#3DSC|3DSC]] (3D Shape Context)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 1980
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#USC|USC]] (Unique Shape Context)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 1960
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#SHOT|SHOT]] (Signatures of Histograms of Orientations)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 352
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#Spin_image|Spin image]]
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 153*
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#RIFT|RIFT]] (Rotation-Invariant Feature Transform)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 32*
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#NARF|NARF]] (Normal Aligned Radial Feature)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 36
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#RoPS|RoPS]] (Rotational Projection Statistics)
-| style="color: red;" | Local
+| style="color: sienna;" | Local
 | 135*
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#VFH|VFH]] (Viewpoint Feature Histogram)
-| style="color: green;" | Global
+| style="color: darkcyan;" | Global
 | 308
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#CVFH|CVFH]] (Clustered Viewpoint Feature Histogram)
-| style="color: green;" | Global
+| style="color: darkcyan;" | Global
 | 308
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#OUR-CVFH|OUR-CVFH]] (Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram)
-| style="color: green;" | Global
+| style="color: darkcyan;" | Global
 | 308
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#ESF|ESF]] (Ensemble of Shape Functions)
-| style="color: green;" | Global
+| style="color: darkcyan;" | Global
 | 640
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#GFPFH|GFPFH]] (Global Fast Point Feature Histogram)
-| style="color: green;" | Global
+| style="color: darkcyan;" | Global
 | 16
 |- style="vertical-align:middle;"
 | [[PCL/OpenNI_tutorial_4:_3D_object_recognition_(descriptors)#GRSD|GRSD]] (Global Radius-Based Surface Descriptor)
-| style="color: green;" | Global
+| style="color: darkcyan;" | Global
 | 21
 |}
-Values marked with an asterisk (*) indicate that the descriptor's size depends on some parameter(s), and the one given is for the default values.
+† Values marked with an asterisk (*) indicate that the descriptor's size depends on some parameter(s), and the one given is for the default values.
 Optionally, you can download a document with a less simple version of the table. Page format is A4, landscape:

← Older revision		Revision as of 08:38, 10 April 2015
Line 44:		Line 44:

	The following table will give you a hint of how many descriptors are there in PCL, and some of their features:		The following table will give you a hint of how many descriptors are there in PCL, and some of their features:
−

	{\| class="wikitable sortable" style="margin: auto; text-align:center" cellpadding="15"		{\| class="wikitable sortable" style="margin: auto; text-align:center" cellpadding="15"