PCL/OpenNI tutorial 3: Cloud processing (advanced) - Revision history

Victorm at 23:09, 1 November 2015

2015-11-01T23:09:09Z

Victorm at 17:29, 6 April 2015

2015-04-06T17:29:11Z

Victorm at 08:38, 11 March 2015

2015-03-11T08:38:07Z

Victorm at 11:35, 23 February 2015

2015-02-23T11:35:58Z

Victorm at 11:22, 23 February 2015

2015-02-23T11:22:49Z

Victorm at 08:54, 17 February 2015

2015-02-17T08:54:12Z

Victorm at 08:50, 17 February 2015

2015-02-17T08:50:06Z

Victorm at 08:49, 17 February 2015

2015-02-17T08:49:01Z

Victorm at 08:34, 17 February 2015

2015-02-17T08:34:51Z

Victorm at 10:21, 28 January 2015

2015-01-28T10:21:18Z

@@ Line 23: / Line 23: @@
 The previous diagram illustrates how the procedure is done. There are mainly two different methods to choose from:
-* '''ICP registration''': ICP stands for [http://en.wikipedia.org/wiki/Iterative_closest_point Iterative Closest Point]. It is an algorithm that will find the best transformation that minimizes the distance from the source point cloud to the target one. The problem is that it will do it by associating every point of the source cloud to its "twin" in the target cloud in a linear way, so it can be considered a brute force method. It the clouds are too big, the algorithm will take its time to finish, so try downsampling them first.
+* '''ICP registration''': ICP stands for [https://en.wikipedia.org/wiki/Iterative_closest_point Iterative Closest Point]. It is an algorithm that will find the best transformation that minimizes the distance from the source point cloud to the target one. The problem is that it will do it by associating every point of the source cloud to its "twin" in the target cloud in a linear way, so it can be considered a brute force method. It the clouds are too big, the algorithm will take its time to finish, so try downsampling them first.
 * '''Feature-based registration''': the algorithm finds a set of keypoints in each cloud, computes a local descriptor for each (we explain what local descriptors are in [[PCL/OpenNI_tutorial_4:_3D_object_recognition_%28descriptors%29|the next tutorial]]; they are like individual signatures of points), and then performs a search to see if the clouds have keypoints in common. If at least 3 correspondences are found, a transformation can be computed. For accurate results, several correspondences must be found. This method is (or should be) faster than the first, because matching is only done for the keypoints, not the whole cloud.
@@ Line 111: / Line 111: @@
 =Model fitting (RANSAC)=
-Given a set of data, it is possible to determine if a part of it fits a certain mathematical model, using an iterative method known as [http://en.wikipedia.org/wiki/RANSAC RANSAC (Random Sample Consensus)]. RANSAC works under the assumption that the data contains ''inliers'' (data can can be adjusted to the model, even with a little noise) and ''outliers'' (data that does not fit the model at all). The algoritm is non-deterministic: every iteration, the accuracy of the result improves. To be precise, the probability of the result being the correct one increases, though it will never be of 100%.
+Given a set of data, it is possible to determine if a part of it fits a certain mathematical model, using an iterative method known as [https://en.wikipedia.org/wiki/RANSAC RANSAC (Random Sample Consensus)]. RANSAC works under the assumption that the data contains ''inliers'' (data can can be adjusted to the model, even with a little noise) and ''outliers'' (data that does not fit the model at all). The algoritm is non-deterministic: every iteration, the accuracy of the result improves. To be precise, the probability of the result being the correct one increases, though it will never be of 100%.
@@ Line 284: / Line 284: @@
 ==Euclidean==
-Euclidean segmentation is the simplest of all. It checks the distance between two points. If it is less than a threshold, both are considered to belong in the same cluster. It works like a [http://en.wikipedia.org/wiki/Flood_fill flood fill] algorithm: a point in the cloud is "marked" as "chosen" for the cluster. Then, it spreads like a virus to all other points that are near enough, and from those to even more points, until none new can be added. Then, a new cluster is initialized, and the procedure starts again with the remaining unmarked points.
+Euclidean segmentation is the simplest of all. It checks the distance between two points. If it is less than a threshold, both are considered to belong in the same cluster. It works like a [https://en.wikipedia.org/wiki/Flood_fill flood fill] algorithm: a point in the cloud is "marked" as "chosen" for the cluster. Then, it spreads like a virus to all other points that are near enough, and from those to even more points, until none new can be added. Then, a new cluster is initialized, and the procedure starts again with the remaining unmarked points.
 In PCL, Euclidean segmentation is done as follows:
@@ Line 838: / Line 838: @@
 =Retrieving the hull=
-For a given set of points, PCL can compute the external hull of the shape, using the [http://www.qhull.org/ QHull library]. The hull could be defined as the points that conform the outermost boundary of the set, like a shell showing the volume. There are two types of hull that you can calculate, the [http://en.wikipedia.org/wiki/Convex_hull convex] and the concave one. A convex hull can not have "holes", that is, the segment that connects any pair of points can never cross "empty" space (not inside the hull). A concave hull, on the other hand, usually takes less area, like you can see in the following images:
+For a given set of points, PCL can compute the external hull of the shape, using the [http://www.qhull.org/ QHull library]. The hull could be defined as the points that conform the outermost boundary of the set, like a shell showing the volume. There are two types of hull that you can calculate, the [https://en.wikipedia.org/wiki/Convex_hull convex] and the concave one. A convex hull can not have "holes", that is, the segment that connects any pair of points can never cross "empty" space (not inside the hull). A concave hull, on the other hand, usually takes less area, like you can see in the following images:
@@ Line 1,024: / Line 1,024: @@
 ==Triangulation==
-Triangulation is a a way of estimating the surface captured by a point cloud, by connecting points with each other, ending up with a continous polygon mesh (three-sided polygons, that is, triangles). After you retrieve this surface mesh, you can for example export it to a format that most 3D modelling tools will understand, like [http://en.wikipedia.org/wiki/VTK VTK (Visualization Toolkit)], which can be opened in Blender or 3ds Max. PCL can also convert from VTK to [http://en.wikipedia.org/wiki/PLY_%28file_format%29 PLY] or [http://en.wikipedia.org/wiki/Wavefront_.obj_file OBJ].
+Triangulation is a a way of estimating the surface captured by a point cloud, by connecting points with each other, ending up with a continous polygon mesh (three-sided polygons, that is, triangles). After you retrieve this surface mesh, you can for example export it to a format that most 3D modelling tools will understand, like [https://en.wikipedia.org/wiki/VTK VTK (Visualization Toolkit)], which can be opened in Blender or 3ds Max. PCL can also convert from VTK to [https://en.wikipedia.org/wiki/PLY_%28file_format%29 PLY] or [https://en.wikipedia.org/wiki/Wavefront_.obj_file OBJ].
 The following code uses PCL's implementation of a greedy triangulation algorithm that works with local 2D projections. For every point, it looks "down" along the normal, and connects neighboring points. For more information, specially regarding the parameters, check the API or the original PCL tutorial:

@@ Line 1,036: / Line 1,036: @@
 main(int argc, char** argv)
+{
-	// Object for storing the point clouds.
+	// Object for storing the point cloud.
 	pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);
 	// Object for storing the normals.

@@ Line 24: / Line 24: @@
 * '''ICP registration''': ICP stands for [http://en.wikipedia.org/wiki/Iterative_closest_point Iterative Closest Point]. It is an algorithm that will find the best transformation that minimizes the distance from the source point cloud to the target one. The problem is that it will do it by associating every point of the source cloud to its "twin" in the target cloud in a linear way, so it can be considered a brute force method. It the clouds are too big, the algorithm will take its time to finish, so try downsampling them first.
-* '''Feature-based registration''': the algorithm finds a set of keypoints in each cloud, computes a descriptor for each (we explain what descriptors are in [[PCL/OpenNI_tutorial_4:_3D_object_recognition_%28descriptors%29|the next tutorial]]; they are like individual signatures of points), and then performs a search to see if the clouds have keypoints in common. If at least 3 correspondences are found, a transformation can be computed. For accurate results, several correspondences must be found. This method is (or should be) faster than the first, because matching is only done for the keypoints, not the whole cloud.
+* '''Feature-based registration''': the algorithm finds a set of keypoints in each cloud, computes a local descriptor for each (we explain what local descriptors are in [[PCL/OpenNI_tutorial_4:_3D_object_recognition_%28descriptors%29|the next tutorial]]; they are like individual signatures of points), and then performs a search to see if the clouds have keypoints in common. If at least 3 correspondences are found, a transformation can be computed. For accurate results, several correspondences must be found. This method is (or should be) faster than the first, because matching is only done for the keypoints, not the whole cloud.
 ICP will probably fail if the difference between the clouds is too big. Usually, you will use features first to perform an initial rough alignment of the clouds, and then use ICP to refine it with precision. Feature-based registration is basically identical to the [[PCL/OpenNI_tutorial_5:_3D_object_recognition_%28pipeline%29|3D object recognition problem]], so we will not look into it here. Features, descriptors and recognition will be dealt with in the next tutorials.

@@ Line 705: / Line 705: @@
 ==RANSAC==
-As we saw in a previous section, PCL offers an implementation of the RANSAC algorithm to fit a set of points to a model. An object exists to perform easy segmentation of the points retrieved by the algorithm.
+As we saw in a previous section, PCL offers an implementation of the RANSAC algorithm to fit a set of points to a model. It will search the cloud and find a list of points that support the chosen model (plane, sphere...), and also compute the coefficients of the model. An object exists to perform easy segmentation of the points retrieved by the algorithm.
 ===Plane model===

← Older revision		Revision as of 11:22, 23 February 2015
Line 1,007:		Line 1,007:

	=Reconstruction=		=Reconstruction=
		+
		+	To learn about how to use PCL to build a smooth, parametric surface representation from a point cloud, you can read the following:
		+
		+	<div style="background-color: #F8F8F8; border-style: dotted;">
		+	* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/bspline_fitting.php Fitting trimmed B-splines to unordered point clouds]
		+	</div>
		+

	==Triangulation==		==Triangulation==

@@ Line 164: / Line 164: @@
 * [http://docs.pointclouds.org/trunk/classpcl_1_1_sample_consensus_model_stick.html pcl::SampleConsensusModelStick]: A stick (a line with user-given minimum/maximum width).
-Also, the RANSAC class offers some functions to configure its behavior. For example, with <span style="color:#FF1493">"setMaxIterations()"</span> you can specify the maximum number of iterations the algoritm will run for.
+Also, the RANSAC class offers some functions to configure its behavior. For example, with <span style="color:#FF1493">"setMaxIterations()"</span> you can specify the maximum number of iterations the algoritm will run for, instead of using a distance threshold (if you do not set a stop condition, RANSAC will just run forever, getting a slightly better result each time).
 <div style="background-color: #F8F8F8; border-style: dotted;">

@@ Line 26: / Line 26: @@
 * '''Feature-based registration''': the algorithm finds a set of keypoints in each cloud, computes a descriptor for each (we mentioned what descriptors were in the previous tutorial; they are like individual signatures of points), and then performs a search to see if the clouds have keypoints in common. If true, a transformation is computed. For accurate results, enough correspondences must be found. This method is (or should be) faster than the first, because matching is only done for the keypoints, not the whole cloud.
-The second method is basically identical to the 3D object recognition problem, so we will not look into it here. Features, descriptors and recognition will be dealt with in the next tutorials.
+ICP will probably fail if the difference between the clouds is too big. Usually, you will use features first to perform an initial rough alignment of the clouds, and then use ICP to refine it with precision. Feature-based registration is basically identical to the [[PCL/OpenNI_tutorial_5:_3D_object_recognition_%28pipeline%29|3D object recognition problem]], so we will not look into it here. Features, descriptors and recognition will be dealt with in the next tutorials.
 <div style="background-color: #F8F8F8; border-style: dotted;">
@@ Line 709: / Line 709: @@
 ===Plane model===
-The following code lets you segment all planar surfaces from a point cloud. This is very important, because you will be able to detect things like the floor, ceiling, walls, or a table in a scene. If you know that the object you are looking for is sitting on a table, you can discard all points that are not supported by it, with the associated performance gain. Take a look at the [http://docs.pointclouds.org/trunk/classpcl_1_1_extract_polygonal_prism_data.html pcl::ExtractPolygonalPrismData] class, which is designed for this. After giving the coefficients of the plane and computing the convex hull, you can then create a 3D prism by extruding the hull a given height, and extract all points that lie inside it.
+The following code lets you segment all planar surfaces from a point cloud. This is very important, because you will be able to detect things like the floor, ceiling, walls, or a table in a scene. If you know that the object you are looking for is sitting on a table, you can discard all points that are not supported by it, with the associated performance gain. Take a look at the [[PCL/OpenNI_tutorial_5:_3D_object_recognition_%28pipeline%29#Segmentation|pcl::ExtractPolygonalPrismData]] class, which is designed for this. After giving the coefficients of the plane and computing the convex hull, you can then create a 3D prism by extruding the hull a given height, and extract all points that lie inside it.
 <syntaxhighlight lang=CPP>#include <pcl/io/pcd_io.h>

@@ Line 15: / Line 15: @@
 The best results are achieved with clouds that are very similar, so you should try to minimize the transformations between them. Meaning, do not run like crazy with a Kinect in your hands, grabbing frames, and expect PCL to match the clouds. It is better to move the sensor gingerly and at steady intervals. If you use a robotic arm or a rotating tabletop with precise angles, even better.
-Registration is a very useful technique because it lets you retrieve full, complete and continous [http://pointclouds.org/documentation/tutorials/in_hand_scanner.php#in-hand-scanner models] of a scene or object. It is also expensive for the CPU. Optimizations have been developed that allow to do cloud matching in real time with the GPU, like [http://pointclouds.org/documentation/tutorials/using_kinfu_large_scale.php KinFu] does.
+Registration is a very useful technique because it lets you retrieve full, complete and continous [http://pointclouds.org/documentation/tutorials/in_hand_scanner.php models] of a scene or object. It is also expensive for the CPU. Optimizations have been developed that allow to do cloud matching in real time with the GPU, like [http://pointclouds.org/documentation/tutorials/using_kinfu_large_scale.php KinFu] does.
@@ Line 30: / Line 30: @@
 <div style="background-color: #F8F8F8; border-style: dotted;">
 * '''Tutorials''':
-** [http://pointclouds.org/documentation/tutorials/registration_api.php#registration-api The PCL Registration API]
+** [http://pointclouds.org/documentation/tutorials/registration_api.php The PCL Registration API]
 ** [http://www.pointclouds.org/assets/icra2013/Miller_presentation_ICRA13.pdf Clean Models from Noisy Sensors: Registration and Reconstruction Techniques]
 ** [http://www.pointclouds.org/assets/icra2013/registration.pdf 3D Registration with PCL]
@@ Line 95: / Line 95: @@
 * '''Output''': Registered source cloud, Transformation matrix
 * '''Tutorials''':
-** [http://pointclouds.org/documentation/tutorials/iterative_closest_point.php#iterative-closest-point How to use iterative closest point]
+** [http://pointclouds.org/documentation/tutorials/iterative_closest_point.php How to use iterative closest point]
-** [http://pointclouds.org/documentation/tutorials/pairwise_incremental_registration.php#pairwise-incremental-registration How to incrementally register pairs of clouds]
+** [http://pointclouds.org/documentation/tutorials/pairwise_incremental_registration.php How to incrementally register pairs of clouds]
-** [http://pointclouds.org/documentation/tutorials/interactive_icp.php#interactive-icp Interactive Iterative Closest Point]
+** [http://pointclouds.org/documentation/tutorials/interactive_icp.php Interactive Iterative Closest Point]
-** [http://pointclouds.org/documentation/tutorials/normal_distributions_transform.php#normal-distributions-transform How to use Normal Distributions Transform]
+** [http://pointclouds.org/documentation/tutorials/normal_distributions_transform.php How to use Normal Distributions Transform]
 * '''API''':
 ** [http://docs.pointclouds.org/trunk/classpcl_1_1_iterative_closest_point.html pcl::IterativeClosestPoint]
@@ Line 169: / Line 169: @@
 * '''Input''': Points, Model, Model threshold
 * '''Output''': Vector of Point indices (inliers)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/random_sample_consensus.php#random-sample-consensus How to use Random Sample Consensus model]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/random_sample_consensus.php How to use Random Sample Consensus model]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_random_sample_consensus.html pcl::RandomSampleConsensus]
 * [http://robotica.unileon.es/~victorm/PCL_sphere_model.tar.gz Download]
@@ Line 263: / Line 263: @@
 * '''Input''': Points, Model type, Model coefficients
 * '''Output''': Projected points
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/project_inliers.php#project-inliers Projecting points using a parametric model]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/project_inliers.php Projecting points using a parametric model]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_project_inliers.html pcl::ProjectInliers]
 * [http://robotica.unileon.es/~victorm/PCL_point_projection.tar.gz Download]
@@ Line 363: / Line 363: @@
 * '''Input''': Points, Cluster tolerance, Minimum cluster size, Maximum cluster size, Search method, [Indices]
 * '''Output''': Vector of Point indices (clusters)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/cluster_extraction.php#cluster-extraction Euclidean Cluster Extraction]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/cluster_extraction.php Euclidean Cluster Extraction]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_euclidean_cluster_extraction.html pcl::EuclideanClusterExtraction]
 * [http://robotica.unileon.es/~victorm/PCL_euclidean_clustering.tar.gz Download]
@@ Line 446: / Line 446: @@
 * '''Input''': Points, Cluster tolerance, Minimum cluster size, Maximum cluster size, Search method, Custom check function, [Indices]
 * '''Output''': Vector of Point indices (clusters)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/conditional_euclidean_clustering.php#conditional-euclidean-clustering Conditional Euclidean Clustering]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/conditional_euclidean_clustering.php Conditional Euclidean Clustering]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_conditional_euclidean_clustering.html pcl::ConditionalEuclideanClustering]
 * [http://robotica.unileon.es/~victorm/PCL_conditional_euclidean_clustering.tar.gz Download]
@@ Line 532: / Line 532: @@
 * '''Input''': Points, Minimum cluster size, Maximum cluster size, Neighbor count, Search method, Normals, Smoothness threshold, Curvature threshold, [Indices]
 * '''Output''': Vector of Point indices (clusters)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/region_growing_segmentation.php#region-growing-segmentation Region growing segmentation]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/region_growing_segmentation.php Region growing segmentation]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_region_growing.html pcl::RegionGrowing]
 * [http://robotica.unileon.es/~victorm/PCL_region_growing_clustering.tar.gz Download]
@@ Line 610: / Line 610: @@
 * '''Input''': Points, Minimum cluster size, Maximum cluster size, Search method, Distance threshold, Point color threshold, Region color threshold, [Indices]
 * '''Output''': Vector of Point indices (clusters)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/region_growing_rgb_segmentation.php#region-growing-rgb-segmentation Color-based region growing segmentation]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/region_growing_rgb_segmentation.php Color-based region growing segmentation]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_region_growing_r_g_b.html pcl::RegionGrowingRGB]
 * [http://robotica.unileon.es/~victorm/PCL_color_region_growing_clustering.tar.gz Download]
@@ Line 696: / Line 696: @@
 * '''Input''': Points, Foreground points (object's center), Object's radius, Sigma, Neighbor count, Foreground penalty, [Indices]
 * '''Output''': Vector of Point indices (object cluster)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/min_cut_segmentation.php#min-cut-segmentation Min-Cut Based Segmentation]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/min_cut_segmentation.php Min-Cut Based Segmentation]
 * '''Publication''':
 ** [http://gfx.cs.princeton.edu/pubs/Golovinskiy_2009_MBS/paper_small.pdf Min-Cut Based Segmentation of Point Clouds] (Aleksey Golovinskiy and Thomas Funkhouser, 2009)
@@ Line 765: / Line 765: @@
 * '''Input''': Points, Model type, Method type, Model threshold, [Optimize coefficients]
 * '''Output''': Vector of Point indices (plane cluster)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/planar_segmentation.php#planar-segmentation Plane model segmentation]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/planar_segmentation.php Plane model segmentation]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_s_a_c_segmentation.html pcl::SACSegmentation]
 * [http://robotica.unileon.es/~victorm/PCL_plane_segmentation.tar.gz Download]
@@ Line 823: / Line 823: @@
 * '''Input''': Points, Model type, Method type, Model threshold, [Optimize coefficients]
 * '''Output''': Vector of Point indices (cylinder cluster)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/cylinder_segmentation.php#cylinder-segmentation Cylinder model segmentation]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/cylinder_segmentation.php Cylinder model segmentation]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_s_a_c_segmentation.html pcl::SACSegmentation]
 * [http://robotica.unileon.es/~victorm/PCL_cylinder_segmentation.tar.gz Download]
@@ Line 927: / Line 927: @@
 * '''Input''': Points, Alpha
 * '''Output''': Hull points (concave)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/hull_2d.php#hull-2d Construct a concave or convex hull polygon for a plane model]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/hull_2d.php Construct a concave or convex hull polygon for a plane model]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_concave_hull.html pcl::ConcaveHull]
 * [http://robotica.unileon.es/~victorm/PCL_concave_hull.tar.gz Download]
@@ Line 1,000: / Line 1,000: @@
 * '''Input''': Points
 * '''Output''': Hull points (convex)
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/hull_2d.php#hull-2d Construct a concave or convex hull polygon for a plane model]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/hull_2d.php Construct a concave or convex hull polygon for a plane model]
 * '''API''': [http://docs.pointclouds.org/trunk/classpcl_1_1_convex_hull.html pcl::ConvexHull]
 * [http://robotica.unileon.es/~victorm/PCL_convex_hull.tar.gz Download]
@@ Line 1,087: / Line 1,087: @@
 * '''Input''': Points, Search method, Maximum distance, Maximum edge length, Minimum angle, Maximum angle, Maximum surface angle, [Normal consistency]
 * '''Output''': Polygon mesh
-* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/greedy_projection.php#greedy-triangulation Fast triangulation of unordered point clouds]
+* '''Tutorial''': [http://pointclouds.org/documentation/tutorials/greedy_projection.php Fast triangulation of unordered point clouds]
 * '''API''':
 ** [http://docs.pointclouds.org/trunk/classpcl_1_1_greedy_projection_triangulation.html pcl::GreedyProjectionTriangulation]