In order to evaluate and demonstrate the performance of the developed bin-picking process, several tests were taken and a demonstration was held to present the versatility of the system.
Small cylindrical pieces
The objects used in the first test were small cylindrical pieces, with a diameter of 37mm. By using these small objects, it is possible to check the precision and accuracy of the process. Some of the pieces were placed in the workspace table with a inclination to force the manipulator to position itself with different Euler angles.
In the 20 tested pieces 6 of them were not successfully grabbed, which represents a percentage of success of 70%. This low percentage is caused by several easily visible problems:
- Surface's Normal
- Centroid
- Laser Sensor Readings
One cause of error is in the determination of the surface’s normal, which sometimes is not properly computed. By positioning the end effector with the Euler angles computed with the wrong surface’s normal, the suction cup will not be parallel to the object’s surface, thus not allowing the object to be gripped, as in figure 1a.
Since the Kinect has a low precision, the centroid’s coordinates can vary up to 3mm from the correct centroid. In this specific test, this error was not one of the reasons for not catching those 6 pieces, however the piece in figure 1b was almost not grabbed because of this, which means that objects smaller than the ones used in this test could have failed more often because of this flaw.
The main cause of error in the picking of objects derives from the oscillation in the laser sensor readings. This, then, can cause two opposite situations, the tool tip could be too far from the object when grabbing it, as in figure 1c, but also too close, as in figure 1d. The piece in the last case was obviously grabbed but if it had been a more fragile object it could have broken it.
1a
1b
1c
1d
Ping Pong Balls
In the second test, the objects used were ping pong balls. With this test it is possible to demonstrate the versatility of this process by having the capability of grabbing objects with curved surfaces. The process held in the first test was replicated four times with the ping pong balls since there were only 5 balls to pick, thus having also a sampling of 20 objects for the experiment.
Coincidentally, in the second trial, 6 ping pong balls were also not grabbed. However, in this case, the main cause of error resulted from the incorrect determination of the surface’s normal. This is explained by the fact that normal vectors in curved surfaces vary along the object’s surface, unlike flat surfaces in which the normal vectors have all almost the same orientation. From the 6 pieces not grabbed, 3 of them occurred because of this specific issue, like the one in figure 2a . Nevertheless, in other times the surface’s normal was poorly determined but since the tool tip got close enough to the surface, the gripper was able to grab the object, as in the case presented in figure 2b. The other two issues observed in the first trial were, as expected, also reported in this second experiment. One object was not grabbed because of the incorrect determination of the object’s centroid, presented in figure 2c, and the other 2 were due to the incorrect measurement of the distance with the laser sensor, as in figure 2d.
2a
2b
2c
2d
In order to solve the errors in the laser sensor’s readings, firstly, the Arduino, the circuit and all the connections, used to receive and interpret the sensor’s output, were placed in a box, thus protecting all the connections.
Afterwards the laser sensor was re-calibrated with another process to ensure that the calibration curve, that links the sensor’s distance displayed in its monitor and the reading received by the Arduino, was as precise as possible. Since the laser sensor was now attached to the manipulator, contrarily to the first time calibrating, it was possible to use the robot’s accuracy to preform a better calibration. This time the calibration was held not only with the sensor’s distance displayed in its monitor but also with the exact position of the robot’s end effector. So, two linear regressions were then computed to evaluate the relation between both these informations with the readings received by the Arduino. Both curves are presented in the following figure. Through the Z coordinate of the robot’s end effector it is possible to ensure that the collected data is always equally spaced, thus obtaining a more precise slope for the calibration curve. Therefore, the equation that converts the reading to a distance was altered to include the scope of the linear regression that uses the end effector’s coordinates (blue curve in the figure) and the intercept of the curve that uses the distance displayed in the sensor’s monitor (red curve in the figure). Consequently the Arduino’s code was changed and uploaded.
Eggs
After the alteration in the laser sensor’s calibration curve a third test was conducted. In this third trial the objects used were eggs, thus demonstrating that this process is sufficiently precise to grab fragile objects without deforming or breaking them. The eggs used were previously emptied in order to prevent the egg’s content to enter the pipes with compressed air, used for the suction, in case of breaking. Consequently, by using empty eggs these will be even more fragile and harder to grab without breaking. In third trial the procedure used in the second trial was replicated since there were only 5 eggs as well.
Now, from the 20 eggs used in this test 18 were successfully grabbed, and figures 3a, 3b and 3c present some of those successes. From the two unsuccessfully grabbed eggs, one was due to the incorrect determination of the surface’s normal, presented in figure 3d, and the other was caused by something not yet witnessed in these trials. Since the eggs become lighter when emptied, at the moment the end effector moves closer to more distant objects, the suction cup, sometimes, can push those objects when approaching them. This will cause situations has the one visualized in figure 3e. To solve this problem the suction can be activated when the manipulator is approaching the object and not after.
3a
3b
3c
3d
3e
Variety of Distinctive Shaped Objects
The last and fourth test was performed now with a variety of distinctive shaped objects. In the last test cobbles, eggs, ping pong balls, duct tapes and small cylindrical objects were scattered in front of the manipulator, within the limits marked on the table. Those limits represent the intersection between the area observed by the Kinect and the space in which the robot can safely move. Once more, the process applied in the other tests was used, and the objects were picked from the largest object to the smallest, i.e., objects represented by point clouds with more points were the first to be picked.
The two objects with flat surfaces were the first successfully grabbed at the first attempt.
4a
4b
The following largest object in the bin-picking process was the Lego piece. Since the surface of the piece captured by the Kinect was too inclined the determination of the correct surface’s normal kept failing, as can be seen in figure 4c in which the object was not grabbed because of this problem. However, after moving the Lego to a less inclined position, presented in figure 4d, the object was then grabbed at the first attempt, figure 4e.
4c
4d
4e
The next object to be picked was the duct tape, which despite being represented by a big point cloud it is a thin object and the grasping area must be properly centered for the suction cup to correctly grab it. Therefore, the first attempt, presented in figure 4f, was unsuccessful. However, it took only one other try to successfully pick up the duct tape, which was recorded in figure 4g.
4f
4g
Afterwards the eggs were all correctly grabbed on the first attempt except for one. The moments of the grasping of the 3 eggs were captured.
4h
4i
4j
The other egg was grabbed after two attempts, however, this one was positioned differently from the others, as can be seen in figure 4k. In this position the observed surface of the egg will be its top which has a more accentuated curvature, making the determination of the correct surface’s normal even harder, which was the cause of error in the first unsuccessful attempt, presented in figure 4k. The second attempt, shown in figure 4l, was unsuccessful due to the fact that the suction cup is required to be closer to the object in this conditions even though the laser reading had been correct.
4k
4l
4m
The test proceeded to the grasping of the smaller objects, starting with the ping pong balls. The first ball was grabbed only at the second attempt, moment recorded in figure 4o, once more due to a incorrect determination of the surface’s normal. Figure 4n shows the first attempt to pick the first ball. The second ball could not be picked at the current position since it was too close to the manipulator and the MoveIt! planner was never able to generate a 100% successful plan to grab the object in those conditions. For this reason a second marking was used to better visualize the manipulators limits, which can be seen in figure 4p near the ping pong ball. However, the MoveIt! planner was able to generate the movement to the centroid of the cylindrical piece, present in the same figure. This happened because, in oder to grab this flat object which is parallel to the table, the end effector would have to be perpendicular instead of reclined, which would be the necessary position to grab the ball.
Finally, the last small cylindrical pieces were all successfully grabbed on the first attempt.
4n
4o
4p
In conclusion, from the 13 objects used in this fourth test, 8 were grabbed on the first attempt and the remaining after one or two other attempts. It should also be noted that through all the tests with fragile objects, as the eggs, none of those were ever deformed or broken in the grasping process, at most they were not collected.