Smart Item Detection Skill
Detection of different objects.
Last updated
Detection of different objects.
Last updated
Similar to the Smart Vacuum Picking Skill, the Smart Item Detection Skill can also be used for vacuum gripper applications, but it differs fundamentally in terms of its structure and thus its application. This is a segmentation skill that determines instances of objects. Based on these segments, pick points are determined and displayed in the user interface. Through the segment recognition, an initial recognition of the position and orientation is possible. Longitudinal and transversal axes of the objects are recognized and can be used for process design in the robot program. The red axis of each possible pick point represents the x-axis of the object. With the help of this object coordinate system, the removal to be performed can be influenced by introducing offsets along and perpendicular to this axis of the gripping point. Besides the red one, there is also the green y-axis and a blue axis in z-direction. The center point is typically shown as a green circle, only with the next pick point displayed in light blue.
Thus, this skill as well as the other Vacuum Skill (Smart Vacuum Picking Skill) are ideal for warehouse applications. By identifying the different instances of the objects and returning the respective dimensions, it is possible to use these in the separation process in a more intelligent way.
Position
Position of the grasp point relative to the robot’s coordinate system (in m)
Orientation
Rotation of the grasp point relative to the robot’s coordinate system (in m)
Dimensions
Object dimensions, aligned with the pick pose axes
To ensure accurate identification of various types of objects, the skill parameters can be easily adjusted to fit your specific needs. Here are some recommendations to help you find the perfect parameters for your application. For your convenience, we've only included descriptions of parameters that differ from the default settings.
Used Objects: small cardboard boxes (70x50x50mm), matt surface
Camera Distance: 550mm
Camera Mount: 30° angle
Skill Parameters:
Pose Estimation: Bounding rectangle (due to the shape of the objects)
other parameters: default values
pca
Principal components use the orientation and the centroid of the mask as position of the pick pose
rectangle
Bounding rectangles, best used for planar rectangular items or cuboids when observed directly from above
ellipse
The ellipse fit, best for planar elliptic and spherical items
plane
Plane fitting selects the plane with the smallest angle towards the camera axis, useful for selecting a pick pose on the plane facing the camera when an item is observed from an angle. Works best for cuboids.
Minimum instance quality
min_score
float
Used to filter out detections with poor confidence. The impact of this parameter can differ between AI models.
Detection class
det_cls
int
Used to select a specific class ID for detections, detections with other class IDs are rejected. Valid IDs differ between models. The default value of -1 enables all classes.
Maximum allowed IoU
max_iou
float
Maximum allowed intersection over union between two detection masks, allows to resolve situations where multiple detections cover the same item. In conflicts, the mask with the highest score is selected.
Offset for pick point
offset_x
, offset_y
float
Allows to shift the pick position. The position can only be shifted within the x-y plane of the pick pose's coordinate frame. The x-axis is depicted as a red line and the y-axis as green.
Distance tolerance
distance_tolerance
float
Only used by the plane-fitting method for pose estimation, refers to the distance tolerance for inliers. All points that are closer to the fitted plane than this parameter are considered to be on the plane.
Minimum inlier fraction
min_fraction
float
Only used by the plane-fitting method for pose estimation, the minimum fraction of inliers a fitted plane must provide to be considered valid.
Optimize gripper orientation
orientation_optimization
boolean
Enables the optimization of the gripper orientation. If enabled the pick pose might be rotated by 180° around the approach axis to minimize the robot rotation between the capture and pick pose.
Enforce GPU usage
force_gpu
boolean
Enforces GPU usage for this skill (could disable GPU usage for other skills or cause performance problems.)
pose (Transformation)
Pick pose for the gripper
cls (int)
ID of the predicted class
dimensions (list)
Object dimensions, aligned with the pick pose axes
coverage (float)
Amount of detected shape covered by mask, changes depending on pose estimation method
optimization_angle (float)
Only present if the parameter optimization_orientation
is set to true
. The parameter represents the rotation angle in radians that was applied to the original pick pose during the orientation-optimization step.
Minimum coverage
min_coverage
float
This parameter is only useful in combination with the bounding rectangle/ellipse/plane fitting methods for pose estimation. For the rectangle and ellipse fitting it represents the intersection over union between the detected mask and the expected shape. For the plane-fitting method it sets a limit on the minimum fraction of the detected mask covered by the fitted plane.
Pose estimation method
pose_method
Method for the estimation of pick poses for each detected mask
Conditions
Camera mount:
Dynamic
Static
Camera distance:
35 – 45 cm
Parts dimensions:
5 – 15 cm
Parts material:
dull - low reflection
Specs
Avg. recognition time:
< 1 seconds
Supported grippers:
Vacuum
Magnet
Single contact point
Features
Measures
Collision detection
Multiple instance recognition
Generalized to a variety of object types
Picking from bulk
Cylinders
Random