Marker & Landmark Detection Skill

This skill is used for the spatial detection of so-called markers. It can be used to determine the relative orientation of the camera and thus of the robot in space. Therefore, robot movements can be executed relative to the translational and rotational position of the marker in the spatial coordinate system. The supplied calibration plate can be used as an initial landmark. This can be used for a first example program. The corresponding marker is searched for within the camera's field of view (FOV), while in the left edge of the image the information obtained is shown. In addition to the ID of the marker, the orientation and position in relation to the robot coordinate system are also indicated. The image also shows the marker position as a coordinate system and the outline of the detected marker in green.

You can also search for other types of markers, in specific "Aruco Markers" and "April Tags". It is possible to search for the next marker or a specific ID - The detected information is displayed in the Web GUI and also passed to the robot. For example, different processes can be stored for differently recognized marker IDs.

If the marker is attached to a machine where the door is to be opened, the robobrain® system can recognize the marker and pass this information to the robot. A relative process is then executed by the robot to open the door.

Another alternative is the positioning of a mobile robot in space. The detected markers and corresponding IDs can be used to determine different positions in the room and thus assist the internal navigation of the AGV/AMR.

Skill Result Information

Position	Position of the grasp point relative to the robot’s coordinate system (in m)
Orientation	Rotation of the grasp point relative to the robot’s coordinate system (in m)
Selected ID	Detected marker id next to the camera

Specifications

Conditions

Camera mount:

• Dynamic

• Static

Supported marker types:

• Aruco marker

• April tags

Specs

Avg. recognition time:

< 0.5 seconds

Features

• Multiple ID recognition

• Search by known ID

• Position and orientation of Marker

• Relative movements

Technical Parameter Description

Parameter

Basic

Name

Marker type

Parameter

marker_type

Type

string

Detections

The type of the markers that should be detected. This skill is able to detect Aruco markers and April tags. Both marker types are organized in dictionaries containing several markers with similar styles. In principle, the correct type/dictionary for markers should be provided by the markers supplier or source. It is, however, also possible to use this parameter to test for which choice markers are robustly detected.

Name

Marker size

Parameter

marker_size

Type

float

Detections

The length of the side of the (square) markers. All supported marker types consist of black squares with different patterns inside. This parameter refers to the side length of the outer black square not the size of a plate or paper where the marker has been printed to. The distance between the camera and the marker is determined using the side length. Thus, the accuracy of this parameter limits the accuracy of the estimated marker position.

Name

Marker ID

Parameter

marker_id

Type

int

Detections

The ID of the marker that should be detected. This parameter allows to select a specific marker ID for the selected type/ dictionary. When a specific ID is selected (values of 0 and larger) only the marker with this ID will be detected and its position and orientation are returned. For a value of -1 all markers of the selected type will be detected. If more than one marker is in the image, the pose of the marker closest to the camera will be returned.

Marker Types

Type	Description
Aruco markers (aruco_*)	The original Aruco dictionary (aruco_original) is supported. In addition, it is possible to select the inner pattern size (aruco_4x4, aruco_5x5, aruco_6x6, aruco_7x7). Sometimes the number of markers within a dictionary is given in addition to the pattern size. The skill supports 50, 100, 250, and 1000 markers for each pattern size.
April tags (apriltag_*)	The dictionaries apriltag_16h5, apriltag_25h9, apriltag_36h10, and apriltag_36h11 are supported.
robominds calibration board (cal_board_2)	The calibration board shipped with every robobrain system

Detections

Detection Data

Type

pose (Transformation)

Description

Valid pick pose obtained from the estimated marker pose by a rotation by pi around the x-axis

Type

marker_pose (Transformation)

Description

Estimated pose of a detected marker or board

Type

id (int)

Description

The marker ID; is set to the first detected ID for a marker board

Type

detected_markers (List[int])

Description

List of IDs of all detected markers in a marker board.

Type

reprojection_error (float)

Description

The reprojection error of the detected marker/ board in pixels. To calculate this value the estimated 3D positions of the marker corners (based on the estimated marker/board pose) are projected onto the (2D) image plane. The reprojection error is the root mean-squared distance between the projected 2S pixel positions and the originally detected corner positions on the image.

Detection Example

{
   "detections":[
      {
         "pose":{
            "x":-0.1759275979020234,
            "y":-0.45028105388126916,
            "z":0.5281056569086842,
            "rx":0.08553337214707792,
            "ry":0.8743439320105111,
            "rz":-3.005322854349089
         },
         "id":7.0,
         "detected_markers":[
            7.0
         ],
         "reprojection_error":256.15457118701585,
         "marker_pose":{
            "x":-0.1759275979020234,
            "y":-0.45028105388126916,
            "z":0.5281056569086842,
            "rx":-0.01620815228924386,
            "ry":2.9640177203291507,
            "rz":0.862326955784828
         }
      },
      {
         "pose":{
            "x":0.04650726067322315,
            "y":-0.44969409639601127,
            "z":0.5362522356527505,
            "rx":0.005653163496416271,
            "ry":0.8493586613220064,
            "rz":-3.002043180699493
         },
         "id":4.0,
         "detected_markers":[
            4.0
         ],
         "reprojection_error":255.9139764368325,
         "marker_pose":{
            "x":0.04650726067322315,
            "y":-0.44969409639601127,
            "z":0.5362522356527505,
            "rx":-0.034052590502290966,
            "ry":3.0192676489588535,
            "rz":0.8542319261027429
         }
      },
      {
         "pose":{
            "x":-0.17891818462436454,
            "y":-0.33804238310729495,
            "z":0.5966476331532716,
            "rx":0.09765965111755973,
            "ry":0.8185317020342537,
            "rz":-3.0249683158533056
         },
         "id":6.0,
         "detected_markers":[
            6.0
         ],
         "reprojection_error":256.0827557242891,
         "marker_pose":{
            "x":-0.17891818462436454,
            "y":-0.33804238310729495,
            "z":0.5966476331532716,
            "rx":-0.00972781806733381,
            "ry":2.9723738848238814,
            "rz":0.8043000788723116
         }
      },
      {
         "pose":{
            "x":0.043673146625022835,
            "y":-0.3372803607584952,
            "z":0.6047304082740007,
            "rx":-0.007401359425139137,
            "ry":0.8105320478321518,
            "rz":-3.0110170759918486
         },
         "id":5.0,
         "detected_markers":[
            5.0
         ],
         "reprojection_error":255.68538998222976,
         "marker_pose":{
            "x":0.043673146625022835,
            "y":-0.3372803607584952,
            "z":0.6047304082740007,
            "rx":0.036671344291137734,
            "ry":-3.0288126521715557,
            "rz":-0.8153224174777772
         }
      }
   ]
}