MoMa2024 Project: B2 Mobile Manipulation

Project Overview

  Our project focuses on robotic arm grasping, with the goal of improving its flexibility and precision. Initially, we replaced the original rigid mechanical claw with a flexible one, enhancing the robustness of the system. This change ensures that the arm can handle a variety of objects more effectively, especially those with different shapes and sizes.

  Object Detection

  To enable the robotic arm to identify and grasp objects, we use an RGBD camera, specifically the Realsense D435i, which provides both color and depth maps of the environment. Using the color image, we employ the YOLOv11 algorithm to detect the location of objects that need to be grasped. Once these objects are detected, we map their positions from the color space to the point cloud space.

  Grasp Pose Estimation

  Next, we input the point cloud data obtained from the depth map into a system called GraspNet-Baseline. This system is designed to analyze the 3D structure and determine the optimal grasp poses for the robotic arm's end-effector (the part that actually performs the grasping). We then filter out the most suitable poses based on the position of the object in the scene.

  Motion Planning

  To ensure that the robot arm moves accurately to the correct position, we convert the grasp pose into world coordinates, which are the global reference frame for the robot's environment. These coordinates are then sent to MoveIt, a motion planning framework, which calculates the optimal path for the arm to reach the object.

  Experimental Results

  Finally, the robotic arm performs the grasping action based on the planned path. In our practical tests, we selected three different objects to grasp, evaluating both the success rate and the time required for each attempt. The results of these tests provide valuable insights into the efficiency and reliability of our system, and guide further improvements.

  Video Demonstrations

  Result Images