Robotic Coffee Maker

This project uses the Franka Emika Panda arm to brew a cup of pour over coffee. It uses computer vision and AprilTags to detect where each object is and how to grasp the object, and it uses Moveit2 to plan the trajectories. This project was completed over a course of 3 weeks.

Source code: GitHub
Moveit2 Wrapper: GitHub
Group Members: Stephen Ferro, Carter DiOrio, Anuj Natraj, Kyle Wang Jihai Zhao

Video Demo

Moveit Wrapper

This project uses a custom wrapper for the Moveit2 API. The purpose of this wrapper is to make implentation easier - it automatically fills in many of the message fields, and offers a simpler way of planning trajectories. Some of the notable features of this wrapper are:

• A grasp planner
  • Offers an easy way to set the expected payload
  • Moves to an approach pose, then a grasp pose, closes the gripper, and moves to a retreat pose
• Plan
  • Calculates inverse kinematics to move the end-effector of the robot to a given position and/or orientation
• Cartesian Trajectory
  • Plans a trajectory to move the end-effector through a series of poses
• Trajectory Constraints
  • Create position/orientation constraints for the end-effector
  • Maintain end-effector orientation while planning and executing a trajectory

AprilTags

AprilTags were used for localization. A camera was mounted on the ceiling so that it could see all of the AprilTags. The following were used:

• One large AprilTag on the table
  • This was used to localize the camera
• One AprilTag for each graspable object
  • These were placed on acrylic plates that the objects were mounted/attached to
  • These allowed the robot to see the general location of each object

Handle Detection

When picking up an object, the robot arm will first move above the AprilTag associated with that object. At that point, the camera mounted onto the arm will search for the handle. This was done by attaching blue and green tape to the handle, and using computer vision to find the handle and the orientation needed.

The image below depicts what the camera sees. The blue tape is found using a color mask, and the centroid of it is drawn in the image. The green tape is found using another color mask. Once these two colors are detected, a line is drawn from the centroid of the blue tape to the centroid of the green tape.

Pouring

Two pouring actions were needed: one for the kettle, and one for the coffee pot. Different actions were used because we wanted the pouring motion to be difficult. However, both actions had an approach pose, which was used to move the kettle/pot to an appropriate position, and a pour pose, which tilted the object to an angle that would start the pour.

When brewing a cup of pour over coffee, a spiral motion is ideal. To achieve this, we did the following:

1. Create transformations
   • Robot end-effector to kettle spout
   • Center of the coffee pot to the handle
2. Create a set of waypoints that represent the spiral
3. Apply the transformations to get a list of waypoints that the end-effector needs to follow in order to achieve the desired path
4. Plan the cartesian path.

Although we don’t need to pour the coffee from the pot using a spiral motion, we did need to increase the angle of the pour. The coffee pot has an hourglass shape. In order to pour all the coffee out, without it spilling out, we did two pours:

1. Pour at 90 degrees
2. Continue and tilt the pot more