Vision-Based Hybrid Detection for Pick and Place Application in Robotic Manipulators

Pick and place using cobots is a common application in industry. In a cyber-physical system, a smarter cobot with vision sensing can decrease the uncertainty in decision-making for acquiring the position of objects in a scene. In this paper, a UR5 cobot is used as a cobot to automatically detect objects on a tabletop utilizing a monocular wrist camera. An improved statistical method and error reduction approach is designed by formulation of a mathematical relation for estimating the position of the object (w.r.t to the manipulator coordinate system) using the object detected in the camera coordinate system. Firstly, correspondence between simulated world coordinates and image coordinates is used to make a relation. This is followed by an error reduction approach by capturing multiple images and gradually moving towards the target centre. The object location accuracy of 99.785% was achieved using the statistical method. The error is reduced up to 2mm which is compensated since the gripper is still able to pick up the object. Our proposed method can be used to accurately approach an object's location and can be used effectively in pick-and-place applications using robotic manipulators. Fruit sorting has been used as a sample application however the proposed method is generic and can be applied to any object.