Research

Research projects I worked on.

My undergraduate research work mostly focuses on developing perception algorithms for robotic manipulation of deformable linear objects (DLOs). Deformable linear objects, such as ropes and cables, are crucial to our everyday lives. However, perception and tracking of deformable objects are more difficult than that of rigid objects because their shapes change under contact. My research can be divided into three stages:

All Deliverables

 

Simultaneous Shape Tracking of Multiple Deformable Linear Objects

The goal of multi-object tracking is to identify objects of interest in each frame of a video sequence and associate them across frames to track their movements. Multi-object tracking for deformable linear objects is difficult because objects can entangle with each other, posing great challenges for instance segmentation. Our work introduces an algorithm for tracking the shape of multiple entangling deformable linear objects from an RGB-D video sequence. Instance segmentation (expensive) is only required for the first frame and all subsequent frames run on semantic segmentation (cheap). This is achieved through the use of Global-Local Topology Preservation with geodesic instead of Euclidean distance, which better represents the distance between separate objects and the distance between different parts of the same object.

 

Deformable Linear Object Shape Tracking Under Occlusion

Deformable linear object tracking estimates the current state of the object from a sequence of segmented RGB-D images and is crucial to closed-loop manipulation tasks. Known failure cases of deformable linear object tracking include occlusion from other objects and itself. We introduce TrackDLO, a real-time, occlusion-robust deformable linear object tracking algorithm. The TrackDLO algorithm improves on previous approaches by addressing three common scenarios which cause their failure: tip occlusion, mid-section occlusion, and self-occlusion. This is achieved through a combination of: the use of a pre-processing step to preserve the total length of the deformable linear object; the application of Motion Coherence Theory to impute the spatial displacement field of the occluded portion of the object; and the use of the geodesic distance metric to better handle self-occlusion.

 

Deformable Linear Object Instance Segmentation

Instance segmentation locates and identifies objects of interest in a scene. Using the “Copy-Paste” data augmentation method, we generated a dataset containing 5000 images of ethernet cables and routers. We trained our instance segmentation model using the Detectron2 implementation of Mask R-CNN with the PointRend mask head. The instance segmentation bitmask predicted is directly used to segment the individual object’s 3D point cloud.

 

All Deliverables

Code
Papers and Reports
Presentations