Publications

2024

1. 

   Kitchen Artist: Precise Control of Liquid Dispensing for Gourmet Plating

   Hung-Jui Huang, Jingyi Xiang, and Wenzhen Yuan

   In IEEE International Conference on Robotics and Automation (ICRA), May 2024

   Abstract PDF Project VIDEO

   Manipulating liquid is widely required for many tasks, especially in cooking. A common way to address this is extruding viscous liquid from a squeeze bottle. In this work, our goal is to create a sauce plating robot, which requires precise control of the thickness of squeezed liquids on a surface. Different liquids demand different manipulation policies. We command the robot to tilt the container and monitor the liquid response using a force sensor to identify liquid properties. Based on the liquid properties, we predict the liquid behavior with fixed squeezing motions in a data-driven way and calculate the required drawing speed for the desired stroke size. This open-loop system works effectively even without sensor feedback. Our experiments demonstrate accurate stroke size control across different liquids and fill levels. We show that understanding liquid properties can facilitate effective liquid manipulation. More importantly, our dish garnishing robot has a wide range of applications and holds significant commercialization potential.

2023

1. 

   TrackDLO: Tracking Deformable Linear Objects Under Occlusion with Motion Coherence

   Jingyi Xiang, Holly Dinkel, Harry Zhao, Naixiang Gao, Brian Coltin, Trey Smith, and Timothy Bretl

   IEEE Robotics and Automation Letters, Aug 2023

   Abstract PDF VIDEO Code Poster

   The TrackDLO algorithm estimates the shape of a Deformable Linear Object (DLO) under occlusion from a sequence of RGB-D images. TrackDLO is vision-only and runs in real-time. It requires no external state information from physics modeling, simulation, visual markers, or contact as input. The algorithm improves on previous approaches by addressing three common scenarios which cause tracking failure: tip occlusion, mid-section occlusion, and self-occlusion. This is achieved through the application of Motion Coherence Theory to impute the spatial velocity of occluded nodes, the use of the topological geodesic distance to track self-occluding DLOs, and the introduction of a non-Gaussian kernel that only penalizes lower-order spatial displacement derivatives to reflect DLO physics. Improved real-time DLO tracking under mid-section occlusion, tip occlusion, and self-occlusion is demonstrated experimentally. The source code and demonstration data are publicly released.

2. 

   Simultaneous Shape Tracking of Multiple Deformable Linear Objects with Global-Local Topology Preservation

   Jingyi Xiang, and Holly Dinkel

   In Workshop on Representing and Manipulating Deformable Objects, IEEE International Conference on Robotics and Automation (ICRA), May 2023

   Abstract PDF VIDEO Code Poster

   This work presents an algorithm for tracking the shape of multiple entangling Deformable Linear Objects (DLOs) from a sequence of RGB-D images. This algorithm runs in real-time and improves on previous single-DLO tracking approaches by enabling tracking of multiple objects. This is achieved using Global-Local Topology Preservation (GLTP). This work uses the geodesic distance in GLTP to define the distance between separate objects and the distance between different parts of the same object. Tracking multiple entangling DLOs is demonstrated experimentally. The source code is publicly released.

2022

1. 

   Wire Point Cloud Instance Segmentation from RGBD Imagery with Mask R-CNN

   Holly Dinkel*, Jingyi Xiang*, Harry Zhao, Brian Coltin, Trey Smith, and Timothy Bretl

   In Workshop on Representing and Manipulating Deformable Objects, IEEE International Conference on Robotics and Automation (ICRA), May 2022

   Abstract PDF VIDEO

   Perception of the shapes of deforming objects like wires enables their monitoring and manipulation by autonomous robots. This paper presents detection, classification, and instance segmentation of deformable wires from a cluttered scene in RGBD imagery. This work uses the Detectron2 implementation of Mask R-CNN trained with the PointRend mask head on the UIUCWires dataset as the framework for wire instance segmentation on RGB imagery, a method demonstrated to perform well for the instance segmentation task in previous work. In this work, the instance bitmask is directly used to segment individual object point clouds, an important step toward wire shape representation for manipulation tasks.