Yilin Wu 吴怡琳

Autonomous systems that efficiently utilize tools can assist humans in completing many common tasks such as cooking and cleaning. However, current systems fall short of matching human-level of intelligence in terms of adapting to novel tools. Prior works based on affordance often make strong assumptions about the environments and cannot scale to more complex, contact-rich tasks. In this work, we tackle this challenge and explore how agents can learn to use previously unseen tools to manipulate deformable objects. We propose to learn a generative model of the tool-use trajectories as a sequence of tool point clouds, which generalizes to different tool shapes. Given any novel tool, we first generate a tool-use trajectory and then optimize the sequence of tool poses to align with the generated trajectory. We train a single model on four different challenging deformable object manipulation tasks, using demonstration data from only one tool per task. The model generalizes to various novel tools, significantly outperforming baselines. We further test our trained policy in the real world with unseen tools, where it achieves the performance comparable to human.

In this work, we introduce spatially-grounded parameterized motion primitives to improve policy generalization for robotic manipulation tasks. By grounding the primitives on a spatial location in the environment, our proposed method is able to effectively generalize across object shape and pose variations.

In this work, we introduce DROID (Distributed Robot Interaction Dataset), a diverse robot manipulation dataset with 76k demonstration trajectories or 350h of interaction data, collected across 564 scenes and 86 tasks by 50 data collectors in North America, Asia, and Europe over the course of 12 months. We demonstrate that training with DROID leads to policies with higher performance, greater robustness, and improved generalization ability.

We present a system for bimanual manipulation that coordinates by assigning roles to arms: a stabilizing arm holds an object stationary while an acting arm acts in this simplified environment.

We build a semi autonomous robotic system to do in-mouth transfer of food safely and comfortably for disabled people. The system is composed of a force-reactive controller to safely accommodate the motions of the user throughout the transfer, a novel dexterous wrist-like end effector to reduce the discomfort and a visual sensor to identify the user mouth.

We propose a general bimanual scooping primitive and an adaptive stabilization strategy that enables successful acquisition of a diverse set of food geometries and physical properties with close-loop visual feedback.

Our work is to train a RL agent to acquire rope-spreading and cloth-spreading skills without any human demonstrations and the method applies to real robots after domain adaptation.

In this work, we introduce spatially-grounded parameterized motion primitives to improve policy generalization for robotic manipulation tasks. By grounding the primitives on a spatial location in the environment, our proposed method is able to effectively generalize across object shape and pose variations.