ContactBench 1.0A Benchmark for Everyday Contact‑Rich Robotic Manipulation

Jingyi Xiang1 Yixuan Wang1 Jeannette Bohg2 Wenzhen Yuan1

1University of Illinois Urbana-Champaign 2Stanford University

Overview of the benchmark: its 3D-printable assets for 36 contact-rich manipulation tasks (containers, utensils, a light bulb, a key and tool rack, a socket and plugs, a knob, fasteners, and tools) laid out on a table; a grid of scenes from the simulation suite; and frames of a robot gripper performing tasks, labeled "Evaluate Robot Systems and Policies".

Motivation and Overview

What constitutes success in contact-rich manipulation?

While many prior works focus on insertion and peg-in-hole tasks or sustained-force tasks such as wiping and peeling, everyday object use spans a much broader range of contact behaviors. ContactBench 1.0 offers a concrete, example-driven answer: we identify eight contact-rich skills drawn from routine human activity, instantiated through 36 reference tasks, more than 40 3D-printable assets, and their simulation counterparts. The assets are designed to produce realistic contact force profiles, be reproducible across popular 3D printers, and support controlled variations to evaluate a robot system's ability to adapt to novel task conditions that might or might not be obvious to the vision modality alone. Together, the benchmark gives the community common ground for comparing methods. Our benchmarking experiments show contact remains hard for current policies, with or without tactile sensing.

Contact-Rich Manipulation Skills and Tasks

Drawn from observations of routine human activities, we identify eight contact-rich manipulation skills that we believe are important for a general-purpose robot performing everyday tasks:

ContactBench 1.0 includes 36 manipulation tasks to concretely evaluate a robot system's proficiency on the eight skills. Many tasks require more than one skill to complete. Due to the difficulty of many tasks for current robot systems, we record humans performing these tasks to demonstrate how they are supposed to completed. Below are some examples of the most challenging tasks that chain several skills together and demand more dexterity:

Browse all 36 tasks

3D-Printable Assets and Task Variations

ContactBench 1.0 includes more than 40 assets to support the included 36 manipulation tasks. Every ContactBench asset can be 3D-printed in standard PLA filaments and assembled with off-the-shelf M3 screws, and many come in variants that change what the task demands. We follow the definitions in A Taxonomy for Evaluating Generalist Robot Manipulation Policies and design assets to support the following types of generalization:

Visual + Behavioral Generalization

Variations to task-relevant objects that affect task behavior and can be observed by vision. Examples:

Behavioral Generalization

Variants that look the same to a camera but require different behavior. Examples:

Browse all asset models (coming soon)

Simulation

We provide simulated benchmark assets and task setups built with the Newton physics engine. The simulation suite reproduces each physical mechanism as an executable model with emphasis on force-realistic contact interactions and additionally includes teleoperation and tactile simulation modules.

Simulation code (coming soon)

Setting Up Benchmark Tasks in the Real World

  1. A 24 by 24 inch board covered edge to edge in strips of hook-side Velcro, with a fiducial marker in its top-right corner.
    Step 1

    Cover the target workspace with hook-side Velcro.

  2. Step 2

    Apply loop-side Velcro to the asset object.

  3. Step 3

    Systematically sample initial conditions and render them onto the Velcro-covered workspace.

  4. Step 4

    Align the physical asset object with the render. The asset will be firmly attached to the workspace through Velcro.

Policy Benchmarking Experiments

We collect data with a customized UMI hand-held gripper and evaluate two vision-only policy learning methods, Diffusion Policy (DP) and π0.5, and their representative visual-tactile variants, Reactive Diffusion Policy (RDP) and Foundation Tactile Policy (FTP-1) on six tasks from our benchmark:

While the policies were able to make progress toward solving each task, the success rate remains at 56.7% for the best performing policy-task combination, with turn doorknob to open door being the exception where all policies achieved near 100% success rate.

Success rate and average task progress

Task DP RDP π0.5 FTP-1
SR Prog SR Prog SR Prog SR Prog
Thread nut onto screw 8/300.39 0/300.24 14/300.66 14/300.63
Tighten screw with Allen key 0/300.58 0/300.36 17/300.82 3/300.45
Plug in charger 2/300.32 4/300.34 6/300.44 4/300.35
Turn doorknob to open door 30/301.00 29/300.98 30/301.00 29/300.98
Retrieve single utensil from bin 0/300.50 not evaluated 16/300.70 not evaluated
Pick up card from platform 14/300.74 not evaluated 2/300.28 not evaluated

SR: number of successful episodes out of 30. Prog: average fraction of task checkpoints reached per episode (0–1).

The trained policies tend to apply excessive forces when completing the tasks.

% of episodes with peak wrist force > 40 N

Task DP RDP π0.5 FTP-1
Thread nut onto screw33%27%27%83%
Tighten screw with Allen key10%0%7%3%
Plug in charger80%83%40%40%
Turn doorknob to open door10%63%7%43%
Retrieve single utensil from bin97%not evaluated47%not evaluated
Pick up card from platform57%not evaluated60%not evaluated

Even if the trained policies are able to complete the tasks, they sometimes take much longer and require more attempts compared to human demonstrations.

Average completion time of successful episodes

Task Human
demos		 DP RDP π0.5 FTP-1
Thread nut onto screw17.5 s1.72×–1.22×1.32×
Tighten screw with Allen key19.7 s––1.50×0.84׆
Plug in charger7.3 s1.06׆1.15׆0.89×0.95׆
Turn doorknob to open door7.6 s0.92×0.91×0.87×0.89×
Retrieve single utensil from bin11.8 s–not evaluated2.05×not evaluated
Pick up card from platform7.5 s1.11×not evaluated0.93׆not evaluated

For the policies, shown as a multiple of the time taken by human demonstrations. †Fewer than 5 successful episodes. A dash (–) marks no successful episodes.

What are challenging for vision-only policies on the selected tasks?

We find vision-only policies struggle to perceive and react to contact events:

  1. Establishing and maintaining contact The policies often miss or lose contact between the fingers, the object, and the environment.
  2. Precise alignment The policies don't make the small, visually subtle corrections needed to align an in-hand object with its target.
  3. Avoiding unsafe forceful interactions The policies often collide hard with task objects and the environment, which can fail the task or be unsafe.
  4. Recognizing task-critical states The policies can't tell some task-critical states apart by sight, such as how many chopsticks are grasped.

Not specific to contact‑rich manipulation

  1. Lack of memory The policies repeat finished stages that still look unfinished, like tightening an already-tight screw.

Establishing and maintaining contact

Precise alignment

Avoiding unsafe forceful interactions

Recognizing task-critical states

Lack of memory

Do visual-tactile policies address the challenges above?

Although intuitively, effective integration of force and/or tactile sensing can address the challenges above, we did not find RDP and FTP-1 to meaningfully improve on their vision-only base policies in success rate, task progress, interaction force, or completion time (as shown in the quantitative results earlier). Across their rollouts, we observed three aspects that limited the benefit of the added tactile input:

  1. Sensor coverage limited the tactile information available to the policy

    Because GelSight Mini only captures contact inside the gripper fingertip, it cannot perceive and avoid collisions happening to other parts of the gripper finger.

    Although vision-only policies performed poorly on retrieve single utensil from bin and pick up card from platform, we could not collect demonstrations or train visual-tactile policies for them because GelSight Mini's fingertip-only coverage and bulky form factor made it unsuitable for either task.

  2. Tactile-guided behaviors weren't learned

    Despite the availability of tactile feedback, we find visual-tactile policies to fail in the same ways as the vision-only policies.

  3. Policies acted on incidental tactile signals rather than task-relevant ones

    In tighten screw with Allen key, policies frequently terminate the tightening motion after the Allen key slips in hand, even though in-hand slip can occur naturally as the task progresses and does not reliably indicate that the screw has been fully tightened.

How well do policies generalize to task changes that are hard to perceive visually?

We conduct two behavioral generalization case studies on turn doorknob to open door, a task where all four policies achieve near-perfect success rates. We alter the task conditions in two ways:

Behavioral Variation: Required Rotation Amount to Unlock Door

While the policies were trained on a door that requires 60° rotation to unlock, we test them on two additional doors that require 72° and 80° rotation to unlock. The three doors are visually indistinguishable.

We find that RDP retains significantly higher success rates on the 72° and 80° variants compared to the other three policies. After investigating how much each policy actually rotates the door under the three task conditions, we find that RDP is the only policy that demonstrated adaptation to the changing task demand:

Left: bar chart of the success rate of DP, RDP, π0.5, and FTP-1 on the 60°, 72°, and 80° doors. Right: box plots of how far each policy turned the knob on each door, with the rotation needed to unlock it marked by a red dashed line. Only RDP's turns grow with the required rotation.
Left: success rate on each door (30 episodes per policy and door). Right: how far each policy turned the knob in each episode.

This shows that tactile-augmented policies have the potential to achieve better behavioral generalization even when they offer no in-distribution gain. However, the extent to which tactile sensing helps also depends on how tactile signals are integrated into the policy.

DP vision
RDP vision + tactile
π0.5 vision
FTP-1 vision + tactile

Visual + Behavioral Variation: Smaller Doorknob Diameter

While the policies were trained on a door with a 60 mm diameter doorknob, we test them on a door with a smaller 55 mm diameter doorknob. Although this change can be perceived visually, the visual differences are subtle.

Across 10 rollouts per policy (we did not run the full 30-rollout protocol since the failure was immediate and consistent across every rollout of every policy), all four policies achieved 0% success rate, and they were nowhere near establishing a stable grasp and completing the task:

DP vision
RDP vision + tactile
π0.5 vision
FTP-1 vision + tactile

We further investigate the gripping width each policy reached when attempting to grasp the doorknob. None of the policies attempted to close the gripper more for the smaller doorknob, indicating that they failed to learn a key aspect of grasping: adapting to the object’s size to achieve stable contact.

Gripping width each policy reached when attempting to grasp the doorknob (mm)
Knob DP RDP π0.5 FTP-1
60 mm (train) 58.9 ± 0.1259.0 ± 0.1358.8 ± 0.1359.2 ± 0.14
55 mm 59.0 ± 0.1159.0 ± 0.1158.7 ± 0.1158.9 ± 0.14

All Policy Rollouts

BibTeX

@article{xiang2026contactbench,
  title   = {{ContactBench} 1.0: A Benchmark for Everyday Contact-Rich Robotic Manipulation},
  author  = {Xiang, Jingyi and Wang, Yixuan and Bohg, Jeannette and Yuan, Wenzhen},
  journal = {arXiv preprint arXiv:2610.24512},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.24512}
}