Task semantics choose the grasp.
The same object can demand different contacts depending on the instruction. OpenDexGrasp treats language, visible parts, and geometry as coupled evidence for the desired action.
Abstract
Functional grasping is not just holding.
Dexterous grasp synthesis has made rapid progress in stable and physically plausible hand poses. Real manipulation, however, requires grasps that preserve task function: a kettle should be grasped by the handle for pouring, a knife should avoid unsafe contact regions during handover, and a spray bottle should leave the trigger accessible for use.
OpenDexGrasp studies open-vocabulary task-oriented dexterous grasp generation. The model must infer intent from natural language, ground the relevant object region from multi-view visual observations and 3D geometry, and generate an executable high-DoF grasp. Its key view is that affordance grounding and dexterous action generation are two supervision views of the same task-conditioned perception-action distribution.
1.24M
automatic grasps in OpenDex-Scale
557.18k
functional poses from VLM annotation
27.42k
teleop and transferred grasps in OpenDex-Align
72.0%
average real-world success rate
Key Insight
One latent space for task, contact, and action.
Affordance is supervision, not a bottleneck.
Point-level affordance labels ground the latent representation on functional surface regions, while inference remains direct grasp generation rather than a cascade of map prediction and pose search.
Coverage must precede alignment.
The C2A Recipe first expands semantic-geometric support with automatic synthesis, then aligns the distribution using high-quality teleoperated demonstrations and category-level transfer.
OpenDexVerse
Coverage-to-Alignment data construction.
OpenDexVerse combines two complementary sources. OpenDex-Scale provides broad semantic-geometric coverage through automatic synthesis and vision-language annotation. OpenDex-Align provides embodied alignment through human teleoperation and dense-correspondence transfer.
OpenDex-Scale
Large-scale functional coverage from automatically synthesized grasps, rendered multi-view images, and VLM functional annotations.
- Categories
- 105
- Instances
- 1.11k
- Grasps
- 1.24M
- Functional ratio
- 44.93%
OpenDex-Align
Compact but high-quality alignment from Manus-glove teleoperation, contact mapping, pose refinement, and transfer to nearby instances.
- Categories
- 95
- Instances
- 2.77k
- Grasps
- 27.42k
- Functional ratio
- 68.02%
Method
Direct generation with shared functional grounding.
Multi-view images and instructions produce open-vocabulary semantic tokens, while the point cloud supplies metric geometry. A transformer-based flow-matching action expert fuses semantic, geometry, and action tokens to generate dexterous grasp poses.
Open-vocabulary semantic-geometric latent
Hidden states from a pretrained vision-language encoder retain correlations among task words, object appearance, and view-dependent part evidence. A point-cloud encoder contributes global geometry and point features for contact-rich generation.
Flow-matching dexterous action generation
The action expert learns a time-dependent velocity field that transports Gaussian noise to a task-conditioned grasp pose, directly modeling the distribution of functional dexterous actions.
Auxiliary affordance grounding
A point-level affordance head supervises the same latent space, improving interpretability and functional contact without adding a separate test-time pose optimization stage.
Results
Better task alignment, feasibility, and transfer.
vs. 50.88% for DexGraspNet 2.0*
vs. 43.22% for DexGraspNet 2.0*
vs. 1.62 cm for DexGraspNet 2.0*
vs. 59.0% for DexGraspNet 2.0*
| Setting | Method | SIV | PD | SD | SR | Human score |
|---|---|---|---|---|---|---|
| Seen functional | DexGraspNet 2.0* | 5.12 | 1.04 | 1.51 | 50.88 | 6.10 |
| Seen functional | OpenDexGrasp | 1.28 | 0.39 | 1.37 | 68.07 | 7.85 |
| Unseen functional | DexGraspNet 2.0* | 6.86 | 1.62 | 4.29 | 43.22 | 5.37 |
| Unseen functional | OpenDexGrasp | 1.39 | 0.48 | 1.80 | 62.96 | 6.62 |
Demos
Language-conditioned grasps on everyday objects.
The learned policy is deployed on a real dexterous hand across seen and unseen categories, preserving functional contact choices while maintaining physically executable grasps.
Use the brush for cleaning.
Grasp the bouquet for floral decoration.
Grasp the massage hammer to relieve muscle tension.
Hold the rice paddle for scooping rice.
Pick up the pitcher to pour water.
Grasp the dustpan to collect debris.
Pick up the water bottle, to stay hydrated.
Grasp the bottle to dispense foam.
Grasp the umbrella to block the rain.
Hand over the small bucket to eat some nuts.
Citation
BibTeX
@inproceedings{zhang2026opendexgrasp,
title = {OpenDexGrasp: Open-vocabulary Task-Oriented Dexterous Grasping},
author = {Jiyao Zhang and Junhan Wang and Tianyu Wang and Zeyuan Chen and Anthony Bolton and Yitong Peng and Hao Dong},
booktitle = {Conference on Robot Learning (CoRL)},
year = {2026}
}