VOFA: Visual Object-Goal Pushing with Force-Adaptive Control for Humanoids

Accepted at IROS 2026


  • Zichao Hu1,
  • Zifan Xu1,
  • Dongsik Chang2,
  • He Yin2,
  • Linh Tran1,
  • Roberto Martín-Martín12,
  • Peter Stone13,
  • Jingyu Qiao2,
  • Joydeep Biswas1
1The University of Texas at Austin, 2Amazon Inc., 3Sony AI
GitHub Code
Model Model
arXiv arXiv

Abstract

The ability to push large objects in a goal-directed manner using onboard egocentric perception is an essential skill for humanoid robots to perform complex tasks such as material handling in warehouses. To robustly manipulate heavy objects to arbitrary goal configurations, the robot must cope with unknown object mass and ground friction, noisy onboard perception, and actuation errors, all in a real-time feedback loop. Existing solutions either rely on privileged object-state information without onboard perception or lack robustness to variations in goal configurations and object physical properties. In this work, we present VOFA, a visual goal-conditioned humanoid loco-manipulation system capable of pushing objects with unknown physical properties to arbitrary goal positions. VOFA consists of a two-level hierarchical architecture with a high-level visuomotor policy and a low-level force-adaptive whole-body controller. The high-level policy processes noisy onboard observations and generates goal-conditioned commands to operate in closed loop across diverse object-goal configurations, while the low-level whole-body controller provides robustness to variations in object physical properties. VOFA is extensively evaluated in both simulation and real-world experiments on the Booster T1 humanoid robot. Our results demonstrate strong performance, achieving over 90% success in simulation and over 80% success in real-world trials. Moreover, VOFA successfully pushes objects weighing up to 17 kg, exceeding half of the Booster T1’s body weight.

Method

VOFA uses a two-level hierarchy for visual goal-directed humanoid object pushing. A high-level visuomotor policy maps proprioceptive history, recent depth observations, and the target object goal to compact motion commands, while a frozen FALCON-based force-adaptive whole-body controller converts those commands into stable joint targets during contact-rich pushing. The high-level policy is trained with a privileged PPO teacher in IsaacGym and distilled into a vision-based student with DAgger. Rewards encourage reaching, pushing, visual tracking, and object-goal alignment, while domain randomization over object properties, scene geometry, camera extrinsics, and depth artifacts enables transfer to the real robot.

VOFA hierarchical policy and force-adaptive controller

Experiments

[Simulation Ablation] Force-Adaptive Control

With Force-Adaptive WBC success

With the force-adaptive controller, the robot stably pushes the object using its end-effector.

Without Force-Adaptive WBC failure

Without it, the robot struggles to apply consistent forces and often resorts to kicking, causing instability.

[Simulation Ablation] Object-Goal Alignment Reward

With Alignment Reward success

The alignment reward enables the policy to reposition around the object before pushing it toward the goal.

Without Alignment Reward failure

Without it, the robot tends to make premature contact and fails to push the object to the goal.

[Real World Deployment] Different Goal Positions

Left Goal

Front Goal

Right Goal

[Real World Deployment] Different Masses

0 lb

5 lb

10 lb

2.5 lb

2.5 lb Goal Behind

[Demo] Closed-Loop Demonstration

Simulation Perturbation

VOFA recovers after the object is externally disturbed during execution.

Real Off-Centered Mass

VOFA adapts online when the object dynamics differ from the nominal setup.

BibTeX

@misc{hu2026vofavisualobjectgoal,
      title={VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids}, 
      author={Zichao Hu and Zifan Xu and Dongsik Chang and He Yin and Linh Tran and Roberto Martín-Martín and Peter Stone and Jingyu Qiao and Joydeep Biswas},
      year={2026},
      eprint={2605.01518},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2605.01518}, 
}