WAM-Policy: World Action Models with On-Policy Distillation for Quadrupedal Loco-Manipulation
논문 있음
저자 및 소속 (6)
저자 순서: 원문 순서
- 최초 공개
- 2026-11-18
- 최신 논문 개정
- v1 · 2026-11-18
- 학회 / 학술지
- 미제공
WAM-Policy adapts LingBot-VA into a language-conditioned policy for quadrupeds that manipulate with the front-left foot. The authors evaluated cross-morphology transfer, on-policy self-distillation and multi-teacher distillation in MuJoCo, and tested the Stage 1.2 OPSD student on a Unitree Go2 Air.
전체 모델 설명 보기To run QLM-Bench, check out the qlm-bench branch separately and use a dedicated simulator environment with Python 3.11 or later. Install the QLM-Bench [live] extra in the WAM-Policy environment to connect the policy and simulator via the local qlm-live-v1 protocol. Set QLM_BENCH_SRC and QLM_PYTHON in configs/site.env to the simulator source directory and interpreter.
논문 있음
저자 순서: 원문 순서
LIBERO arm evaluation and ablation guide
출처확인 날짜 · 2026-11-20
확인 날짜 · 2026-11-18
일부 학습 데이터가 공개되어 있으나 전체 학습 데이터의 공개를 의미하지 않습니다.
| 파일 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
|---|---|---|---|
| Stage 1.1 transfer teacher 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Stage 1.2 OPSD student S300 EMA 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Stage 2.1 push-box specialist 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Stage 2.1 door-opening specialist 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Stage 2.2 M-OPD student S300 EMA 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Supplementary LIBERO OPSD S300 EMA | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Quadruped ablation · Teacher cache only S300 EMA 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Quadruped ablation · Teacher cache only S2000 EMA 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Quadruped ablation · Streaming only S300 EMA 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Quadruped ablation · OPSD S900 EMA 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| LIBERO arm ablation · Teacher cache only S300 EMA | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| LIBERO arm ablation · Streaming only S300 EMA | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| LIBERO arm ablation · Collected once S300 EMA | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| LIBERO arm ablation · SFT only S692 EMA | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| LIBERO arm ablation · Teacher cache only S2000 EMA | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| LIBERO arm ablation · OPSD S900 EMA | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
| Quadruped ablation · RGB + proprioception S5000 학습 데이터: 공개 접근 | 접근 방법 | 가중치 라이선스 | 상업적 사용 |
출처확인 날짜 · 2026-11-20
Physical results are reported by the authors for the Stage 1.2 OPSD student. The Stage 2.2 aggregate is a simulation result. Arm checkpoints are LIBERO experiments. The deploy/ directory uses CC BY-NC 4.0. The remaining WAM-Policy main-branch code and the officially released policy weights use Apache-2.0. The QLM-Bench simulator branch code, the QLM-Bench dataset, and the RAMBO controller weights distributed with it use CC BY-NC 4.0. The shared command and live interface files on main are provided under that branch’s Apache-2.0 license. The quadruped evaluation used one quadruped platform, one whole-body controller and one manipulating leg (the front left). Standard100 varied the scenes of the ten training tasks. It did not test transfer to other legged robots or generalization to unseen tasks or instructions. Stage 2.1 tuned only box pushing and door opening. The policy was trained entirely in simulation and performed worse on the physical robot. For v2, the authors plan to replace the two cameras with one wide-angle fisheye camera near the egocentric camera position, paired with LiDAR. The announcement gives a planned sensor setup. Release and weight availability dates are still unannounced. 가중치나 코드가 공개되어도 로봇에서 즉시 실행할 수 있다는 의미는 아닙니다. 문서에서 구성, 데이터 및 인터페이스 요구 사항을 확인하세요.
Apache-2.0
다중 본체
공식
공식 가중치
행동 / 비디오
Future video and action sequences
사족 로봇 / 로봇 팔
다중 작업
강화 학습 / 지도 미세조정
MuJoCo benchmark and data for ten quadrupedal loco-manipulation tasks.
800 episodes · CC BY-NC 4.0
현재 버전 또는 선택한 가중치의 검증된 독립 평가 결과가 없습니다.
발표 기관 평가 · 보고된 결과 · 독립 재현 안 됨
QLM-Bench · Standard100 ↗Success rate: 85 %
mujoco-standard-test100-v1; seeds 42, 43, 44
72 episodes on the RTX 5070 Ti workstation with CPU VAE, and 228 on a second workstation with GPU VAE and concurrent clients.
MuJoCo Standard100, 100 cases across 10 tasks, 3 seeds, 300 episodes. Author-reported aggregate. Video/action steps 1 / 2.
Simulation and policy weights
QLM-Bench Standard100 uses ten MuJoCo tasks, ten held-out scenes per task, and seeds 42, 43 and 44 for 300 episodes per policy. Task time limits range from 24 to 60 seconds. The benchmark has 1,000 synthesized demonstrations. Stage 1.1 fine-tuned LingBot-VA on 800 of them and reached 77.0% with 25 video and 50 action sampling steps. Its Stage 1.2 OPSD student (rank-256 LoRA, 300 updates, EMA weights) reached 76.7% with one video and two action steps. Stage 2.1 uses separate rank-32 push-box (r14) and open-door (r04) RL teachers. Stage 2.2 starts from the Stage 1.2 student and distills the specialist teachers by task. The released Stage 2.2 M-OPD student (300 updates, EMA weights) reached 85.0% in simulation. Both students load the Stage 1.1 teacher as the inference base. LIBERO arm weights are supplementary experiments.
Physical trials
The authors ran the Stage 1.2 OPSD student trained on simulation data only on a Unitree Go2 Air, using the Go2 Air’s built-in egocentric camera and a RealSense D435i raised above the trunk for a top-down view. The robot carried an RDK X5 board. An off-board RTX 5070 Ti workstation ran the policy and VAE over wired Ethernet. RAMBO executed the base and front-left foot commands through whole-body control. Successes were 3/10 for button pressing, 1/10 for ball kicking and 3/10 for box pushing (7/30 total). On the physical robot, sampling took 0.159 s per action chunk. The complete control loop, including observation encoding, took 0.66 s. Each chunk covered 0.32 s of robot time, so the loop fell short of real time and the controller held its last command between chunks.
Real-robot deployment package
The authors provide a ROS 2 Jazzy package for Unitree Go2. The documented setup uses Ubuntu 24.04, Python 3.12 and an NVIDIA GPU on the host, wired Ethernet, and a RealSense D435i on an RDK X5 camera host alongside the Go2 front camera. The package uses RAMBO for whole-body control and QLM-Bench [live] for the qlm-live-v1 policy connection. The main ROS dependencies are rclpy, rcl_interfaces, unitree_go, unitree_api, geometry_msgs, std_msgs and sensor_msgs, with launch, launch_ros and ament_index_python at runtime. Fast DDS is the default, with Cyclone DDS documented as an option. The default policy is Stage 1.2 OPSD, which uses the Stage 1.1 teacher as its inference base. The paper reports hardware trials on Go2 Air. Physical results for Go2 Pro, X and EDU, as well as for the other policy options, are unconfirmed.
See sections 4–5, tables 1 and 7, and appendices E and G–H of the paper, as well as the authors’ Limitations and Future Work update.