WAM-Policy Release snapshot c320076b

공식 가중치

WAM-Policy adapts LingBot-VA into a language-conditioned policy for quadrupeds that manipulate with the front-left foot. The authors evaluated cross-morphology transfer, on-policy self-distillation and multi-teacher distillation in MuJoCo, and tested the Stage 1.2 OPSD student on a Unitree Go2 Air.

전체 모델 설명 보기

To run QLM-Bench, check out the qlm-bench branch separately and use a dedicated simulator environment with Python 3.11 or later. Install the QLM-Bench [live] extra in the WAM-Policy environment to connect the policy and simulator via the local qlm-live-v1 protocol. Set QLM_BENCH_SRC and QLM_PYTHON in configs/site.env to the simulator source directory and interpreter.

논문 및 보고서 (1)

WAM-Policy: World Action Models with On-Policy Distillation for Quadrupedal Loco-Manipulation

논문 있음

arXiv: 2611.24011DOI: 10.48550/arxiv.2611.24011
저자 및 소속 (6)

저자 순서: 원문 순서

  1. Junlin Long동등 기여

    Australian Centre for Robotics, USYD

  2. Xu Deng동등 기여

    UNSW Sydney

  3. Yiran Wang

    Australian Centre for Robotics, USYD

  4. Yue YangCorresponding author

    Maincode

  5. Yang ZhaoCorresponding author

    La Trobe University

  6. Zeyu ZhangProject lead

    Maincode

최초 공개
2026-11-18
최신 논문 개정
v1 · 2026-11-18
학회 / 학술지
미제공

관련 모델 버전

논문 ↗PDF ↗

접근 및 사용 방법

공개 코드 자료
공개 접근
모델 파일
공개 접근
학습 데이터
일부 공개

LIBERO arm evaluation and ablation guide

코드 라이선스

코드 및 학습 / 미세 조정

접근 방법
공개 접근

확인 날짜 · 2026-11-18

코드 라이선스
Apache-2.0

학습 데이터

일부 학습 데이터가 공개되어 있으나 전체 학습 데이터의 공개를 의미하지 않습니다.

모델 파일

모델 파일
Stage 1.1 transfer teacher

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Stage 1.2 OPSD student S300 EMA

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Stage 2.1 push-box specialist

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Stage 2.1 door-opening specialist

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Stage 2.2 M-OPD student S300 EMA

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Supplementary LIBERO OPSD S300 EMA접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

The adapter and its pinned LIBERO-Long base are published under Apache 2.0.

Quadruped ablation · Teacher cache only S300 EMA

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Quadruped ablation · Teacher cache only S2000 EMA

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Quadruped ablation · Streaming only S300 EMA

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

Quadruped ablation · OPSD S900 EMA

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

LIBERO arm ablation · Teacher cache only S300 EMA접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

The adapter and its pinned LIBERO-Long base are published under Apache 2.0.

LIBERO arm ablation · Streaming only S300 EMA접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

The adapter and its pinned LIBERO-Long base are published under Apache 2.0.

LIBERO arm ablation · Collected once S300 EMA접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

The adapter and its pinned LIBERO-Long base are published under Apache 2.0.

LIBERO arm ablation · SFT only S692 EMA접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

The adapter and its pinned LIBERO-Long base are published under Apache 2.0.

LIBERO arm ablation · Teacher cache only S2000 EMA접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

The adapter and its pinned LIBERO-Long base are published under Apache 2.0.

LIBERO arm ablation · OPSD S900 EMA접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

The adapter and its pinned LIBERO-Long base are published under Apache 2.0.

Quadruped ablation · RGB + proprioception S5000

학습 데이터: 공개 접근

접근 방법가중치 라이선스상업적 사용
명시된 조건에 따라 허용약관

출처확인 날짜 · 2026-11-20

조건 안내

Apache 2.0 applies to these weights. QLM-Bench data and RAMBO weights have separate CC BY-NC 4.0 terms.

출시 정보

출처확인 날짜 · 2026-11-20

배포 전 확인 사항

Physical results are reported by the authors for the Stage 1.2 OPSD student. The Stage 2.2 aggregate is a simulation result. Arm checkpoints are LIBERO experiments. The deploy/ directory uses CC BY-NC 4.0. The remaining WAM-Policy main-branch code and the officially released policy weights use Apache-2.0. The QLM-Bench simulator branch code, the QLM-Bench dataset, and the RAMBO controller weights distributed with it use CC BY-NC 4.0. The shared command and live interface files on main are provided under that branch’s Apache-2.0 license. The quadruped evaluation used one quadruped platform, one whole-body controller and one manipulating leg (the front left). Standard100 varied the scenes of the ten training tasks. It did not test transfer to other legged robots or generalization to unseen tasks or instructions. Stage 2.1 tuned only box pushing and door opening. The policy was trained entirely in simulation and performed worse on the physical robot. For v2, the authors plan to replace the two cameras with one wide-angle fisheye camera near the egocentric camera position, paired with LiDAR. The announcement gives a planned sensor setup. Release and weight availability dates are still unannounced. 가중치나 코드가 공개되어도 로봇에서 즉시 실행할 수 있다는 의미는 아닙니다. 문서에서 구성, 데이터 및 인터페이스 요구 사항을 확인하세요.

배포 및 사용 자료

추론 방식
OPSD and M-OPD students: 1 video step and 2 action steps
실행 환경 및 프레임워크
PyTorch

구조 및 입출력

모델 유형
행동 정책
아키텍처
WAM
백본
LingBot-VA
행동 생성
플로 매칭
입력
언어 / RGB 이미지
출력
Go2 Air: 16 commands per chunk, 9 values per command
출시일
2026년 11월 18일

적용 범위 및 학습 방식

코드 라이선스

Apache-2.0

본체 범위

다중 본체

구현 출처

공식

기록 유형

공식 가중치

출력

행동 / 비디오

예측 출력

Future video and action sequences

로봇 형태

사족 로봇 / 로봇 팔

작업 범위

다중 작업

학습 방식

강화 학습 / 지도 미세조정

학습 데이터

  • QLM-Bench

    미세조정공개됨

    MuJoCo benchmark and data for ten quadrupedal loco-manipulation tasks.

    800 episodes · CC BY-NC 4.0

평가 결과

현재 버전 또는 선택한 가중치의 검증된 독립 평가 결과가 없습니다.

발표 기관 평가 · 보고된 결과 · 독립 재현 안 됨

QLM-Bench · Standard100 ↗

Success rate: 85 %

mujoco-standard-test100-v1; seeds 42, 43, 44

72 episodes on the RTX 5070 Ti workstation with CPU VAE, and 228 on a second workstation with GPU VAE and concurrent clients.

MuJoCo Standard100, 100 cases across 10 tasks, 3 seeds, 300 episodes. Author-reported aggregate. Video/action steps 1 / 2.

전체 매개변수 및 검증 설명 (모델 카드)

Simulation and policy weights

QLM-Bench Standard100 uses ten MuJoCo tasks, ten held-out scenes per task, and seeds 42, 43 and 44 for 300 episodes per policy. Task time limits range from 24 to 60 seconds. The benchmark has 1,000 synthesized demonstrations. Stage 1.1 fine-tuned LingBot-VA on 800 of them and reached 77.0% with 25 video and 50 action sampling steps. Its Stage 1.2 OPSD student (rank-256 LoRA, 300 updates, EMA weights) reached 76.7% with one video and two action steps. Stage 2.1 uses separate rank-32 push-box (r14) and open-door (r04) RL teachers. Stage 2.2 starts from the Stage 1.2 student and distills the specialist teachers by task. The released Stage 2.2 M-OPD student (300 updates, EMA weights) reached 85.0% in simulation. Both students load the Stage 1.1 teacher as the inference base. LIBERO arm weights are supplementary experiments.

Physical trials

The authors ran the Stage 1.2 OPSD student trained on simulation data only on a Unitree Go2 Air, using the Go2 Air’s built-in egocentric camera and a RealSense D435i raised above the trunk for a top-down view. The robot carried an RDK X5 board. An off-board RTX 5070 Ti workstation ran the policy and VAE over wired Ethernet. RAMBO executed the base and front-left foot commands through whole-body control. Successes were 3/10 for button pressing, 1/10 for ball kicking and 3/10 for box pushing (7/30 total). On the physical robot, sampling took 0.159 s per action chunk. The complete control loop, including observation encoding, took 0.66 s. Each chunk covered 0.32 s of robot time, so the loop fell short of real time and the controller held its last command between chunks.

Real-robot deployment package

The authors provide a ROS 2 Jazzy package for Unitree Go2. The documented setup uses Ubuntu 24.04, Python 3.12 and an NVIDIA GPU on the host, wired Ethernet, and a RealSense D435i on an RDK X5 camera host alongside the Go2 front camera. The package uses RAMBO for whole-body control and QLM-Bench [live] for the qlm-live-v1 policy connection. The main ROS dependencies are rclpy, rcl_interfaces, unitree_go, unitree_api, geometry_msgs, std_msgs and sensor_msgs, with launch, launch_ros and ament_index_python at runtime. Fast DDS is the default, with Cyclone DDS documented as an option. The default policy is Stage 1.2 OPSD, which uses the Stage 1.1 teacher as its inference base. The paper reports hardware trials on Go2 Air. Physical results for Go2 Pro, X and EDU, as well as for the other policy options, are unconfirmed.

See sections 4–5, tables 1 and 7, and appendices E and G–H of the paper, as well as the authors’ Limitations and Future Work update.

출처 또는 수정 제안 보내기