hooneyskywalker/g1-motion-tracking-policies

G1 motion tracking policies

Code: github.com/hooneyskywalker0127/g1-motion-tracking
Videos: YouTube playlist

Six seconds of walk2_subject4. The trained policy on the left, the reference it is following on the right, both in Isaac Lab.

Whole-body control policies that make a Unitree G1 humanoid follow human motion capture clips in simulation. One policy per motion.

Why they exist: while building a swarm humanoid simulation, GEAR-SONIC from NVIDIA's GR00T Whole-Body Control was used to drive the G1. Using a controller and building one teach different things, so this pipeline was built end to end. SONIC also tracks a reference motion, so this is the same problem solved by a different hand.

The pipeline behind them: human mocap → retargeting to the G1 → reference motion → policy training. Stages 1 to 3 have no physics; physics enters at stage 4, where the robot has to hold itself up.

What is here

policies/<sequence>/ holds one trained policy each.

• model_29999.pt — RSL-RL checkpoint at (30,000) iterations
• policy.onnx — the actor network exported for inference, 160 in and 29 out
• run_name.txt — the training run folder the checkpoint came from

eval/<sequence>.json holds the measured numbers for that policy.

How to run a policy

policy.onnx is the actor network on its own. It takes one observation vector and returns one action. It holds no reference motion, so the caller supplies the reference terms.

The observation is 160 values concatenated in this order.

index	term	dim
0:29	reference joint position at the current motion frame	29
29:58	reference joint velocity at the current motion frame	29
58:61	motion anchor position in the robot anchor frame	3
61:67	motion anchor orientation, first two columns of the rotation matrix	6
67:70	base linear velocity in the base frame	3
70:73	base angular velocity in the base frame	3
73:102	joint position relative to the default pose	29
102:131	joint velocity relative to the default pose	29
131:160	previous action	29

The action is an offset on the joint position target: target = default joint position + action, for all (29) joints. Control runs at (50) Hz (sim step (0.005) s, decimation (4)), the same rate as the reference motion.

import numpy as np, onnxruntime as ort

sess = ort.InferenceSession("policies/walk1_subject1/policy.onnx")
obs = np.zeros((1, 160), dtype=np.float32)      # fill as in the table above
action = sess.run(["actions"], {"obs": obs})[0]  # (1, 29)

Joint order is the articulation order of the G1 asset used by BeyondMimic. Read it from the environment rather than assuming one.

How they were trained

BeyondMimic on Isaac Sim (5.1) and Isaac Lab (2.3.2), PPO, (4,096) environments, (30,000) iterations, one policy per motion. The reference motions come from LAFAN1 retargeted to the G1 with GMR.

Reward is tracking accuracy against the reference, not gait plausibility.

Results

Measured over (100) rollouts with domain randomization off. Completion rate is the share of rollouts that reach the end of the clip without the anchor body crossing a height or orientation threshold. The error column is the mean global body position error over all rollouts.

sequence	completion	mean alive	E_g-mpbpe (mm)
walk1_subject2	100%	100%	79
walk1_subject5	100%	100%	90
walk3_subject2	100%	100%	79
walk3_subject5	100%	100%	85
aiming1_subject1	100%	100%	89
walk2_subject1	100%	100%	114
dance2_subject3	100%	100%	103
walk4_subject1	100%	100%	60
walk2_subject4	99%	100%	90
run2_subject4	99%	100%	177
walk1_subject1	99%	99%	83
jumps1_subject1	98%	99%	151
obstacles3_subject3	98%	99%	162
walk2_subject3	96%	98%	129
walk3_subject4	0%	88%	116
walk3_subject1	0%	78%	112
obstacles2_subject1	0%	18%	368

Three sequences complete no rollout. walk3_subject4 and walk3_subject1 track most of the clip before the anchor crosses the threshold, at 88% and 78% of the frames. obstacles2_subject1 is the outlier at 18%.

obstacles2_subject1 fails on purpose to be informative: that clip has the actor climbing stairs, with the pelvis above (1.05 m) for (48.7) seconds. The training ground is flat, so the reference is not reachable and the error never drops. It is kept here as a record of what the selection criterion missed, since the criterion only looked at retargeting foot error and never asked whether the target was physically possible on flat ground.

sim-to-sim — does it survive a second simulator

A policy that only works in the simulator it was trained in is not evidence of anything. The same onnx actor was loaded into MuJoCo with no retraining and no fine-tuning, and all seventeen sequences were replayed for the full clip length.

Six seconds of walk2_subject4. Isaac Lab on the left, the same policy in MuJoCo on the right, both panels on the same instant.

two_standards.txt has the per-sequence numbers under both pass criteria on the matched protocol, and sim2sim_table.png renders the same tables. sym/<sequence>_simdr.json holds the Isaac side of each.

There is no single pass criterion

BeyondMimic scores by termination. The episode ends the moment any of three conditions fires, and all three look at z alone.

condition	quantity	threshold
anchor_pos	|ref_anchor_z − rob_anchor_z|	0.25
anchor_ori	|ref_gravity_z − rob_gravity_z|	0.8
ee_body_pos	ankles and wrists, |ref_rel_z − rob_z|	0.25

PolySim counts a rollout failed once the mean global body position error crosses 0.5 m. None of the termination conditions sees horizontal drift; PolySim's threshold is built to catch exactly that.

One caveat. PolySim's text says mean body position error over 0.5 m, but the released code tests whether any single body exceeds a curriculum threshold (1.5 m by default) and disables that check in the default configuration. The numbers below implement the text.

Is anything lost in transfer

The same policy was run in Isaac, where it was trained, and in MuJoCo, which it had never seen. The numbers do not drop.

metric	Isaac	MuJoCo	over
Success rate	0.765	0.775	17 sequences
Success rate (excluding the three at zero)	0.929	0.941	14 sequences
E_g-mpbpe	108.3 mm	101.3 mm	14 sequences
E_mpjpe	0.594 rad	0.593 rad	14 sequences

100 runs each. Error is averaged over completing runs only, so the three that complete none (obstacles2_subject1, walk3_subject1, walk3_subject4) drop out of the last three rows.

The most dynamic five seconds of dance2_subject3. Isaac Lab on the left, the same policy dropped into MuJoCo on the right. This sequence is where the two simulators agree most closely — 0.99 against 1.00 completion, 104.4 against 104.3 mm global error.

The rest of this section is how those numbers were produced.

Putting two simulators side by side requires the two columns to be the same quantity. Four things were matched: one scorer for both sides, the same initial perturbation distribution, reference and robot read at the same instant, and no early termination on either side with both criteria computed afterwards.

That last one matters most. With Isaac's termination enabled an episode ends the moment the robot falls, so global error after the fall is never recorded and a failed rollout scores as a PolySim success. MuJoCo has no termination, rolls to the end, and a fallen robot always crosses 0.5 m. The two numbers would carry the same name while measuring different things.

Three conditions: sim is Isaac without perturbation, sim-dr is Isaac with it over 100 environments, sim2sim is MuJoCo with the same perturbation over 100 trials. The window is full clip length.

	sim	sim-dr	sim2sim	retention
BeyondMimic success	0.779	0.765	0.775	101.4 %
PolySim success	0.668	0.633	0.609	96.3 %
global body error	105.6 mm	108.3 mm	101.3 mm	106.9 %
local pose, re-anchored	40.5 mm	40.6 mm	38.0 mm	106.7 %
joint angle	0.594 rad	0.594 rad	0.593 rad	100.2 %

Retention is MuJoCo/Isaac for success and Isaac/MuJoCo for error, so 100 % means nothing was lost either way. Nothing is lost.

Above 100 % does not mean MuJoCo is the better engine. Contact handling and the solver differ. The sentence this supports is that there is no transfer loss, and no more. The comparable published figure is PHUMA appendix D.3, 90.5 % and 93.2 % going from Isaac Gym to MuJoCo.

Per sequence, ordered by completion rate.

column	meaning
S_bm	share of rollouts that never trip any of the three BeyondMimic termination conditions, at the thresholds in the table above
S_poly	share of rollouts whose mean global body error never crosses 0.5 m, judged independently of S_bm
global	mean body position error in world coordinates (mm); grows with root drift
local	the same error after re-anchoring the reference to the robot anchor (mm), which removes root drift and leaves posture

Isaac columns are sim-dr (100 environments), MuJoCo columns are sim2sim (100 trials). Errors are averaged over completing trials only, so the three with none are undefined.

sequence	S_bm Isaac	S_bm MuJoCo	S_poly Isaac	S_poly MuJoCo	global Isaac	global MuJoCo	local Isaac	local MuJoCo
walk4_subject1	1.00	1.00	0.99	0.96	61.1	55.0	36.8	33.9
walk3_subject2	1.00	1.00	1.00	0.99	83.2	70.3	34.5	31.1
walk1_subject2	1.00	1.00	0.99	1.00	79.8	73.7	34.6	31.1
walk3_subject5	1.00	0.99	0.88	0.88	86.5	88.8	35.7	37.0
aiming1_subject1	0.99	1.00	0.93	0.86	89.7	74.2	35.6	32.4
walk1_subject5	1.00	0.94	0.99	0.86	91.2	86.2	33.9	31.5
dance2_subject3	0.99	1.00	0.99	1.00	104.4	104.3	45.4	42.3
walk2_subject1	1.00	1.00	0.91	0.95	115.2	101.7	43.2	41.0
walk1_subject1	1.00	1.00	0.96	0.99	77.7	69.3	33.9	30.6
walk2_subject4	0.99	0.99	1.00	0.98	91.7	77.6	40.3	36.7
run2_subject4	0.83	0.73	0.00	0.00	181.1	174.9	47.0	44.5
jumps1_subject1	0.98	0.97	0.05	0.14	155.3	143.3	42.6	40.6
obstacles3_subject3	0.58	0.80	0.62	0.75	165.6	160.1	54.7	51.1
walk2_subject3	0.64	0.76	0.45	0.00	134.0	138.8	50.2	48.8
walk3_subject4	0.00	0.00	0.00	0.00	—	—	—	—
obstacles2_subject1	0.00	0.00	0.00	0.00	—	—	—	—
walk3_subject1	0.00	0.00	0.00	0.00	—	—	—	—

The seventeen fall into four groups by motion type.

The ten walking, aiming and dance clips hold S_bm at 0.94 or better on both sides and S_poly at 0.86 or better, with 61-115 mm global and 31-45 mm local error. All four metrics sit close together across the two simulators.

Running and jumping (run2_subject4, jumps1_subject1) hold S_bm at 0.73-0.98 while S_poly drops to 0.00-0.14. They do not fail by falling, they fail by drifting, and their global error of 155-181 mm is the largest of the seventeen. One criterion alone hides this entirely.

obstacles3_subject3 and walk2_subject3 go the other way on S_bm: 0.58 and 0.64 in Isaac against 0.80 and 0.76 in MuJoCo. walk2_subject3 is the one exception worth naming — its S_poly falls from 0.45 to 0.00, the single cell where MuJoCo is clearly worse.

The three with no completed rollout have no error to report. obstacles2_subject1 survives 8.6 % of its clip and is unconverged; walk3_subject1 and walk3_subject4 reach 77-87 % and fail near the end.

Three things run through all of it. MuJoCo is worse than Isaac in only three cells out of the fourteen that report error. Error magnitude is set by motion difficulty rather than by simulator: 61-115 mm for walking, 104-166 mm for obstacles and dance, 155-181 mm for jumping and running. And failure splits in two: a low S_bm means the robot fell, while a low S_poly alone means heading error accumulated over a long clip.

The two scorers were checked against each other first. Isaac's env 0 rollout was dumped in full, rescored with the MuJoCo scorer and compared against the online values: all five metrics agree to four decimal places, the residual coming from the dump being float16.

Posture crosses over, position does not

Joint angle error is 0.594 rad in Isaac and 0.593 rad in MuJoCo. Posture crosses over with essentially no loss.

What does not cross over is position. Re-anchoring MuJoCo's 101.3 mm global body error to the robot anchor drops it to 38.0 mm, so 62 % of the error is root position and heading drift rather than posture. The same drift appears in Isaac (108.3 to 40.6 mm), so it is not a MuJoCo artefact.

The three that do not complete

Fourteen of seventeen complete the full clip. The other three are the zeros above, and they differ in kind: obstacles2_subject1 survives 8.6 % of its clip and is unconverged, while walk3_subject1 and walk3_subject4 reach 77-87 % and fail near the end, where the LAFAN1 actor sits or lies down.

On video it is one moment. The episode restarts from frame 0 once a termination condition fires, so the robot snaps back to its initial pose.

walk3_subject1 at 195 s, anchor height.

walk3_subject4 at 218 s, ankle and wrist height.

Retargeting Matters(arXiv:2510.02252) reports 96-100 % on the same LAFAN1, the same G1 and the same BeyondMimic, and states the two steps that account for the gap: it excludes motions with complex environment interaction, and it offsets each retarget by its mean minimum body height. Neither was done here.

kobe — does it hold outside LAFAN1

All seventeen above are LAFAN1 walking and dance, with no high-difficulty one-shot motion among them. One ASAP motion, kobe, was run through the same pipeline: 206 frames, 4.1 s.

The full 4.1 s. Isaac Lab on the left, MuJoCo on the right. Over 100 MuJoCo trials: BeyondMimic success 1.000, PolySim success 0.990, global body error 128.0 mm. No transfer loss here either.

One clip is not a sample. What it supports is that transfer holds outside LAFAN1, and nothing beyond that.

This clip is not placed against PolySim's Table III, for three reasons. The 0.100 cited there for IsaacSimDR → MuJoCo is the single-simulator DR baseline the table exists to argue against, not PolySim's own result; PolySim's row is the last one, IsaacSim+IsaacGym+Genesis, at 1.000. The paper names neither its 14 nor its 5 evaluation motions — Kobe is the only motion named in the text — so the set cannot be matched. And the trainer differs: PolySim uses HumanoidVerse with ASAP rewards and teacher-student, this uses BeyondMimic. Retargeting Matters reports sim2sim success mostly at 100 % on the same LAFAN1, G1 and BeyondMimic, so the 1.000 here is the ordinary value for this lineage, not a win.

eval_polysim/<sequence>.json holds the Isaac side under the 0.5 m criterion, next to the original eval/ numbers under Isaac's own thresholds.

Source data and license

The reference motions derive from LAFAN1 by Ubisoft, released under CC BY-NC-ND 4.0. That license does not permit sharing adapted material, so the retargeted motion data is not included here — only the trained weights and the measured numbers. The onnx export carries the actor network alone and no reference motion.

Use is non-commercial. Credit Ubisoft for LAFAN1.