Sample Efficiency in Reinforcement Learning: A Foal Needs 62 Attempts, a Robot Needs Ten Billion Steps

Sample efficiency in reinforcement learning has a benchmark from biology, and it is not flattering. In 2024 a team publishing in the Journal of Equine Veterinary Science filmed 14 healthy foals continuously from birth to 24 hours of age. Mean time to first successful stand: 56.4 minutes. Mean time to first walk: 1 hour and 1.9 minutes. Mean number of failed attempts before that first stand: 61.9, with a range of 14 to 103.

Sixty-two tries. That is the entire training budget.

Now the machine. A quadruped or humanoid locomotion policy trained from random initialisation burns through somewhere between one and ten billion simulated control steps before it walks dependably on uneven ground. Compute is the usual explanation for the gap and it is wrong. We have obscene amounts of compute. It bought speed, not efficiency. What is missing is priors.

Four minutes on the wall clock, a billion steps underneath

The paper that reset expectations was Rudin, Hoeller, Reist and Hutter in 2021, out of ETH Zurich and NVIDIA. Thousands of simulated ANYmal robots in parallel on a single workstation GPU, with a terrain curriculum that promoted robots to harder ground as they succeeded. Flat-terrain policies in under four minutes. Uneven terrain in twenty. Orders of magnitude faster than anything before it.

Read that carefully. The four minutes is wall clock. The sample count underneath did not shrink. It grew, and the parallelism absorbed it.

Five years on, the whole industry is built on that trade. NVIDIA shipped an early developer release of Isaac Sim 6.0 in January 2026 and took it to general availability mid-year, with multi-backend physics covering both PhysX and Newton and a Warp-native core. Isaac Lab 3.0 sits on top of it. Newton itself was contributed to the Linux Foundation on 29 September 2025 by Disney Research, Google DeepMind and NVIDIA, built on NVIDIA Warp and OpenUSD, Apache-2.0, with MuJoCo Warp as its primary backend, and reached 1.0 at GTC 2026. MuJoCo 3.8.0 landed on 24 April 2026, and MuJoCo Warp is reported to run complex scenes more than 100x faster than the earlier JAX implementation. Genesis reports 43 million frames per second on a single RTX 4090 for a Franka manipulation scene, roughly 430,000 times real time.

None of that made the algorithm need fewer samples. It made samples cheap.

Sixty-two attempts is a plausible number for calibrating a controller you already have. It is an absurd number for finding one from scratch.

Simulation is priced per watt, reality is priced per operator

Genesis AI put a number on the asymmetry when it released Genesis World 1.0 on 27 May 2026. A comprehensive evaluation pass for a robot policy, run physically, needs more than two hundred hours of continuous operation with one operator and one robot station. The same evaluation in their simulator takes under half an hour.

That ratio is the real reason everything happens in simulation, and it has nothing to do with fidelity. A simulator has no gearboxes. It does not overheat, does not need its foot re-seated, does not run out of battery and does not require a human to walk over and put it back at the start line.

I have spent enough time in Shenzhen demo halls to know what the physical bottleneck actually looks like, and it is not FLOPS. It is a young engineer in a company polo shirt kneeling on the carpet to reposition a humanoid's left foot for the ninth time in an hour while a colleague swaps a battery pack. The Chinese-language spec sheets I read from domestic suppliers are candid about this in a way the English marketing material is not. They quote service intervals for the 谐波减速器, the harmonic drive, in operating hours. Every real-world rollout spends down that interval. Simulation spends nothing.

Domain randomization is a bet on variance, not accuracy

The standard bridge from a simulator to a machine is domain randomization, introduced by Josh Tobin and colleagues at IROS 2017. The insight was to stop trying to make the simulator correct and instead make it vary so wildly that reality reads as one more sample from the training distribution. Randomise textures, lighting and camera pose for vision. Randomise mass, friction, motor gains, sensor noise and control latency for physics.

The latency case shows how granular this gets. Real hardware shows something like 9 to 12 milliseconds between a command going out and the joint position visibly changing, so training randomises sensor-to-actuator latency across a 0 to 20 millisecond band and the policy learns to be indifferent.

OpenAI pushed the method to its limit with automatic domain randomization on the Rubik's cube hand in 2019, ratcheting difficulty upward whenever the policy mastered the current setting. The bill was roughly 13,000 years of simulated experience across 64 V100 GPUs and 920 worker machines.

Domain randomization is not free. A policy forced to succeed across a wide band of dynamics cannot exploit the actual dynamics of the actual robot. You pay for transfer in conservatism: stiffer gaits, higher torques, worse energy numbers. A July 2026 paper from Yamamori and colleagues inverts the whole arrangement. Instead of making the simulator match the robot, they shape each joint's closed-loop response with a two-degree-of-freedom feedforward-feedback controller so the physical actuator behaves like the idealised second-order model the simulator assumed, then transfer zero-shot with no fine-tuning across a 7-DOF arm, a wheeled-legged robot and a humanoid.

That is a good idea and it is also an admission. If the fix for the reality gap is to redesign reality, randomization was never solving the problem. It was paying it off in instalments.

What the foal brings that a random policy does not

Rhythmic stepping in mammals is produced by central pattern generators: networks of interneurons sitting entirely within the spinal cord, capable of generating alternating limb movement with no descending command and no timed sensory input. This has been understood for over a century. Isolated spinal cord preparations from newborn mice produce fictive locomotion in a dish. Infant stepping is observed in anencephalic children, which puts the pattern below the brain.

The rest of the prior is not neural at all. Skeletal proportions tuned to the gait. Tendons that store and return elastic energy. Reflex arcs closing at latencies no learned controller can match. Proprioceptive maps already wired to the right muscles. A vestibular system calibrated before birth.

So the foal is not solving the problem we set our robots. It is not searching the space of all mappings from joint states to torques. It has a controller and it is tuning the free parameters against a body it has never used. Sixty-two attempts is a plausible number for calibration. It is an absurd number for policy search from scratch.

Curricula, pretraining and world models are how the field buys priors

Everything that has improved robot learning since 2021 is, structurally, an attempt to install a prior.

Curricula are the cheapest version. Rudin's terrain progression encodes a fact about the ordering of the problem that the agent would otherwise have to discover.

Pretraining is the expensive version. NVIDIA's GR00T N1.7 is a 3-billion-parameter vision-language-action model pretrained on 20,000 hours of EgoScale human video alongside heterogeneous robot data, then post-trained on tens to a few hundred demonstrations per task. The associated synthetic data pipeline generated 780,000 trajectories in 11 hours from a small set of human demos.

World models are the interesting version. Meta's V-JEPA 2-AC was built from under 62 hours of unlabelled robot video from the DROID dataset, then deployed zero-shot on Franka arms in two labs it had never seen, hitting 65 to 80 percent on reach, grasp and pick-and-place, planning in about 16 seconds per step against roughly 4 minutes for pixel-generating alternatives. Sixty-two hours. The number keeps showing up.

Two 2026 papers make the direction explicit. SimDist, at RSS 2026 (arXiv 2603.15759), pretrains encoders, reward models and value functions in simulation, then in the real world updates only the latent dynamics and treats adaptation as supervised system identification. ABD-Net (arXiv 2603.19078, March 2026) goes further and puts the prior in the architecture itself, adapting the inertia propagation mechanism of the Articulated Body Algorithm into a graph network, validated on Unitree G1 and Go2 hardware.

Read those together and the pattern is unmistakable. Nobody is making reinforcement learning more sample-efficient. Everybody is relocating the sample cost out of the task and into a pretraining stage that gets amortised across many tasks. Which is precisely what evolution did.

The fair objection: evolution already paid the bill

Here is the strongest version of the counter-argument, and it deserves better than a dismissal.

The foal's 62 attempts are the visible tip of an invoice settled over something on the order of 350 to 400 million years of tetrapod locomotion. Selection ran a search that was staggeringly sample-inefficient by any measure we would accept: enormous populations, a fitness signal with no gradient information, uncountable rollouts, most of them fatal. The animal did not learn to walk in an hour. The lineage learned it, and the foal loaded the weights. Comparing 62 attempts to ten billion steps is comparing one fine-tuning run to another while pretending only one of them had a pretraining phase.

That objection is correct. It also fails to rescue the position it is normally deployed to defend, for three reasons.

First, the size of the bill is not the interesting property. Its amortisation is. Evolution paid once and the payment is reused, with modification, across every vertebrate that has walked since. We pay per robot, per morphology and often per task. Change a knee actuator and the locomotion policy usually gets retrained. The foal's ancestors did not retrain when the femur got longer.

Second, the artefact is small. A human genome runs to about 3.1 billion base pairs, which at two bits per base is under 800 megabytes uncompressed, and it specifies an entire organism rather than a walking controller. Whatever the locomotion prior is, it is a rounding error inside a file smaller than a mobile game. GR00T N1.7 is 3 billion parameters. If the thing we are approximating is orders of magnitude more compact than our approximation, the problem is representation, not budget.

Third, and least comfortable, evolution's output is not a policy. It is a construction recipe, and development executes it at runtime using the body's own physics as substrate. A frozen weight matrix is a different kind of object. You do not get from one to the other by buying more GPU hours.

Putting the prior back in the body

This is roughly where neurobotics and embodied AI research have gone. Tad McGeer showed in 1990, in the International Journal of Robotics Research, that a properly proportioned two-legged machine will settle into a human-like gait walking down a shallow slope with no motors, no controller and no energy input beyond gravity. The control was in the geometry.

The modern descendants of that idea are series elastic actuators, tendon-driven hands, compliant feet and CPG-shaped low-level controllers with learning confined to the layer above. All of them move computation out of the network and into the mechanism. Newton's stated emphasis on contact-rich behaviour, walking on gravel or snow and handling delicate objects, matters here for a specific reason: body-borne priors are worthless unless the simulator models contact well enough for the body to matter.

And this is where the current pretraining boom has an unresolved problem. A prior about a body has to be a prior about a particular body. A 3-billion-parameter model trained on 20,000 hours of human video holds a prior about bodies in general and none about the one it is bolted into.

The first unrehearsed thing

If the foal is calibrating rather than learning, then 62 attempts and ten billion steps are not two scores on one test. They are answers to different questions, and only one of those questions is the one the field says it is asking.

We report sample efficiency as a property of algorithms. In animals it is mostly a property of what the animal was handed. Genesis makes the engineering version of this point without sentiment: their pitch is that sim-to-real becomes a compute problem. Perhaps it does. But a compute problem is one you solve by spending, and the foal solved it by inheriting, and nothing in the current stack inherits anything from the specific machine it will run on.

The demo humanoids in Shenzhen walk beautifully now. Watch one for twenty minutes and the walking is never what fails. What fails is the first unrehearsed thing.

Tools referenced

NVIDIA Isaac Sim, reviewed here: NVIDIA Isaac Sim review.

MuJoCo, reviewed here: MuJoCo review.

Genesis World, reviewed here: Genesis World review.

Sources

Motor development of stable born healthy foals during the first 24 hours (Journal of Equine Veterinary Science, 2024): https://pubmed.ncbi.nlm.nih.gov/38768873/

Rudin et al., Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning: https://arxiv.org/abs/2109.11978

Tobin et al., Domain randomization for transferring deep neural networks from simulation to the real world (IROS 2017): https://dl.acm.org/doi/10.1109/IROS.2017.8202133

Linux Foundation announces the contribution of Newton by Disney Research, Google DeepMind and NVIDIA: https://www.linuxfoundation.org/press/linux-foundation-announces-contribution-of-newton-by-disney-research-google-deepmind-and-nvidia-to-accelerate-open-robot-learning

Genesis AI, The Role of Simulation in Scalable Robotics and Genesis World 1.0: https://www.genesis.ai/blog/the-role-of-simulation-in-scalable-robotics-genesis-world-10-and-the-path-forward

Meta AI, Introducing the V-JEPA 2 world model and new benchmarks for physical reasoning: https://ai.meta.com/blog/v-jepa-2-world-model-benchmarks/

Yamamori et al., Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning (arXiv 2607.02205): https://arxiv.org/abs/2607.02205

NVIDIA Isaac-GR00T repository (GR00T N1.7 foundation model): https://github.com/NVIDIA/Isaac-GR00T