Skild AI ships S1, a robot model prompted by one video

Why it matters

If S1's internal results hold in industrial deployments, robot operators could teach new jobs with a video instead of funding hours of teleoperation and another training run.

Deepak Pathak (@pathak2206) and Abhinav Gupta launched S1 on Tuesday, extending Skild AI's robotics foundation model from adapting to unfamiliar robot bodies into learning manipulation tasks from a single video demonstration.

Skild AI said in a thread on X that S1 can watch a person demonstrate a task, translate the demonstration into robot actions and execute the task without fine-tuning its model weights. The examples include brewing pour-over coffee, potting a plant, assembling a kit and frying a pancake, with some sequences running for as long as 10 minutes.

That deployment loop is the center of Skild AI's pitch. Robot policies typically require operators to collect task-specific demonstrations and run another training cycle before deployment. S1 instead places a video demonstration in the model's context window, much as a user supplies instructions to a language model through a prompt.

In Skild AI's technical report, Pathak and Gupta describe pre-training as an outer loop that teaches the policy how to interpret demonstrations. At inference time, the video acts as an inner loop, guiding the robot without changing the underlying weights.

One demonstration, 11 minutes to deployment

Skild AI documented one plant-potting test in which workers began recording a human demonstration at 9:16 p.m. The recording finished at 9:22 p.m., and S1 started attempting the task autonomously at 9:27 p.m. Skild AI put the elapsed time from demonstration to hardware execution at 11 minutes.

The longer demonstrations require S1 to track progress and combine several manipulation skills. For the four featured tasks, Skild AI says the full sequences did not appear in the model's pre-training data. Skild AI separately searched its data for examples of pancake flipping after S1 completed that motion and said it found none.

The distinction matters because robots can appear to learn a new job when the requested behavior closely resembles their training material. S1's strongest claim is that a visual prompt can specify an unfamiliar action or a new arrangement of familiar actions well enough for a robot to carry out the sequence.

Skild AI also showed S1 responding when test conditions departed from the demonstration. In the published examples, the robot retried failed actions, used a cup when a watering can was unavailable and changed its movement when objects were relocated. These demonstrations suggest the policy is interpreting the goal rather than replaying a memorized trajectory, though the evidence and evaluation were produced by Skild AI.

Internal tests show a large gap on unseen tasks

Skild AI compared video-prompted in-context learning with language-prompted vision-language-action policies using matching architectures, compute and datasets ranging from 1,000 to 100,000 hours. On Skild AI's internal benchmark of unseen tasks, the in-context policy reached a 66% average per-step success rate at 100,000 hours, versus 9% for the language-prompted policy.

The advantage was smaller on familiar tasks. At the 1,000-hour training level, the language-conditioned policy scored 53%, while the in-context policy scored 43%. Skild AI says the in-context approach overtook language prompting as the training set grew.

Those results come from two internal benchmark suites, and Skild AI used human intervention to recover from failures so that every step could be graded. The tests therefore measure per-step performance rather than fully autonomous completion rates across every rollout.

Skild AI also post-trained a conventional policy with as many as 2,000 teleoperated examples of a new task. By Skild AI's interpolation, one video prompt delivered performance comparable to roughly 380 post-training examples. The post-trained policy eventually passed S1, reaching 86% with 2,000 demonstrations against S1's 66% single-prompt result.

A commercial test for a heavily funded model

Pathak and Gupta founded Skild AI in 2023 after research careers at Carnegie Mellon University's Robotics Institute. Pathak's work spans machine learning, computer vision and adaptive robot learning; Gupta has focused on computer vision, self-supervised learning and visual reasoning. Their shared bet is that one model can control multiple robot designs rather than binding software to one machine and one task.

Skild AI has raised substantial capital to test that thesis outside the lab. In January, Skild AI announced a $1.4 billion Series C led by SoftBank, with participation from Nvidia's NVentures, Macquarie Capital and Bezos Expeditions. Skild AI said the financing valued it at more than $14 billion. Existing investors Lightspeed, Felicis, Coatue and Sequoia Capital also participated.

S1 is initially being deployed with a limited group of industrial partners, with a wider rollout planned over the next several months. That phased release puts the model's central claim into commercial settings, where changing objects, layouts and operating requirements make repeated data collection expensive.

For Skild AI, one-shot task learning also feeds its data strategy. Faster deployments can create more real-world robot interactions, which Skild AI can return to the pre-training pipeline. S1 turns prompting into both a product feature and a way to collect the next training set.