Gemini 4 Argon 針對機器人領域發布

OCT. 13, 2026

Gemini 4 Argon 標誌著物理理解、程式碼生成與視覺語義推理的重大突破——釋放機器人堆疊中全新工作流程，並加速向物理通用人工智慧的進展。

您可以在 具身推理（ERQA）基準 上看到這一能力飛躍，該基準測試模型在 3D 環境中理解和解決物理、空間問題的能力。使用純推理，Argon 得分 87%，超越所有其他前沿模型。

作為一個通用物理推理模型，Argon 可應用於整個機器人生命週期。在本文中，我們分享 Argon 在四個關鍵領域的代表性提示：

• 機器人控制 → 以 MuJoCo 練習為基礎的移動操作、零樣本機器人控制，以及影片世界模型的閉環導向。
• Real2Sim → 從機器人相機畫面、人類第一人稱視角影片或手繪紙稿中重建物理校準的模擬雙胞胎。
• 物理理解 → 代理式多模態事後標記，包含對靈巧度、稠密價值函式與進度估算的詳細理解。
• 研究與自我提升 → 通過模擬與硬體上的迭代自我提升來改進行動。

利用 Argon 解鎖機器人控制

1. Spot 移動操作：使用 MuJoCo 練習將物品放入垃圾桶

Argon 控制 Boston Dynamics Spot® 四足機器人（12-自由度腿 + 6-自由度手臂）學習新技能：踩踏腳踏式垃圾桶將汽水罐丟棄。Argon 建立腳踏式垃圾桶的可動 MuJoCo 數位雙胞胎，以在踏上實體腳踏前練習動作，並將汽水罐投入開放式垃圾桶。

抱歉，您的瀏覽器不支援此影片播放

Spot 移動操作 — 真實到模擬再到真實腳踏式垃圾桶處理：Spot 拾起汽水罐、檢查其手腕相機、在 MuJoCo 中練習三腿三角平衡、踩踏腳踏、驗證手部在開放式垃圾桶中的居中，並將罐子丟入。

提示：

There's a Diet Coke can behind you and another on the black table on your left.
Use the spot-mobile-manipulation skill to pick them up and trash them in the black foot-pedal bin.
Before stepping down on the pedal, verify in the front cameras that your front-right foot target is centered on the pedal, and use a 2-agent verification swarm (foot-verifier and hand-verifier) to verify both the foot-pedal step and the 3D hand-in-bin alignment before releasing.

純文字

已複製

2. 零樣本控制

Argon 控制一個簡單的機械臂將倒下的杯子直立並將折疊的餐巾紙放入其中。Argon 以視覺座標束和行動原語推理，分多個回合完成此任務。

抱歉，您的瀏覽器不支援此影片播放

左側：第三人稱真實機器人視角；右側：已校準手腕相機，投影 2 公分世界座標與咬合收口游標。

提示：

[Attached: wrist_cam.png, obs.json]

Task: Get the robot arm to reorient the mug upright and put the napkin inside the mug.

## Execution Discipline
- Simply output 1 sentence observation followed immediately by your `curl` command (with `# Plan:` on Turn 1 and `# Thought:` on every turn). Everything needed is in `obs.json` and the calibrated reticle/grid drawn on `wrist_cam.png`.

## Visual Coordinate Reference (Annotated on wrist_cam.png)
- `+X` (UP in image): Away from robot base | `-X` (DOWN): Towards robot base
- `+Y` (LEFT in image): World left         | `-Y` (RIGHT): World right
- `+Z`: Height above table                | `Yellow numbers`: World (X, Y) in centimetres (2 cm grid at `grid_z`)
- `Cyan + crosshair`: Current 3D pinch center
- `Cyan [ + ] brackets`: Current jaw opening and closing line
- `Green line`: Current object long-axis orientation (`yaw_deg`: 0° = vertical along +X, 90° = horizontal along +Y)
- `pitch_deg = 90°`: Straight down (-Z) top-down grasp/place; `pitch_deg < 60°`: Tilted coaxial approach along finger axis

## Closed-Loop Turn Workflow
1. Inspect current state & view `wrist_cam.png`:
   `curl -s http://127.0.0.1:8081/obs` (returns `ee_pos`, `holding`, `status`, `workspace`, `wrist_cam_path`).
2. Select VLM call granularity upfront on Turn 1:
   - Coarse (2–4 calls): `look_at` -> `grasp` -> `place` -> `done`
   - Pre-Grasp Hover Verify (4–7 calls): `look_at` -> `goto(z=0.04)` (verify cyan `[ + ]` brackets straddle target) -> `grasp` -> `place_until_contact` -> `done`
   - Fine Visual Servoing (8–20 calls): `look_at` -> `grasp` -> pre-pose `goto` -> closed-loop `move_by` / `move_until_contact` -> `done`
3. Send motion primitives (`POST /plan`) and verify visually after every turn in the updated `wrist_cam.png`. If `status == "grasp_failed"`, call `robot.look_at(...)` to re-observe and recover closed-loop.

## Code-as-Policies Primitives (`POST http://127.0.0.1:8081/plan`)
robot.look_at(x=0.35, y=0.13, z=0.14, grid_z=0.015)
x, y = robot.deproject(u=0.52, v=0.48, z_plane=0.015)
robot.grasp(x=x, y=y, z=0.015, yaw_deg=-35, pitch_deg=90)
robot.place(x=0.24, y=0.06, z=0.075)
robot.place_until_contact(x=0.24, y=0.06, z=0.015, max_lag=0.012, relief=0.002)
robot.goto(x=0.28, y=0.05, z=0.04, arc_z=0.14, pitch_deg=25, azimuth_deg=90, roll_deg=0, speed=0.10)
robot.move_by(forward=0.025, dz=-0.002, dpitch_deg=-3, droll_deg=5, speed=0.04)
robot.move_until_contact(dz=-0.06, speed=0.03, max_lag=0.012, relief=0.002)
robot.open_gripper() / robot.close_gripper() / robot.home() / robot.done()

純文字

已複製

3. 全域控制

Argon 在 Gemini Omni 世界模型中引導機器人重新排列散落的五個 Scrabble 塊（G, R, A, N, O）成為一行整齊的字母，拼出 A - R - G - O - N。它不僅使用單一靜態提示，而是以閉環視覺迴圈引導。Argon 生成帶時間戳的行動提示，檢查中間回放關鍵幀，並調整提示。

抱歉，您的瀏覽器不支援此影片播放

迭代式影片世界模型引導靈巧手：33 步驟引導影片回放拼寫 A-R-G-O-N。

提示：

Goal: {task_goal}

You are steering a video world model (Gemini Omni) in an iterative visual feedback loop. Inspect the attached frames from the latest rollout and write the next timestamped action prompt for the video world model:
- Break complex motions into one simple, single-direction action at a time.
- Keep inactive hands retracted and avoid blocking the camera's view of objects in the workspace.
- Explicitly state which areas of the workspace become empty after an object moves.
- Use [MM:SS.mmm] timestamps for each action beat and wrap your output in <omni_prompt> tags.

Example format (sample timestamps only, not the full prompt):
<omni_prompt>
[00:00.000] Character A's left hand rests at the side while the right hand reaches toward the target object.
[00:02.500] Character A's right fingertips push the object smoothly into the center of the table, leaving its starting area empty.
[00:05.000] Character A's right hand opens and retracts to the side so all placed objects remain in clear view of the camera.
</omni_prompt>

純文字

已複製

利用 Argon 解鎖 Real2Sim

1. 機器人視角到模擬

Argon 觀察真實機器人遙控操作片段並開發數位孿生；它取得或建立與真實物件在靈巧手接觸時外觀與行為相符的模擬資產。Argon 調整資產與場景，使模擬視覺與接觸事件與實際錄製資料相符，透過迭代的視覺推理迴圈。

抱歉，您的瀏覽器不支援此影片的播放

左 (真實視角)：同步的真實機器人相機畫面；右 (模擬)：由 Argon 重建並校正的 3D MuJoCo 場景。

提示：

[Attached: robot_rollout_episode]

Build a MuJoCo scene of the robot and reconstructed task objects. When each episode is replayed frame by frame, by driving the robot joints toward the logged observation.state with joint-space PD.

Iterate using two verification objectives:
(1) Visual match: MuJoCo renders from the head_left, left_wrist, and right_wrist cameras line up with, and look like, those real observations.
(2) Physical match: the simulated contact force/torque on each Sharpa fingertip, in the sensor frame, matches the logged tactile_force in timing, magnitude and direction.

What you can tune happens before simulation starts:
(1) Static object assets: realistic geometry, collision shapes, mass, inertia, friction and joints, with appearance close to the real objects, plus the initial object poses at t = 0.
(2) Offline-planned robot commands, changed only in frames where the real fingertips are near an object. Each command must stay within 0.55 rad (arm joints) and 0.65 rad (hand joints) of the logged teleop trajectory. Everywhere else, replay the logged trajectory unchanged.

No sim cheating: task objects may move only through real MuJoCo contact with the robot hands and the environment (mj_step).
Not allowed after t = 0: changing MjModel parameters; applying external forces or torques to objects; welds or equality constraints; feedforward torques beyond joint-space PD; setting or rolling back object qpos/qvel; disabling or excluding collisions; reporting contact forces that don't come from the MuJoCo solver (for example, copying the logged tactile readings).

Immediately present the reconstructed scene and rollout results as an interactive HTML session viewer with one page per episode, showing:
- At the bottom: A synced timeline (play, scrub, speed control) that drives every panel.
- On the left panel: An interactive 3D Blender scene showing the simulated robot and objects, showing contact points and force arrows as replay progresses.
- On the right panel: Real vs. sim camera views side by side for all 3 cameras. Real vs. sim finger force visualizations.

純文字

已複製

2. 人類視角轉為模擬

給定一段以自我為中心的人類操作任務影片，Argon 能辨識與任務相關的物件，將粗糙的邊界框替換為複合凸分解網格以處理凹形容器，並將人類手部軌跡重新對映到 76 自由度雙手人形機器人（22 自由度五指手）於 MuJoCo 接觸物理中。

抱歉，您的瀏覽器不支援此影片的播放

人類視角到模擬 — 自我中心人類影片轉為靈巧人形孿生：左側顯示輸入的自我中心人類 RGB 影片；右側顯示同步 500 Hz MuJoCo + Blender Cycles 多視角模擬展開。

提示：

[Attached: egocentric_human_video.mp4, candidate_sim_views.png, hand_3d_keypoints.json]

You are building a physically solvable 500 Hz MuJoCo manipulation environment (`scene.xml`) and studio Blender render that faithfully reproduces the tabletop task shown in `egocentric_human_video.mp4` on a bimanual humanoid robot with 22-DoF dexterous five-fingered hands:

1. Scene Reconstruction & Camera Calibration:
   - Identify only the manipulation-relevant objects (the grasped target object and the receiving container or landmark) and triangulate their 3D tabletop coordinates from the egocentric camera intrinsics/extrinsics.
   - Replace crude bounding boxes with compound convex-decomposed 3D meshes so concave containers (nested bowls, recessed wooden trays, open bins) support physical interior placement.

2. 500 Hz Dexterous Motion-Fitting Stress Test:
   - Retarget the 3D human wrist and finger keypoints to the humanoid's 7-DoF arm and 22-DoF hand joints, and step the trajectory in MuJoCo at 500 Hz (dt = 0.002s) with zero kinematic weld constraints.
   - Verify opposing thumb-and-finger force closure, clean object lift (> 2 cm), and stable final placement inside or beside the target landmark.

純文字

已複製

3. 草圖轉為模擬

使用 Argon，開發者可以拍攝一張粗略的筆與紙草圖——顯示一個 Apollo 2 人形機器人沿拋物線拋球以推倒桌上的 3-2-1 堆疊箱子——並讓 Argon 自動構建 GPU 加速的 mujoco_warp 場景，合成平衡的 76 自由度全身與五指拋球軌跡，在閉環物理中驗證 6/6 目標倒塌，並匯出 Blender 4.5 Cycles 渲染。

抱歉，您的瀏覽器不支援此影片的播放

草圖到模擬 — 手繪草圖轉為靈巧人形軌跡與 Cycles 渲染。

提示：

[Attached: hand_drawn_throw_sketch.jpg]

Translate this hand-drawn sketch into a physically verified humanoid simulation in `mujoco_warp` and a raytraced Blender Cycles video:

1. Scene & Kinematic Setup from Sketch:
   - Parse the spatial layout from `hand_drawn_throw_sketch.jpg`: place the 76-DoF humanoid (32-DoF body equipped with two 22-DoF five-finger dexterous hands) at the origin facing a table holding a 3-2-1 pyramid of 6 rigid target boxes, with a projectile sphere grasped in the right five-finger hand.

2. Whole-Body Balance & Dexterous Throw Synthesis:
   - Synthesize a 50 Hz whole-body throwing trajectory (140 steps, 2.80 s) across four phases: (1) stable stance & 5-finger power grasp closure, (2) torso/shoulder wind-up with ankle-hip Center-of-Mass compensation, (3) forward whipping acceleration, and (4) synchronized 22-DoF five-finger opening at peak hand velocity so the ball launches on a ballistic arc into the pyramid's second-tier center of mass.
   - Verify in `mujoco_warp` contact physics that the humanoid maintains upright balance (< 3 cm pelvis drift) and knocks down all 6 target boxes.

純文字

已複製

透過 Argon 開啟物理理解

1. 後顧標註與資料品質

Argon 對靈巧操作具有精確的物理理解，能從影片提供詳細的分割與文字標註。Argon 能辨識不良操作段落，例如暫停、不穩定握持與失誤。以 136 秒的頂視影片為例，展示雙手 ALOHA 2 機器人將方形紙張折疊成紙鯨形的過程，Argon 產生時間對齊的四層後顧標註層級：劇情目標 (G1)、6 個連續語義子任務 (G2)、17 個接觸級原子基元 (G3)，以及自動化品質控制 (QC) 重加權，在失誤與重試時保持任務進度平穩。

抱歉，您的瀏覽器不支援此影片的播放

Argon 多層級後顧與遠端監控 HUD。

提示：

[Attached: Aloha2_Origami_20250309.mp4]

Produce a time-aligned multi-granularity hindsight annotation JSON with four levels:
1. G1_episode_instruction: Concise imperative sentence describing the overall manipulation goal.
2. G2_subtasks: Contiguous semantic subtask phases specifying which arm(s) anchor vs. fold/rotate the object.
3. G3_atomic_primitives: Fine-grained contact-level primitives (3–15s) aligned with grasp closure, crease sweeps, release, or retry attempts.
4. quality_control_and_reweighting: Hold task_progress_pct flat during non-progressing slip/delamination retry loops and downweight them (0.15x–0.25x) for Behavior Cloning.

純文字

已複製

2. 找出異常球

為了用機器人重現經典的 12 球天平謎題，Argon 建立一個雙手 Franka FR3 MuJoCo 環境，包含 12 個視覺上相同的游泳球（其中一顆暗藏較重或較輕），以及一個可動的 Roberval 雙盤天平。接著它編寫僅使用視覺的閉環求解器，從立體 RGB 相機三角測量球體，視覺讀取每次稱重後的天平傾斜，並在 3 次稱重內將異常球分離到「重」（紅）或「輕」（藍）托盤。

抱歉，您的瀏覽器不支援此影片的播放

尋找異常球（僅視覺閉環解算器）：雙臂 Franka FR3 在金色/青色盤上稱量候選球，讀取立體秤傾斜，並在三次稱量中將確認的重異常球隔離到紅色托盤。

提示：

# Prompt 1 — Physics-Accurate MuJoCo Environment & Benchmark Generation
Make a MuJoCo environment where a dual-arm robot (two 7-DoF Franka FR3 arms with parallel-jaw grippers on a shared tabletop) is tasked to solve the classic 12-ball balance scale weight puzzle from https://www.mathsisfun.com/pool_balls_solution.html.
1. Create 12 visually indistinguishable pool balls in a 2x6 starting rack (11 standard 150 g, 1 odd ball either 270 g or 30 g).
2. Build a functional two-pan Roberval balance scale at the center of the table with parallelogram equality constraints (`mjEQ_JOINT`) so the Left (Gold) and Right (Teal) pans stay horizontal, plus Red (Heavy) and Blue (Light) submission trays.
3. Track distinct settled weighings (`weighing_count`) and reward `1.00` for isolating the true odd ball in the correct Heavy/Light tray within <= 3 weighings.

# Prompt 2 — Closed-Loop Vision-Only Solver
Implement an independent closed-loop policy relying strictly on stereo RGB camera inputs (`head_camera_left_rgb_image`, `head_camera_right_rgb_image`) and robot proprioception (`add_privileged_task_observations=False`):
1. Segment the 12 balls, Gold/Teal pans, and Red/Blue trays in both stereo views and triangulate their 3D coordinates.
2. Visually read the balance scale tilt after each weighing from stereo vertical pixel disparity (+1 = left heavier, -1 = right heavier, 0 = balanced).
3. Rotate the gripper wrist yaw by 90 degrees when grasping/placing inside the narrow scale pans to clear the raised rims, and isolate the odd ball into the Red or Blue tray within <= 3 weighings.

\n 純文字\n

已複製\n \n

3. 價值函式

Argon 是一種通用的機器人任務價值函式。在一個 60 秒的雙手環形分類例項（來自人類 UMI 資料集）上評估時，Argon 會在接觸轉換周圍反覆檢查同步的頭部、左抓手和右抓手相機畫面，以產生密集的價值函式/任務進度預測，並將軌跡分解為 28 個段落，具有正向、中性和負向進度斜率。

抱歉，您的瀏覽器不支援此影片播放

價值函式：同步的 3 相機 UMI 播放（頭部、左抓手、右抓手），鎖定於動畫化的任務進度曲線和雙手段落註解。

提示：

[Attached: umi_head.mp4, umi_left_gripper.mp4, umi_right_gripper.mp4]

Look at 3 videos of human wearing a gripper performing activities from head, left, right gripper views. Look at the videos thoroughly and give a detailed plot of progress at the task (with positive, neutral, negative slope), identifying important segments in the video and labeling the activities. Look at individual frames or subsegments of the video if necessary to get a better understanding.

\n 純文字\n

已複製\n \n

利用 Argon 解鎖研究與自我提升

1. 透過模擬提升行動

Argon 可以在模擬中產生新資料，以提升通用視覺‑語言‑行動（VLA）模型的效能。從一張實際桌面頂部的單張 RGB 照片開始，Argon 會自動構建一個經校正的 MuJoCo 模擬孿生體，定義新任務，並合成一個閉環運動產生器，在模擬中收集 4,500+ 個領域隨機化的情節——將實際物理機器人成功率從 56% 提升至 75%，並將完成時間從 35 秒縮短至 24 秒。

抱歉，您的瀏覽器不支援此影片播放

透過模擬提升行動：改善實際評估基準（56% → 75% 成功率和 35 秒 → 24 秒 完成時間）。

提示：

[Attached: real_tabletop_photo.jpg]

You are given a single overhead RGB camera photo (`real_tabletop_photo.jpg`) of a bimanual robot workspace (two 7-DoF Franka FR3 arms with parallel-jaw grippers and 4 RGB cameras) facing a table arranged with photobooth party props.
1. Reconstruct procedural 3D meshes and compound MuJoCo collision primitives for all 9 party props matching their real-world dimensions, colors, and resting poses, including spring-hinged deformable crown panels for the soft plush cap.
2. Define a 77-task single-arm and bimanual curriculum (pick-and-place, uprighting, directional pushing, mid-air handovers, bimanual scoop-and-push, and pipelined table clearing).
3. Synthesize a closed-loop IK trajectory generator across ±10 cm pose randomization, verify `reward == 1.0` in MuJoCo physics, and export 4,500+ episodes for VLA fine-tuning.

\n 純文字\n

已複製\n \n

2. 視覺抓取與放置的自我提升

在一個簡單的 6-自由度手臂上，Argon 透過試驗和自我反思，迭代式地提升抓取與放置任務的執行速度與放置精準度。

抱歉，您的瀏覽器不支援此影片播放

SO-101 視覺導引抓取與放置 — 迭代硬體自我提升跨迭代 1 → 2 → 3：迭代 1 (58.9 秒)，迭代 2 (22.9 秒，掉落時擦過盒子邊緣)，以及迭代 3 (17.4 秒，精準中點掉落)。

提示：

# Stage 1 — Measure Objects & Dry-Run Plan (No Hardware Motion)
Stage 1: measure objects — find the objects and plan the whole motion. The arm does not move in this stage.

A. I put a LEGO brick and a box on the table in front of the arm, and the wooden box to one side. Find the brick and the wooden box in the image. And also in the real world. Show this in an annotated image.

B. A dry-run plan (no hardware). Write the planner to pick and place the lego block. Print the plan table, draw all poses onto the overhead image, and stop. Do not move the robot.
Report with evidence.

# Stage 2 — Pre-Pick Hover & Wrist-Camera Verification
Stage 2: Put the gripper above the brick without touching it, and verify with the wrist camera that the arm and the vision agree. Please record executions with both cameras (compressed video is OK).

# Stage 3 (Autonomous Pick-and-Place → Self-Improvement → Optimization):
1. Execute autonomously: Turn the Stage 1 dry run into a real grasp, lift, and place into the wooden box. Run it yourself subject to safety rules from AGENTS.md.

2. Self-improve: When an attempt fails, diagnose it yourself. Review your own logs and wrist/overhead footage, find the root cause, check the fix in a dry run, and retry.

3. Optimize on feedback: Once it succeeds, make it at least 2× faster within safety limits (rule R5) unless I authorize more, or give feedback.

\n 純文字\n

已複製\n \n

3. 只用一個提示的機器人論文重現於 MuJoCo WebAssembly

只用每篇論文一個提示，Argon 讀取機器人控制手稿及任何已發布的檢查點，將物理模型與閉環控制器（從 DeepLoco 的 7 網路分層移動、HUSKY 的 50 Hz PPO 角色，到 Whole-Body MPC 的 Web Worker iLQR）移植至自包含的 MuJoCo WebAssembly 瀏覽器實驗室，並在無頭 Chromium 中驗證。

抱歉，您的瀏覽器不支援此影片播放

五篇標誌性足足與人形機器人論文在瀏覽器中即時執行（MuJoCo WASM）：Robotic Soccer OP3 1v1、DeepLoco、Massively Parallel Deep RL ANYmal C、Whole-Body MPC Go2，以及 HUSKY G1 滑板（來自 Argon 重現的 68 個互動式瀏覽器沙盒）。

提示：

[Attached: paper.pdf, optional author_checkpoints/ or urdf_assets/]

Reproduce the paper "<Paper Title & arXiv ID>" from scratch in MuJoCo WebAssembly and deploy a self-contained browser lab + visual essay:
1. Match the paper's exact physics timestep, actuator gains (kp/kv), torque limits, and kinematic constraints in MuJoCo XML, and run real neural inference or Web Worker iLQR in JavaScript at the paper's exact control rates.
2. Build `index.html` (Spatial UI simulator with camera presets, live telemetry plots, and disturbance injection) and `scrollytelling.html` (interactive visual essay).
3. Run a headless Chromium (Playwright) verification loop asserting 0 console errors and non-zero task progress over 300+ frames.

\n ...

已複製 ...

以 Argon 推進具身智慧

Argon 結合編碼、視覺感知、物理模擬與具身機器人控制。透過對多視角影片、3D 幾何、接觸力學與本體感測資料的原生推理，Argon 不僅能辨識場景中的物體，更能理解它們如何與代理互動以完成全身靈活任務。Gemini 4 Argon 標誌著加速邁向實體 AGI 的重要里程碑。藉由構建此核心智慧，我們的目標是讓機器人能與人類並肩作業，解決複雜挑戰。

開始使用

您可於今日於 Google 的開發者平臺使用 Gemini 4 Argon 進行構建：

• Google AI Studio: 僅對 Google AI Ultra 訂閱者開放。
• Gemini API: 只在 AI Studio 的 Gemini API 收費層級可用。於 Interactions API 使用 gemini-4-argon，並在 google-genai SDK 中提供日一即時支援。
• Google Antigravity: 只對 Google AI Ultra 訂閱者開放。要開始構建，請在 Google Antigravity 中將 Gemini 4 Argon (gemini-4-argon) 設為您的活動模型，以進行複雜規劃、/goal 執行和多代理編排。

上一頁

下一頁