跳到正文
𝕏 Rohan Paul(x.com)·· 12 天前品質評分48
重點導讀

summary_zh: Robbyant 開源採用 Apache 2.0 授權的機器人觸覺模型 LingBot-Touch,將指尖感測器讀數轉成控制軟體可用的訊號。

AI 伴隨問答

自備 Key 深度分析

針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:

Rohan Paul
Rohan Paul@rohanpaul_ai
·12 天前
𝕏

Looks like a brilliant open source release for robotics.

A great problem of robotics is that a camera can’t tell a robot how hard it’s squeezing or whether a grip is slipping, but a fingertip sensor can.

Robbyant released LingBot-Touch, an open-source model (Apache 2.0) that gives robots a sense of touch by turning fingertip sensor readings into signals their control software can use.

Robbyant trained three same-size encoders (5.46M parameters each) on the same touch data for LingBot-Touch. At 4,000 hours, the JEPA version, which predicts the masked parts in latent space, got 49.3% top-5 on tactile→video retrieval. MAE stayed under 9%. The VAE got 3.1%, worse than it did at 1,000 hours.

I think the reason is the data itself. A finger only presses a few spots, so most of every frame is close to zero, and the readings that aren't zero are noisy. Redrawing that means copying zeros and noise nobody can predict, and a JEPA only predicts the representation, so it can leave both out. The paper names the sparsity and the low signal-to-noise ratio as the reasons they went with JEPA.

The sparsity shaped training too. With 80% of taxels masked from step one, a lot of clips had no contact left to see and the gradients blew up, so they ramp the mask from 40% to 80% over the first 300 steps. Targets come from an EMA encoder at four depths, following V-JEPA 2.1.

To isolate the effect of multi-source data, they hold each sensor's own data fixed and add only the other modalities. With the target sensor's data unchanged, the multi-source model improves on seven of the eight sensor-task pairs, indicating that data from heterogeneous sensors directly benefits each individual sensor.

推文來源 · X (Twitter)在 X 參與討論

讀者討論

理性交流 · 即時討論