視障與低視力使用者出門時,可能需要協助看懂周遭環境或讀取文字。
- 社群開發者 Rebel-Gloria 打造了 iPhone 和 iPad app PRTS,內建 MiniCPM-V 4.7 1B,可離線描述拍到的場景、朗讀招牌和檔案,也能用語音追問畫面中的細節。支援的裝置還可用另一套 LiDAR 系統提供口頭方向和節奏震動提示。
- 部分線上功能與系統語音辨識則需要網路。
AI 伴隨問答
針對這篇報導有疑問?可直接點選快捷問題或向 AI 追問背景脈絡與技術細節:
📱 Point your camera. Hear what’s in front of you.
Community developer Rebel-Gloria built PRTS, an iPhone and iPad app for blind and low-vision users, with MiniCPM-V 4.7 1B powering its default on-device visual understanding. 💙
👀 Scene descriptions: Capture an image and hear a concise description of your surroundings.
📖 Read text: Photograph signs, notices, or printed pages and have them read aloud.
🎙️ Ask follow-up questions: Use voice input to ask about details in the current scene.
📳 Obstacle guidance: A separate LiDAR-based system provides spoken directions and rhythmic haptic cues on supported devices.
♿ Accessible controls: VoiceOver, Dynamic Type, physical buttons, Back Tap, and Siri Shortcuts offer multiple ways to interact. A dedicated low-vision mode retains the camera preview, scene descriptions, and text reading.
🔒 Routine scene descriptions and text reading can run offline with the built-in MiniCPM-V model. Optional online features and system speech recognition have separate connectivity requirements.
A thoughtful community exploration of visual intelligence for more accessible everyday experiences. 🛠️
📂 Explore PRTS:
https://t.co/o0Wes81Gr1
🚀 MiniCPM-V GitHub:
https://t.co/iYDxpa52tn
🤗 On-device model:
https://t.co/9pKlAjgQif