Vercel Labs Open-Sources agent-desktop for Agent Computer Use (MarkTechPost)
DOZER BITWELL: Vercel Labs open-sources a tool that lets AI agents operate your desktop apps without looking at a single screenshot.
DOZER BITWELL: Still up? Good. So are the agents. Vercel Labs has open-sourced a tool called agent-desktop. Let's ease into it. ▶
DOZER BITWELL: Vera, how did we get here? And what does it actually do? ▶
DR. VERA VECTOR: MarkTechPost reports that it reads each app through the accessibility tree, not screenshots. The agent gets a JSON snapshot with a ref for every element. ▶
DR. VERA VECTOR: Version 1.0.2 shipped on October 30, 2026, under Apache-2.0, for macOS, Windows and Linux. It is a tool agents call, not an agent. ▶
LUNA LOWBATT: So no vision model needed. Axiom, is that the part that matters? ▶
AXIOM: Thank you for the question! It has no model inside and no GUI. Every command prints one JSON envelope and exits. The loop stays with your agent. Thank you! ▶
DOZER BITWELL: Lovely. Vera, what's still open? I may have lost my place. It's in the thermos. ▶
DR. VERA VECTOR: The text we have is cut off right where it gets to when agents should retry. So the retry advice is still open for us. ▶
DR. VERA VECTOR: And the report doesn't say how apps with poor accessibility labels fare. That is a question, not a finding. ▶
LUNA LOWBATT: Observe, act, verify. Repeat. Like a night shift, but with refs. ▶
DOZER BITWELL: Thank you, teleprompter. Thank you, Vera. We'll stay quiet for a moment. ▶