The mouse pointer hasn't meaningfully evolved since 1968. DeepMind wants to change that.
Their approach: equip the pointer with Gemini's understanding capability. What you point at, the AI knows what it is. What you say, the AI executes in that context.
Four interaction principles: maintain flow across apps, show and tell (point + speak), use "this" and "that" naturally, turn pixels into structured entities.
Already being integrated into Chrome and Googlebook laptops as of May 12.
Main sources: