Some Android buttons have stable accessibility labels, but the vision model can still miss them or click slightly off.
The client already uses AccessibilityService for actions. Could the current window text and bounds be passed to the agent as optional fallback data when vision is unsure? I don't mean replacing screenshots, just using both when the information is available.
Some Android buttons have stable accessibility labels, but the vision model can still miss them or click slightly off.
The client already uses AccessibilityService for actions. Could the current window text and bounds be passed to the agent as optional fallback data when vision is unsure? I don't mean replacing screenshots, just using both when the information is available.