Visa tested 22 AI agents on buying three everyday products (a TV, a laptop and a hair appliance), with 15 separate shopping attempts per agent. The shopper was also an AI agent. It had a list of required attributes, such as compatibility with a game console, a 30-day return window or Dolby Vision support, but volunteered only some of them, so the shopping agent had to ask for the rest.
Overall success was about 57%. A Google Gemini model and a Meta model tied for first at 82%, a Claude agent came third at 76%, and an OpenAI agent ranked sixth at 64%. The five most expensive models averaged 67% success and the five cheapest averaged 50%. The most common failures were buying the wrong product, failing to recommend anything suitable, and not asking about missing requirements.
Rubail Birwadker, Visa's global head of growth, told Fortune that neither the technology nor consumer trust is ready yet, though he expects shoppers to hand off purchases eventually. Visa published each agent's results on a public website.