Shopify just released an AI shopping agent that shops your store through UCP — has anyone tested whether it can actually check out on theirs?

Shopify Discussion
, , , ,
1

Today Shopify published a reference implementation of an AI shopping agent, built on Anthropic’s open-source commerce-agents blueprint (GitHub - Shopify/claude-for-commerce-examples: Shopify implementations of Anthropic's commerce-agents blueprint: a storefront shopping agent over UCP and Sign in with Shop, and a merchant agent over the Admin API · GitHub). It searches your catalog, builds a real cart, and hands the buyer to your checkout. It talks to the store through UCP — the /.well-known/ucp endpoint every Shopify store now serves.

That changes the question. It’s no longer “will AI agents shop my store someday.” It’s “does my store work when one tries today.”

From walking stores this week, here’s where agents actually get stuck — none of these look broken to a human:

  1. Add to cart is a button with no accessible name (icon only, no aria-label). A human clicks it. An agent can’t find it.
  2. Price rendered by JavaScript after load, so the product page reads as “no price” at the moment the agent looks.
  3. Cookie/consent wall that swallows the first click — the agent never reaches the product.
  4. Variant must be selected before the cart button works, but the page doesn’t say so anywhere a machine can read.
  5. Password-protected or “coming soon” mode still on for a live store’s MCP endpoint.

None of these are visible in your analytics, because a bot that bounces leaves no session.

Two honest questions for this board:

  • Has anyone here run the reference agent against their own store yet? What happened?
  • If you want a plain-English report of where an agent stalls on your store, reply with the store and I’ll run the walk on a handful of them in this thread, no charge. Not selling anything here — I want the failure patterns.
2

Strong point. I’d add one test before treating this as a new traffic source: separate machine readability from commercial completion. Can the agent identify the correct variant, preserve market/currency, understand shipping constraints, create the cart, and hand off a checkout that a human can actually complete? Log each transition and compare agent-created carts with human sessions by product, market and checkout outcome. A successful UCP response is an observable fact; a successful purchase path is a different result. I’d also keep a human control run because an agent can fail on semantics while the storefront remains perfectly usable for shoppers—or the reverse.

3

Agreed, and that’s the line I try to hold: a 200 from the UCP endpoint scores nothing on its own. What gets graded is the path — find the product, read price, currency and availability, add the right variant to a real cart, reach a checkout a human could finish — with every transition logged.

What I don’t do yet, and you’re right to name it: a human control run on the same product, and explicit checks that market/currency survive the handoff and that shipping constraints are understood. Those are the next checks. The variant one is partly there already — “must pick a variant before the cart button works” is one of the most common stalls I see.

And yes to the other half: a store can be perfectly usable for people and still fail an agent on semantics, or the reverse. That’s why the report never says “your store passed” from a UCP response alone.

4

Good list, and the pattern under all of it is the same: an agent treats your product data as the literal state of the shop. A human reconciles the button with the fine print. An agent acts on the field it can read, and moves on.

A few more stalls I keep seeing, all invisible to a person:

Two prices in the markup with nothing saying which is current. Sale, compare-at, per-variant. The human eye picks the big bold one. The agent sees two numbers and either guesses or backs out.

Availability that disagrees with itself. The button says Add to cart, the JSON-LD says OutOfStock, or the reverse. The agent trusts the field, not the button, so it either skips a sellable product or tries to buy one you cannot ship.

Variant-level stock hidden behind a parent that reads “in stock”. Buyer asks for the medium in blue, that variant is out, but only product-level availability is exposed. The agent adds it and checkout fails at the last step.

Market and currency mismatch. A person gets redirected to the right market by a script. The agent lands on the default .com, reads USD, and the answer it gives a UK buyer is wrong before it starts.

And the one that hides upstream: the storefront loads fine for people, but the UCP or crawl endpoint is blocked at the CDN. Worth checking /.well-known/ucp resolves and is not behind a bot rule.

The fix for all of it is coherence. The button, the visible text and the structured data need to say the same thing, because the agent only believes the machine-readable one.

Happy to compare notes on failure patterns.

5

Ran it against our store today. The /.well-known/ucp is there and the MCP endpoint at /api/ucp/mcp answers - 13 tools, search_catalog through complete_checkout.

One thing that surprised us: every catalog call needs meta.ucp-agent.profile and the store fetches that URL before returning anything. Unreachable gives profile_unreachable, wrong content type gives profile_malformed. So the UCP path never touches your DOM. Your aria-label and JS price stalls are the browsing agent, different failure mode.

6

You’re right, those are two separate test paths. UCP tool calls bypass the theme DOM, so aria labels, consent overlays, and delayed JS prices only affect browser-driven agents.

What I’d test today:

  • Host the agent profile on a public HTTPS URL, return the expected content type, and make sure CDN or bot rules allow Shopify to fetch it.
  • Run search_catalog, product detail, variant selection, cart creation, and checkout handoff as separate calls. Save each response and cart ID.
  • Test one product with multiple variants and one sold-out variant. Confirm price, inventory, market, and currency stay correct through checkout.
  • Repeat from a non-default market. Reaching complete_checkout is not enough if the buyer lands in the wrong currency or shipping is unavailable.

That distinction should be explicit in any audit: UCP checkout path versus browser-agent storefront path.

7

This is the list I’d have written after another month of walks, and you got there first. The one that bites hardest in practice is your fourth: the storefront redirects a human to the right market, the agent lands on the default .com and quotes USD to a UK buyer — nothing errors, the answer is just wrong. Same with availability: the agent trusts the JSON-LD over the button every time.

We check /.well-known/ucp on every walk now and your “blocked at the CDN” case is real — the storefront is fine for people and the agent path is dead behind a bot rule. Happy to compare failure patterns; I’ll send what the walks are seeing across stores this month.