Tracking AI-driven profit - 8 checks every Shopify merchant can do now

Hey Community!

We’re hosting an AMA about one of the biggest shifts happening in ecommerce right now: AI is moving from helping shoppers research products to helping them discover, compare, and increasingly buy inside AI conversations.

Shopify says AI-referred orders grew nearly 13× year over year in Q1 2026, while referral sessions from AI chatbots grew more than 8×. That makes the opportunity real — but it also creates a new question for merchants:
Are you ready not only to sell through AI, but also to measure whether AI-driven growth is actually good for your business?

Here’s an 8-point checklist you can complete in under 30 minutes. Bring your results — and the questions you uncover — to our live AMA on 11am - 1pm, August 25, ET.

Note: This post is for all Shopify merchants. The examples below use Shopify’s current Agentic Storefronts, Sidekick, and app-extension direction; you do not need any specific third-party app to follow the checklist.

QUICK WALKTHROUGH: Audit your agentic-commerce readiness in 5 steps

Step 1: Check where AI can already sell your products

Open Agentic Storefronts in Shopify Admin and review the AI channels available to your store. Look at which surfaces are active and which ones are producing orders, sales, or conversions.

Review which AI channels are available for your store and where your products can already be discovered or purchased.

Step 2: Find what AI understands — and what it misses

Review your product data and the queries where your products do or do not appear. Start with your top 10 products: titles, descriptions, variants, availability, specifications, and attributes.

Complete product information gives AI more context to understand when a product is relevant to a shopper’s request.

Step 3: Compare quality, not just volume

Pick one AI channel or AI-referred traffic segment and compare orders, conversion rate, AOV, and refunds with your normal store baseline.

Step 4: Add the profit layer

Revenue answers “How much did I sell?” Profit asks “What did I keep after the costs required to generate and fulfill those sales?” Make sure the costs that matter to your business are represented somewhere in your reporting workflow.

Example: An AI channel generates $10,000 in sales. At first glance, that looks great. But after $3,500 in product costs, $1,200 in fulfillment and shipping, $2,000 in advertising, and $800 in refunds, the actual contribution is much smaller.

Two channels can generate the same revenue but create very different profit.

An AI-generated order is still only a good order if the unit economics work.

Step 5: Ask one business question — then one follow-up

Try Sidekick with a real question, then ask the follow-up that helps you decide what to inspect next. This is the difference between using AI as a search box and using it as an operating assistant.


Bonus tip: treat AI traffic like a new sales channel

If ChatGPT, Copilot, Gemini, AI Mode, or another AI surface starts sending traffic or orders, don’t judge it only by revenue. Use the same discipline you would apply to a new paid channel or marketplace.

  • Traffic and orders generated

  • Conversion rate versus your store baseline

  • Average order value

  • Products being recommended and purchased

  • Refund / return behavior

  • Contribution to profit after the costs that matter to your business

What does “agentic reporting” mean?

A useful way to think about AI reporting is as a progression:

Level AI does Merchant value
1. Retrieve Find the relevant metric or report Faster access to data
2. Explain Put the number in context Understand what the number means
3. Recommend Suggest the next thing to inspect Shorten the path from insight to decision
4. Act with confirmation Prepare or stage a change, then ask you to confirm Turn analysis into action without giving up control

Most merchants do not need a fully autonomous AI agent today. The real opportunity is reducing the repetitive work between finding the right data, understanding what it means, and deciding what to do next.

Example: what connected reporting inside Sidekick can look like

The screenshots below use one connected app only as an example. The broader point is Shopify’s Sidekick App Extensions model: apps can make relevant data or guidance available in the same merchant conversation.


Bring your questions to the AMA!

Completed the checklist? Found a gap in your data? Not sure what AI should be allowed to do for your store? Bring a real example to our live AMA on on 11am - 1pm, August 25, ET.

Common questions we’ll cover:

  • What does agentic commerce actually change for a normal Shopify merchant?

  • How do I know whether AI-driven traffic is valuable — not just new?

  • Which product data matters most when AI agents decide what to recommend?

  • What metrics should I track for AI shopping channels?

  • How should I measure profit from AI-driven sales?

  • Where does Sidekick fit into day-to-day reporting and operations?

  • What can third-party apps safely expose to Sidekick?

  • What should AI be allowed to recommend or prepare — and what should always require merchant confirmation?

See you at the AMA!

I love this topic!! :laughing:

Thanks so much, Ellie! :blush: Really glad you enjoyed the topic. Looking forward to the discussion in the AMA!

Two different pipes get mixed up in this conversation, and they need different numbers.

Catalog / Agentic Storefronts is how ChatGPT product cards and in-chat checkout get your products. That is fed by admin attributes, variants, stock, GTIN. If a field is empty or stale, you get filtered out silently. An llms.txt file does nothing here.

Citations are the other pipe. ChatGPT-User and OAI-SearchBot fetch your pages when someone asks a question. GPTBot is training. Seeing GPTBot in logs is not evidence you will be named today.

On attribution: a lot of AI-referred sessions still land as Direct. So AI revenue in a dashboard can be both under and over counted. Treat it like a new channel with a messy UTM story. Compare refund rate and contribution, not just order count.

Sidekick is useful as a map. It is not the source of truth for whether a channel is profitable. Check the actual order, then the cost.

AU stores: open Settings > Apps and sales channels and see whether Agentic Storefronts is even on. A lot of us have not opened it.

Thanks Tim — that makes sense, good point on separating product discovery from citation crawling. Appreciate the clarification!