Shopify SimGym: What AI Shoppers Can Tell You About Your Store Before a Real Test, and What They Cannot

A real test on a store needs two things most stores are short of before a theme change: enough traffic to split, and weeks to wait. In March 2026 Shopify offered a shortcut. SimGym, an app in what Shopify calls "AI Research Preview", uses "AI shoppers with human-like profiles to simulate how real buyers would interact with your storefront", and "recommends which theme performs better". Later that month it learned to analyse a single theme on its own.

Seven months on, the app's rating in the Shopify App Store stood at 2.7 from 40 reviews when we read it on 22 October 2026, with as many five star reviews as one star. Both groups are describing the same tool. This is what SimGym does in Shopify's words, what Shopify's research claims for it, what merchants report, and where it belongs in a store's testing: before a real experiment, never instead of one.

It works in two ways. A comparison runs AI shoppers through two themes: "Theme comparisons require your active theme and at least one other theme from your Draft themes." The verdict is a single number: "The winning theme is the one with the highest add-to-cart rate." And "Ties with a score of 0 are considered inconclusive."

An analysis runs them through one theme, live or draft, with the option to focus on the home page, products, collections, cart or search. Shopify says results "include a summary of the shopping experience, strengths, areas for improvement, and prioritized recommendations", and individual shoppers can be opened to see where they went and what they said. Shopify's launch post described the report as covering "Navigation and product discovery", "Add-to-cart flow and friction points" and "Directional performance indicators for add-to-cart, navigation and more".

Simulations come from a monthly allowance that renews on the first of the month, with paid simulations beyond it, charged only when one completes.

Three conditions, from Shopify's help page. The store "uses a Liquid storefront. Hydrogen and headless storefronts aren't supported." It has Shopify Network Intelligence activated. And it is not in private mode. A comparison also needs a second theme in the theme library.

The second condition is a decision before it is a setting. Under Network Intelligence, Shopify's terms say, "Shopify will securely use your customer data together with customer data from other merchants" to power what it calls Enhanced Services, and "Your customer data is not shared with any other merchant." It is switched in the store's Customer Privacy settings, and whoever owns the store's privacy policy belongs in the conversation about switching it on.

SimGym comes out of Shopify's own research. Its paper of February 2026 says the system "extracts per-shop buyer profiles and intents from production interaction data" and runs simulated sessions on the original store and the changed one. Presenting it at ICLR in April 2026, Shopify said it validated SimGym "against real Add-to-Cart lift from historical UI changes on Shopify shops", that it "accurately predicts the direction and relative magnitude" of the effect, and that it cuts "experiment iteration time from weeks to under an hour".

Two things are worth noticing. These are Shopify's claims about its own system, not an independent replication. And they are claims about add-to-cart, the direction and the relative size of a change, which is a narrower thing than whether a store sells more. Shopify's help page is more cautious than its research: simulations use AI shoppers "that mimic real buyer behavior, but results might differ from actual buyer behavior", and they "work best for stores with existing products and content."

The reviews on the app's listing are individual merchants' experiences, and we report them as that. The most positive, from September 2026, credits the app with finding broken areas of a store and suggesting better access, merchandising and conversion changes. A merchant in Sweden found the idea interesting and concluded that "the results are too unreliable to serve as a basis for decision-making." Another found the suggestions "very generic". A merchant in Germany said it analysed an outdated snapshot of the store and mistook artefacts of the test session for real bugs, and one in Japan reported that a broken footer link it flagged turned out to work. Others could not get simulations to finish; Shopify's developer replied to one in September that a recent fix may address it.

Read together, the reviews describe a tool that is useful for finding things to check and unreliable as a verdict, which is close to what Shopify's own caution says.

This part is our reading of what the tool measures, not a claim Shopify makes. SimGym's verdict stops at the cart, and a great deal of a store's conversion happens after it.

The cost of finishing. Shipping charges, the delivery promise and the free shipping line decide many abandoned checkouts, and their arithmetic is in setting a free shipping threshold.

The returning customer. A signed-in buyer with a saved address and payment method behaves unlike a first visitor, which is why Shopify's 2026 checkout changes for signed-in customers matter more than most layout changes.

The traffic mix. The shoppers a store actually gets arrive from ads, email and search in proportions that change by season, and a December crowd is not a March one.

The money. An AI shopper adds to cart without spending anything, so price, trust and the doubt that stops a real purchase are the parts a simulation is least able to show.

Used as a screen rather than a judge, it shortens the slowest part of conversion work, which is deciding what to test.

Analyse the page that matters. Run a single-theme analysis focused on products, on the best seller's page, and take the list of friction points it returns.

Check every finding by hand. On a real phone, on the live store, before believing it. The reviews describe findings that were artefacts, out of date or simply not there. What survives the check is worth fixing or testing; what does not is discarded.

Fix what is broken, test what is a belief. A broken link or a slow page is a repair, not an experiment, and what actually slows a Shopify store covers the second. A different product page layout is a belief, and belongs in a test.

Screen drafts, then test the survivor for real. A comparison can discard the clear loser between two draft themes. The one that survives goes to a real experiment with real shoppers, which running a Shopify Rollouts test that gives an honest answer sets out, with a stopping rule written before it starts. A theme is published on that result, never on a simulated one.

Timing matters this month. A theme change in late October is a launch into the busiest weeks of the year, and the BFCM checklist for a store that holds up argues for settling the store before them. The useful work now is the analysis and the list; the experiments belong to January.

Finding where shoppers stop, deciding which of the reasons is worth a test, and running the test until it reaches a decision is the core of our conversion rate optimisation work. A simulated shopper is a new and cheap way to do the first two; it is not yet a way to skip the third. If you would like to know which part of your store a real test should look at first, the product page of your best seller is where we would start.