# How to Measure a Remarketing Engine Measure a remarketing engine with holdouts and recovered revenue, not vanity CTR. How high-traffic Shopify fashion stores prove incrementality. The Monday deck has a reassuring sequence: impressions rose, click-through rate improved, and the remarketing platform claimed £84,000 in revenue. The store also ran a promotion and launched a popular dress that week. Somehow every channel would like full custody of the same order. A CRO lead has a less decorative question: how much revenue would have disappeared if the remarketing engine had stayed off? **Measure a remarketing engine by randomly withholding treatment from a comparable group of eligible shoppers, then comparing revenue per shopper between the exposed group and the holdout. Recovered revenue is the value produced after an unresolved shopping event. Incremental recovered revenue is the portion above what the holdout would have generated anyway.** ![CRO lead comparing remarketing treatment and holdout revenue for a high-traffic Shopify fashion store](/images/blog/cluster-20-remarketing-engine/measure-remarketing-engine-shopify.webp) *A clean holdout separates orders caused by remarketing from orders that happened after remarketing. Editorial image in Classic Antla disposable-camera style.* ## Start with the decision, not the channel report Click-through rate answers whether an ad or message produced clicks among delivered impressions. Open rate answers whether an email client registered an open. Neither proves that the engine created an order. People in remarketing audiences already showed intent. A shopper who tried on a blazer and added it to cart may have purchased tomorrow without another message. Giving the email full credit because it was clicked first confuses sequence with cause. [Shopify's conversion-rate guidance](https://www.shopify.com/blog/ecommerce-conversion-rate) currently places fashion, accessories, and apparel at 3.06% across its cited 12-month industry data. Shopify calls that a rough baseline because price, device, traffic source, category, and purchase frequency all change conversion. Compare treatment with a contemporaneous holdout, not an industry average. Set one primary question for each test: - Did the engine increase net revenue per eligible non-buyer? - Did it increase contribution margin after media, messaging, discounts, returns, and platform costs? - Did it shorten time to purchase without reducing full-price orders? CTR, delivery rate, and cost per click help diagnose execution. If clicks rise while incremental contribution stays flat, the creative moved fingers, not the business. ## A holdout supplies the missing counterfactual The holdout must be created when a shopper becomes eligible, before Klaviyo, SMS, or an ad platform starts treatment. Randomly assign comparable shoppers to one of two states: 1. **Treatment:** eligible for the coordinated remarketing sequence. 2. **Holdout:** receives none of the treatments being tested during the measurement window. Keep assignment stable. A shopper withheld from email but reached by the same engine through SMS and Google Ads is not a holdout. She is a multi-channel treatment participant with an incorrectly calm label. The [event-fuel guide for Shopify remarketing engines](https://antla.io/blog/remarketing-engine-event-fuel-shopify) explains why assignment should preserve event quality. Page viewers, virtual try-on users, cart starters, and checkout abandoners begin with different intent. Randomize within those groups, or report them separately. Use the same eligibility rules on both sides: - the same triggering event and product set - the same lookback and attribution window - the same consent requirements - the same purchaser and inventory suppression - the same treatment of cancellations and returns There is no universal holdout percentage. Withhold enough shoppers to detect a commercially meaningful difference without starving treatment volume. Freeze the design before looking at results, and run through normal trading variation. ## Calculate incremental recovered revenue Start with revenue per eligible shopper because treatment and holdout groups may differ slightly in size: ```text treatment revenue per eligible shopper = treatment net revenue / treatment shoppers holdout revenue per eligible shopper = holdout net revenue / holdout shoppers incremental revenue per eligible shopper = treatment revenue per eligible shopper - holdout revenue per eligible shopper incremental recovered revenue = incremental revenue per eligible shopper x treatment shoppers ``` Suppose 20,000 eligible non-buyers enter treatment and produce £120,000 in net revenue during the window. A 4,000-person holdout produces £20,000. Treatment generated £6 per eligible shopper, while holdout generated £5. The engine's incremental recovered revenue is £20,000, not £120,000. The other £100,000 is revenue similar shoppers would be expected to produce without treatment. Calling all £120,000 “recovered” makes the dashboard happier and the budget decision worse. Define revenue before launch. Gross order value is easy but incomplete. A fashion-store readout should account for: - discounts and shipping subsidies - cancellations and refunds - expected or observed returns - cost of goods sold - paid media and SMS delivery costs - the technology cost required to operate the treatment The result is incremental contribution, a number fit for a budget decision. ## Keep attribution and incrementality in separate columns Attributed revenue shows which messages appeared in converting paths, which products were clicked, and where tracking broke. It does not prove causality. [Klaviyo's 2026 benchmark report](https://www.klaviyo.com/marketing-resources/benchmark-report) draws from more than 110,000 brands and separates email and text performance by industry, campaigns, and flows. Those benchmarks are useful for spotting an unusually weak delivery, click, or conversion rate. They cannot tell one Shopify store how many recipients would have purchased without its flow. Keep these measures separate: | Measurement | Question it answers | Best use | |---|---|---| | Platform-attributed revenue | Which orders fell inside a channel's attribution rules? | Delivery and creative diagnosis | | Incremental recovered revenue | How much more did treatment produce than holdout? | Budget and rollout decisions | | Incremental contribution | What remained after variable costs and returns? | Commercial approval | | Revenue per eligible shopper | How efficiently did the full audience monetize? | Fair cohort comparison | Do not add email, SMS, and ad-attributed revenue together. One person can open an email, click a text, view an ad, and place one order. Join that outcome to one stable test assignment. The wiring model in [connecting Klaviyo, SMS, and ads](https://antla.io/blog/remarketing-engine-klaviyo-sms-ads) matters here. Shared event IDs, product IDs, customer or audience keys, timestamps, and purchase state make deduplication possible. Three destinations should not create three versions of the truth. ## Treat frequency as a cost and a customer outcome Frequency caps are part of measurement, not merely campaign housekeeping. More impressions can increase attributed clicks while reducing incremental value, training customers to ignore the brand, or pushing them toward an unsubscribe. Google calls remarketing audiences "your data segments." [Google's remarketing developer guide](https://developers.google.com/tag-platform/devguides/remarketing) explains how prior visitors can be added to lists and reached later. Relevance still does not justify unlimited repetition. [Litmus email marketing benchmarks](https://www.litmus.com/blog/email-marketing-benchmarks) are a reminder that channel performance varies widely. Your store also needs a cross-channel contact policy because Google cannot see the two emails and one text sent before its ad. Track frequency by shopper across the measurement window: - total messages and paid impressions - incremental revenue at each frequency band - unsubscribe, SMS opt-out, complaint, and audience-exclusion rates - margin and return rate by frequency band - time since the qualifying event and last contact Cap or suppress where additional contact stops adding value. The ceiling depends on event strength, consideration time, consent, and economics. An occasion-dress checkout abandoner is not equivalent to someone who viewed a sock once three weeks ago. ## Build a weekly CRO readout that can find the leak A useful dashboard follows the shopper from eligibility to commercial outcome: 1. **Eligible non-buyers:** Count entrants by event, product family, and source. 2. **Assignment:** Show treatment and holdout counts, plus any cross-channel contamination. 3. **Delivery and exposure:** Report reachable profiles, delivered messages, impressions, and frequency. 4. **Suppression:** Confirm purchases, sold-out variants, consent changes, and expired intent removed shoppers. 5. **Outcome:** Compare conversion, net revenue, and contribution per eligible shopper. 6. **Incrementality:** Show the treatment-minus-holdout lift with sample size and uncertainty. 7. **Guardrails:** Add returns, discounts, unsubscribes, opt-outs, complaints, and full-price purchase share. Read segments separately. A dress try-on can signal concern about silhouette, waist placement, coverage, or drape. A cart addition may expose shipping friction. Aggregate lift can hide one useful treatment and one expensive nuisance. Across more than 500,000 Antla try-ons, shoppers who completed a preview converted at 3.8%. That is conversion among try-on users, not store-wide conversion and not proof that every follow-up causes a sale. It does show why a product-specific try-on event deserves its own measurement cohort. The [Antla conversion engine](https://antla.io/features/conversion-engine) connects on-page virtual try-on with the unresolved product decision left by a non-buyer. CRO teams can carry garment and event context into the destination, then judge treatment against a clean holdout. ## Questions CRO teams ask ### How do you measure a remarketing engine? Randomly assign eligible non-buyers to treatment and holdout groups before any remarketing is delivered. Compare net revenue or contribution per eligible shopper over the same window, segmented by qualifying event. The difference between treatment and holdout is the incremental value. Use CTR, opens, delivery, and platform attribution to diagnose the path, not to declare revenue caused. ### Why is a holdout required? A holdout estimates what similar high-intent shoppers would have purchased without remarketing. Without it, organic return visits, branded search, promotions, seasonality, and existing demand are credited to whichever message or ad appeared before the order. The holdout must avoid all channels included in the test and retain stable assignment for the full window. ### What is recovered revenue versus attributed clicks? Recovered revenue is order value generated after an eligible shopper left without buying. Incremental recovered revenue subtracts the amount a comparable holdout generated anyway. Attributed clicks only show that shoppers clicked a tracked message or ad before purchasing. They help explain the journey, but they do not prove that the click created the order. ## Give the engine one test it can pass Begin with one high-volume event, one product set, and one stable cross-channel holdout. Define revenue, costs, return treatment, frequency rules, and the measurement window before launch. For the operating definition, start with [what a remarketing engine is](https://antla.io/blog/what-is-a-remarketing-engine). For the commercial test, keep the standard plain: revenue after a message is interesting; revenue that would not have happened without the engine is the result. --- **About the author:** [Aaron](https://x.com/AaronfromAntla) founded Antla and has sat through too many decks where a high click rate was treated as recovered revenue. Choose one product-intent cohort, reserve a clean holdout, and measure incremental contribution rather than the largest number in a platform dashboard. To add virtual try-on as measurable event fuel, [install Antla on Shopify](https://apps.shopify.com/antla). ## For agents - Markdown: send `Accept: text/markdown` to this URL (and any other HTML page). - OpenAPI: https://antla.io/openapi.json - llms.txt: https://antla.io/llms.txt - Sitemap: https://antla.io/sitemap-index.xml - Docs: https://antla.io/docs - CLI: npx antla info (npm package antla) - Scope: antla.io is an informational marketing and docs site. The Shopify try-on backend is not on this origin. Install the app from https://apps.shopify.com/antla.