How to Measure Whether a Cart Change Raised Order Value

To know whether a cart change raised order value, track six numbers together: average order value, units per order, offer attach rate, reward unlock rate, conversion rate and profit per order. A change has worked only if profit per visitor went up, which needs order value and conversion to be read side by side. This guide defines each metric, compares a before-and-after reading with an A/B test, points out the seasonal traps, and gives you a one-page table to fill in each week.

Which metrics show whether a cart change worked?

Six, each answering a different question.

• Average order value (AOV): revenue divided by the number of orders. Did orders get bigger?
• Units per order: items sold divided by orders. Did people add more things, or just pricier ones?
• Offer attach rate: orders containing the offered product divided by all orders. Did shoppers take the offer?
• Reward unlock rate: orders that reached a reward threshold divided by all orders. Is the threshold in the right place?
• Conversion rate: orders divided by visitor sessions. Did the change cost you buyers?
• Profit per order: revenue minus the cost of goods, shipping you paid, and the cost of any gift or discount, divided by orders. Did the bigger orders earn more?

Why is AOV not enough?

Because it can rise while the business does worse. AOV counts only people who bought, and counts revenue, not profit. There are two ways it misleads.

It ignores lost orders. Raise a free-shipping threshold and some shoppers add an item to reach it, while others decide the cart is not worth the delivery charge and leave. The ones who leave vanish from AOV, so the average goes up partly because smaller orders stopped happening.

It ignores what the reward cost. A free gift that lifts order value by a few dollars and costs more than that to give away has lost money on every order that claimed it.

Here is an example with made-up round figures. Say that before a change, 1,000 visitors place 30 orders at an AOV of $50, with $15 profit per order. After it, 1,000 visitors place 26 orders at an AOV of $56, with $16 profit per order.

• AOV is up by $6. It looks like a success.
• Total profit was 30 × $15 = $450, and is now 26 × $16 = $416.
• Profit per visitor fell from $0.45 to about $0.42.

The orders grew and the store earned less. Only conversion and profit, read alongside AOV, reveal that.

Before and after, or an A/B test?

An A/B test is the more reliable of the two; before and after is what many stores can manage. Each has its place.

Before and after compares a period with the change against a period without it. It is simple and needs no special tool. Its weakness is that the two periods differ in more than your change: different traffic, different weather, a different email calendar.

To make it as fair as possible:

• Compare whole weeks with whole weeks.
• Use at least two weeks on each side.
• Change nothing else during the period: no price changes, no new campaigns.

An A/B test shows the old and new versions during the same weeks, to randomly split visitors. Whatever else is going on affects both groups equally, so the difference between them is down to the change. Its cost is time: each version gets only half the traffic. Our guide to A/B testing cart offers on a low-traffic store explains how to run one properly.

If your cart app can run the split for you, use it. Marka Cart Drawer includes A/B tests on offers and designs in its paid plans, with each visitor kept on one version and a winner called only when the result is clear. Basic analytics are on the free plan.

What are the seasonality traps?

Any comparison across time carries whatever else changed in those weeks. The ones that catch people most often:

• Sale periods. Order values and conversion both behave differently during a promotion. A change launched the week before a sale will look brilliant or terrible for reasons that are not its own.
• Traffic mix. A paid campaign brings in new visitors who may convert and spend differently. If the share of paid traffic changed, so did your averages.

Two defences help. Compare with the same weeks last year as well as with the weeks just before. And keep a simple log of dates: when you changed the cart, when campaigns ran, when stock ran out.

A simple weekly review table

Fill in one row per metric each week, with the baseline alongside. The figures below are made up to show the layout.

Metric	Baseline (4-week average)	This week	Direction
Visitor sessions	5,000	5,200	Up
Orders	150	152	Flat
Conversion rate	3.0%	2.9%	Flat
Average order value	$50	$54	Up
Units per order	1.8	2.0	Up
Offer attach rate	10%	14%	Up
Reward unlock rate	30%	36%	Up
Profit per order	$15	$15.50	Up
Profit per visitor	$0.45	$0.45	Flat

How to read it:

• AOV up, conversion flat, profit per order up: the change is working.
• AOV up, conversion down: the threshold or offer is pushing some shoppers away. Check profit per visitor before deciding.
• Unlock rate very high: the threshold is too low, and you are rewarding orders you would have had anyway. See how to calculate a free shipping threshold.

In this sample week, order value and attach rate rose while profit per visitor did not move. The honest reading is 'not proven yet'. One week is a single observation, so look for the same direction across several consecutive weeks before calling a change a success or a failure.

A short checklist

1. Record a four-week baseline for all six metrics before changing the cart.
2. Change one thing, and write the date in a log.
3. Fill in the weekly table, including profit per visitor.
4. Compare with the weeks before and with the same weeks last year.
5. Check the log for sales, campaigns and stock-outs before trusting a movement.
6. Wait for several weeks pointing the same way, then keep, adjust or reverse the change.

For ideas on what to change in the first place, see our guide to Shopify cart drawers and order value. If you want the cart to report on its own offers and run the comparison for you, look at what Marka Cart Drawer includes.