Same Metric Name, Different Number: The Risk In Every Platform Migration

When a business checks whether a new commerce platform covers its existing reporting, the instinct is to look for the metric by name. Is there a "Net Sales" figure? Yes. Is there a "Return Rate"? Yes. Tick, move on. It's a reasonable first pass. It's also where a subtle risk hides, because a metric with a familiar name can still be built on different logic underneath. If you're mid-migration yourself, this is the check that's easy to skip because it looks like it's already been done.

We ran into this during a reporting gap analysis for two global-fashion e-commerce sister brands, after their migration to Shopify. We ended up producing something that was originally out of scope: a full terms-and-definitions glossary because "same name" kept turning out not to be a reliable signal of "same number."

The proposition: how the same-sounding metric can mean something different

Take "Gross Sales," Shopify's definition calculates the value of all purchased items before taxes, shipping, discounts, returns or fees, while including pending, cancelled and unpaid orders, and explicitly excluding test and deleted orders. Counter that to a legacy system's own order lifecycle and that can yield a different “Gross Sales” number, despite both being legitimate calculations.

Or take "Average Order Value." Shopify calculates it by excluding gift cards from both the numerator and the order count, and by excluding post-order adjustments like edits, exchanges or returns, taxes, duties, shipping and discounts. That's a specific, deliberate set of exclusions, reasonable on its own terms, but different enough from a brand's existing AOV definition (built, perhaps, around net-of-returns figures) that a trading team comparing "this month's AOV" against last year's could be comparing two different calculations without realising it.

"Discounts" is another interesting one. Shopify defines it as line-item discount plus a proportional share of any order-level discount, applied before tax, and explicitly tied to discount codes rather than compare-at pricing. A brand that had been tracking discounting through markdown percentages against compare-at price, a very common retail practice, would find that its old definition and Shopify's simply don't overlap cleanly, even though both report on "discounts."

Fulfilment timing metrics carried the same trap. Shopify distinguishes "time to ship," "time to fulfil" and "time to deliver" as three separate, precisely scoped measures, each anchored to a different pair of timestamps in the order lifecycle. A brand used to a single blended "delivery time" figure would need to decide which of the three actually corresponds to what its operations team has historically meant by that number, rather than assuming the first plausible-sounding match is the right one.

We found this pattern often enough across the 433 metrics we mapped that documenting definitions became as important as documenting availability. For one of the two brands specifically, this mattered most in its Pricing Report. This report was used daily, feeding almost all of the brand's pricing analysis, where 37 metrics were flagged as high priority. A metric showing up as a "direct match" in a gap analysis spreadsheet can still mislead a business if nobody checks what's actually being calculated underneath the label, and a daily-use pricing report is exactly the wrong place to discover that after the fact.

What this means for any retailer trusting a new dashboard

The fix here doesn't require new tooling. It requires discipline at exactly the point where it's tempting to skip it, because the metric appears to already exist. Document every KPI's definition explicitly as part of the migration, not just its presence or absence. Where a metric matters enough to be in a trading review, run a sample of real transactions through both the old and new systems side by side, and check that the numbers actually reconcile before anyone downstream is asked to trust the new dashboard.

Treat this as a data governance exercise with a shelf life beyond the migration itself. A well-documented set of KPI definitions doesn't just protect the cutover. It gives every team a shared reference for what a number means, which pays off long after the new platform has settled in and someone new joins the trading team.

Considering the substantial protection it provides, this is a cost-effective step.Building a glossary alongside a migration is a documentation exercise, not a technical one. It doesn't require new infrastructure, just a disciplined pass through the metrics that actually get used, capturing exactly how each is calculated in both the old and new systems side by side. Compared with the cost of a wrong pricing decision made on a misread number, or the slower cost of a trading team quietly losing trust in dashboards they can no longer explain, that's a modest investment for real protection.

A gap analysis that checks if a metric exists in the new platform only addresses half of the question. The other half, is whether or not the metrics mean the same thing. Taking into consideration this potential difference, being on the lookout for it and addressing before the project even starts, will save you countless hours and help ensure you get to reporting transparency on day one.

Key takeaways

• A metric appearing to "match" by name during a platform migration doesn't guarantee it's calculated the same way underneath.
• Shopify's own definitions for common metrics like Gross Sales, Average Order Value and Discounts each carry specific inclusions and exclusions that may not align with a brand's legacy definitions.
• Mindera built a full terms-and-definitions glossary alongside a fashion brand reporting gap analysis for exactly this reason.
• Document KPI definitions explicitly during any migration, and reconcile real transactions across old and new systems before trusting the new numbers.
• A documented set of KPI definitions is a data governance asset that outlasts the migration itself.

If you're mid-migration and want a second pair of eyes on whether your "matching" metrics actually match, we're happy to talk through what that check could look like.