Sample Ratio Mismatch Checker

Check whether your A/B test's traffic actually split the way it was meant to, before you trust any of its results.

Your numbers

How many variants

Number of variants

Traffic was meant to split equally

Turn off if you ran an unequal split, e.g. a 90/10 holdout.

Visitors seen

Visitors, A

Visitors, B

Your result · updates as you type

Sample ratio mismatch detected

p < 0.0001

This split is very unlikely to happen by chance. Don't trust the test's results until you find the cause.

Total visitors: 20,000
Chi-square statistic: 23.805
P-value: < 0.0001

Common causes: a redirect or bot filter that only fires for one variant, a caching layer serving the wrong variant to some visitors, or the test SDK dropping events differently by variant. Fix the cause before reading any conversion result from this test.

Breakdown

A: observed vs expected: 10,345 vs 10,000 (51.73% of traffic)
B: observed vs expected: 9,655 vs 10,000 (48.28% of traffic)

Why the traffic split matters before anything else

If your test was meant to split visitors 50/50 (or any other ratio), a chi-square test checks whether the actual split you observed is plausible given that intention. A skewed split is a warning sign that something in the test setup is broken: fix it before trusting any conversion result from the same test.

Chi-square statistic = Σ (observed − expected)² ÷ expected, compared against a chi-square distribution with (variants − 1) degrees of freedom

Example: the default numbers

Visitors, A: 10,345 (expected 10,000)
Visitors, B: 9,655 (expected 10,000)

Chi-square is 23.805, giving a p-value of less than 0.0001. That's far below the usual 0.01 flagging threshold: this is a sample ratio mismatch, and the test's results shouldn't be trusted until the cause is found.

For comparison, a healthier split like 10,062 vs. 9,938 gives a p-value well above 0.01, the kind of small imbalance that happens by chance in every test and needs no action.

Common causes of a mismatch

A redirect, bot filter, or cache rule that only fires for one variant, quietly dropping some of its visitors before they're counted.
The test tool's SDK failing to log an exposure event for one variant on certain browsers or devices.
A pre-existing audience exclusion (like an old coupon segment) that overlaps more with one variant's traffic than the other's.

Questions people ask

What is a sample ratio mismatch?

It's when the actual visitor split between A/B test variants doesn't match the split you intended, by more than chance would explain. It usually points to a bug in tracking or targeting, not a real result.

What p-value counts as a mismatch?

A chi-square p-value below 0.01 is the usual flag. Above that, small imbalances are normal sampling noise and don't need investigation.

Can I still trust the conversion results if there's a mismatch?

No. If visitors weren't split the way you intended, something about your setup likely also affects who ends up in each group, or how their events are counted, which can bias the conversion rates themselves.

Does this work for more than 2 variants?

Yes, up to 4. The test degrees of freedom adjust automatically, and you can set a custom expected split instead of an equal one, for example an intentional 90/10 holdout.

Last reviewed 30 September 2026.