Skip to content
Blue Radar

A/B tests

Compare versions of a page by the visitors each one converts.

You decide who sees which version. Blue Radar counts, per version (a variant), the visitors who saw it and the ones who converted. Variants need the script on the page.

Tag the variant

Name the variant a page shows, in one of three ways.

A meta tag in the page:

html
<meta name="br-variant" content="hero-b">

An attribute on any element:

html
<section br-variant="hero-b">...</section>

Or JavaScript, before or after the script loads:

js
blueradar.setVariant("hero-b")

Each page view that has a variant counts as a view of that variant. A variant name can be up to 512 bytes long, spaces and punctuation included.

Count conversions

Add br-variant-track with the event name to anything people click:

html
<button br-variant-track="signup">Sign up</button>

Add br-variant-track-value to send a number with it, such as an order total:

html
<button br-variant-track="purchase" br-variant-track-value="49.90">Buy</button>

The click counts for the page's variant, or for the element's own br-variant. From JavaScript:

js
blueradar.trackVariantEvent("signup")
blueradar.trackVariantEvent("purchase", 49.9)
blueradar.trackVariantEvent("signup", null, "hero-b")
  • An event name is up to 64 bytes. A longer one keeps its first 64.
  • A conversion also counts as a custom event with the same name.

Read the results

Open the website, then A/B tests. Pick the date range and, when there is more than one, the conversion event.

Column Meaning
Visitors Different visitors who saw the variant
Conversions Times the event was sent for the variant
Rate Visitors who converted, divided by Visitors
Value Sum of the values sent, shown when events carry values

Variants are ordered by visitors. The 32 most active get a row of their own, the rest add up in Other variants. Leading marks the variant with the highest rate, among those with at least 30 visitors once two have that many.

Significance

The line under the table tests the leader against the runner-up, the next best rate, with a two-proportion z-test.

  • Beats means the difference is significant at 95% confidence.
  • Not significant yet means the gap could still be chance. Keep the test running.
  • Not tested says why, when the numbers cannot back a verdict:
    • not enough visitors yet: each of the two needs at least 30 visitors, and enough conversions and non-conversions, for the test to hold
    • some A/B events in this range were not counted
    • some variants or events are counted together as other
    • more visitors converted than were counted as shown the variant
    • visitor numbers are estimates below 1,000 visitors

Decide how long a test runs before you start, and stop it then. Checking every day and stopping at the first significant result finds differences that are not there.

Visitors

A visitor is a combination of IP address and browser, the same as on the report.