Call the winner once it has actually won.

You send the variant on your events. ClickClacks does the maths. Every result carries an always-valid interval, so checking it every morning never fools you.

Start tracking free All features
percent confidence, chosen per test
90/95/99
split check that hides a broken result
p<0.001
guardrail events watched beside the success metric
20
day window for the success event
1–90

What it does

ClickClacks doesn’t assign variants or manage feature flags. It reads the variant from an event you already send, counts who converted inside the window, and tells you when the difference is real, and not before.

Plan

Know how long it will take before you start.

Choose the smallest change worth finding and a confidence of 90, 95 or 99%. ClickClacks works out the people needed per variant from your own baseline, and a likely finish date from your own traffic.

  • A trade-off table shows what a smaller change costs in days.
  • Setup confirms it has seen both variant values in the last 24 hours.

Check any day

Look every morning. It won’t fool you.

Results use always-valid (sequential) confidence intervals, built to hold at every look. Until the interval clears zero, the verdict is “Not enough data yet”, with how many more people are needed and when the test should finish.

  • No peeking penalty: checking daily does not inflate the error rate.
  • You can stop a test at any point. You just can’t conclude from it early.

Guardrails

See what the winner did everywhere else.

Watch up to 20 other events next to the success metric. Guardrails don’t pick the winner. They show you, with their own intervals, whether the winner is doing damage somewhere else.

  • Every metric gets A, B, the lift and its interval.
  • Step metrics show where along the way the lift came from.

Split check

A broken split can’t sneak into a decision.

Each test compares the real split with the expected 50 / 50 using a chi-squared test. At p < 0.001 the result is hidden, the likely causes are listed, and shipping a variant is refused until the assignment is fixed.

  • Shows how far off the split is and how many people are missing from one arm.
  • “Show it anyway” is there, clearly labelled as untrustworthy.

Questions it answers

Ask it the way you’d ask a colleague.

How ClickClacks answers it

The verdict turns significant only once the always-valid interval for the lift clears zero. Until then it says “Not enough data yet”, whatever the observed number looks like.

+2.5 pts

Lift in sign-ups · +22.3% relative

Who it’s for

Three ways teams use it.

  • Agencies

    The client’s developers already split the new landing page with their own tooling. Send the variant on the pageview and bring the client an interval and a finish date, not a hunch.

    Variant on an existing pageview

  • Indie developers

    Assign the variant however you like, even a coin flip in your own code, and send it on the event you already fire. ClickClacks tells you when it’s callable.

    variant: b on signup_started

  • Product teams

    Guardrails keep refunds and support tickets on the same page as the success metric, and the split check stops a broken rollout from being shipped.

    Up to 20 guardrails per test

How it works

Under the hood

  1. 01Send the variantAdd it as a property on an event you already send, e.g. experiment: pricing_layout_v3, variant: b. A person’s first exposure decides their variant.
  2. 02Describe the testPick the exposure event, the property holding the variant, and the success event with a window of 1 to 90 days.
  3. 03Check it whenever you likeThe interval narrows as people arrive. When it clears zero, the verdict turns significant.
  4. 04Stop or shipStop at any time. Once a variant is significant, ship it. Shipping is refused while the split looks broken.

Spec sheet

from the product

Statistics
Always-valid intervals
Confidence
90, 95 or 99%
Variants compared
Two
Expected split
50 / 50
Split check
Chi-squared, p < 0.001
Guardrails
Up to 20 events
Success window
1–90 days
Assignment
First exposure decides

Same events, different question

Works well with

All features →

FAQ

Experiments, in detail.

  • No. Your app decides who sees what, however it does today, and sends the variant as a property on an event. ClickClacks reads it and does the maths.

  • Their first qualifying exposure decides their variant, permanently. Conversions only count if they happen at or after that exposure and inside the window.

  • The intervals are always-valid (a sequential method), built to hold at every look. A fixed-horizon test loses its guarantees when you peek; this one doesn’t.

  • It compares the observed split with the expected 50 / 50 using a chi-squared test. Below p < 0.001 the result is hidden, the likely causes are listed, and shipping is refused until you fix the assignment and restart.

  • Results compare two variants. If more values turn up in the variant property, the two largest arms are compared and the page notes that smaller arms were left out.

  • Up to 20 extra events watched beside the success metric, such as refunds or support tickets. They don’t decide the winner; they show, with their own intervals, whether the winner is doing damage.

Start hearing your clicks.

One script tag. Your first report in minutes.

Start tracking free Sign in

4.3 KB · pageviews, clicks and scroll depth, automatically