# Experiments

> Analyze A/B tests in ClickClacks: forward exposures from your flag tool, map variants, set a success metric and guardrails, read always-valid intervals, catch sample ratio mismatch, and record a decision.

- Canonical URL: https://clickclacks.io/docs/guides/experiments
- Section: Product guides
- Last updated: 2026-09-26

Your app or flag tool decides who sees which variant. ClickClacks watches the exposures, measures what happens next, and tells you when the answer is real, with results you can check as often as you like.

> **Part of the Pro plan**
>
> Experiments are included on Pro. See [pricing](https://clickclacks.io/pricing).

## How it works {#how}

- **You assign, we observe.** ClickClacks doesn’t split traffic or deploy anything. Your code or flag tool sends an exposure event that names the variant.
- **First exposure wins.** A person’s variant is the one in their first qualifying exposure. If they later see another arm, they stay in the first.
- **No variant, no count.** Exposures without the variant property are left out.

Already running tests? The empty Experiments page scans the last 7 days for exposure-like events and lists _tests already running_, with a confidence of **High confidence**, **Likely** or **Possible**. Click **Set up analysis** on one to start from it.

## Set up an experiment {#setup}

Click **New experiment**. The setup has five steps:

1. **What you’re testing**: a name, such as _Pricing page layout_, and an optional hypothesis.
2. **Exposure**: choose **Detected**, **Your tool** or **Custom**. Then set who is in the test (_Anyone who fires_ an event), optionally only when a property names this test, and which property holds the variant. A strip checks the last 24 hours and says what it has seen.
3. **Variants**: every value seen is listed with its count. Rename arms, merge aliases into one arm, ignore junk values, and pick the control. You can have up to 8 arms. Set the **Expected split** to **Even split** or **Custom shares**.
4. **Success metric**: _They fire_ an event _within_ a number of days (1 to 90, 7 by default), and whether it should increase or decrease. Click **Add a guardrail** for metrics that must not get worse. Set the smallest change worth finding, for example 2 points, at 90%, 95% or 99% confidence.
5. **Review**: check everything, then click **Start experiment**. Or **Save draft** to finish later.

While you set up, **Live exposure check** shows arm counts and warns about problems such as unmapped values, people who saw more than one arm, or an uneven split. **Before you start** estimates how many people you need and when you are likely to finish. Halving the change you want to catch roughly quadruples the wait.

## Sending exposures {#tools}

Pick **Your tool** and ClickClacks expects these names:

| Tool | Event | Test key | Variant |
| --- | --- | --- | --- |
| GrowthBook | `experiment_viewed` | `experiment_id` | `variation_id` |
| PostHog | `feature_flag_called` | `feature_flag` | `feature_flag_response` |
| LaunchDarkly | `flag_evaluated` | `flag_key` | `variation` |
| Optimizely | `experiment_decided` | `flag_key` | `variation_key` |
| VWO | `variation_shown` | `campaign_id` | `variation` |
| Statsig | `experiment_exposure` | `experiment` | `group` |
| Your own code | `experiment_viewed` | `experiment` | `variant` |

From your own code, send one event when the person sees the variant:

```title="exposure"
window.clickclacks('event', 'experiment_viewed', {
  experiment: 'pricing-layout',
  variant: 'annual-first',
})
```

Send the variant as text. True or false values arrive as `"true"` and `"false"`. More on sending events in [Events and identify](https://clickclacks.io/docs/events.md).

## Always-valid results {#always-valid}

Classic A/B statistics assume you look at the result once, at a finish date you fixed in advance. Peek every day and stop when it looks good, and you will crown false winners.

ClickClacks uses **always-valid intervals** everywhere. The interval holds at every look, so checking daily doesn’t spoil the result. Until the interval clears zero, the result stays _Not enough data for a conclusion_. There is no fixed finish date: the planning estimate says roughly how long it may take.

## Reading results {#read}

- With two arms, the headline says _Treatment improves the primary metric_, _Treatment is worse on the primary metric_, or _Not enough data for a conclusion_. It shows the observed difference in points and percent, and the interval as a bar. It is conclusive when the bar doesn’t cross zero.
- With more arms, each treatment is compared with control. You see _Winner_, _Control wins_ or _No treatment beat control_. Each comparison uses a stricter threshold so the confidence you chose holds for the whole experiment.
- **Guardrail harm detected** means a guardrail got worse. A good primary result doesn’t cancel that risk.
- The **Planning estimate** shows how many people you have out of about how many you need.

Results count from the moment you start, across all sources, within each experiment’s own start date and conversion window, not the date range in a filter bar.

## Broken assignment (SRM) {#srm}

If the arms’ sizes differ from the split you expected by more than chance allows (p < 0.001), the page says **Assignment looks broken — results are not trustworthy** and hides the results. This is a sample ratio mismatch. It usually means a bug: one arm fails to send its exposure, redirects lose people, or bots land in one arm. Fix the cause. Don’t read around it.

## Stopping and deciding {#decide}

- **Record decision**: pick the selected variant and confirm that you understand this records a decision and doesn’t deploy anything. Observation ends now. Roll out the change in your own flag tool.
- **Stop test**: ends observation without a decision.
- Once running, the exposure, metrics and window are locked. You can still rename, edit the hypothesis and fix the variant mapping. Every change is recorded under **Setup changes**.

## Tips and gotchas {#tips}

- **No exposures yet?** Send one from your app with the snippet. It appears in the mapper within a minute.
- **Typos split your arms.** If `Treatment` and `treatment` both show up, merge them in Variants.
- **From a friction finding**, **Set up an experiment for this fix** starts a draft named after it.

## Recipes {#recipes}

### Test a new pricing layout with GrowthBook.

1. Forward GrowthBook’s `experiment_viewed` events to ClickClacks.
2. Click **New experiment**, choose **Your tool** › GrowthBook, and type the experiment key.
3. Map the variants and pick the control.
4. Set the success metric to `Trial started` within 7 days, increase. Add a guardrail on `Refund requested`, decrease.
5. Click **Start experiment** and check back whenever you like.

### Is it safe to stop early?

1. Open the experiment and read the interval bar.

If it clears zero, the result is conclusive, even on day three, because the interval is always-valid. If it still crosses zero, stopping now means “we don’t know”, not “there is no difference”.

## Related {#related}

- [Events and identify](https://clickclacks.io/docs/events.md): Send exposure events.
- [Friction](https://clickclacks.io/docs/guides/friction.md): Test a fix for a finding.
- [Funnels](https://clickclacks.io/docs/guides/funnels.md): Pick a success metric people already reach.
- [Recipes](https://clickclacks.io/docs/guides/recipes.md): More questions answered step by step.