#Reading an experiment's results

The page of an experiment (Experimente › name) and experiments_results over MCP show the same numbers. Only the arm comparison of the primary metric decides – and only once, at the end (How X3 decides). Everything else on the page is descriptive: it explains a result or finds a broken arm.

#Progress and evaluation

The progress line reads like "Läuft · Tag 12 von 28 · 403 von 1.180 Besuchstagen". X3 evaluates every running experiment hourly as a background run experiments.evaluate; the page shows it live ("Auswertung läuft", then "Zuletzt ausgewertet 14:03"). Members with config:write can start one with Jetzt auswerten (POST /v1/projects/{project}/runs with kind: "experiments.evaluate"). An evaluation in between never decides – it checks harm stops, the split (SRM), tracking gaps and the projection.

#Follow-ups (Braucht Dich)

Follow-ups are tasks of the shared task list (tasks_list, the bell, Cockpit): ready to start, code missing, split problem, tracking gap, variant stopped, probably not decidable, decided (roll out?), data problem and code cleanup. They open and resolve themselves when the situation changes ("Von selbst erledigt: Experiment gestartet"); you can also mark them done, snooze or dismiss them like any other task.

#Arm comparison

One row per arm: exposed units (visit-days, visitors or page loads), conversions, rate and its range. While the experiment runs the range is always valid – you may look at it any time without inflating errors. Effect against the control and "Wahrscheinlich die beste" appear only with the decision. Guardrails show ✓, ✗ or "nicht prüfbar" with the drop that could go unnoticed.

#Funnel by arm

For any funnel of the project: "reached step k / exposed units" per arm. Step-to-step rates between arms are not comparable (different people get further), and a step only one arm has shows "—".

#Course over time

The cumulative rate of the primary per arm since the start, each with its always-valid band. Days left out as tracking gaps are hatched. While the bands overlap, nothing is decided.

#Further metrics and segments

• Further metrics (up to five, set at creation): rates per arm; "auffällig" marks a difference that survives the Benjamini–Hochberg correction for many comparisons (q = 0.1). They never decide.
• Segments: the primary per device and per source of the first exposed visit. Rows below 30 units in an arm read "zu wenige". Whoever looks at many segments always finds a "winner" somewhere – they are labelled descriptive only.

#Purchases and revenue (shop projects)

In projects with the mode shop or mixed the primary can be Käufe (bought yes/no per unit) or Umsatz pro Einheit. Purchases follow the shop counting rule: one row per order id, held or guarded rows do not count, a fully refunded order drops out, revenue is net of refunds and only in the project currency. Revenue decides only with at least 100 purchases per arm in total; the values are capped at the 99th percentile of the 28 days before the start (never at the experiment's own data). Below that threshold "bought yes/no" decides and the result says so (revenue_fallback_purchase). In shop projects a proxy primary needs purchase as a guardrail.

Qualified leads are every counted lead not marked "Disqualifiziert" – a lead marked lost still counts as qualified.

#Behaviour per arm

Each arm links the page map, the attention heatmap, the mass replay and the recordings list with the filter experiment=<key>:<arm> (a visible filter chip) and the experiment's period; "Neben Kontrolle öffnen" opens the page map in segment comparison. The same filter works on behavior_heatmap, behavior_elements, behavior_mass_replay, behavior_visits and recordings_list. What exists follows your capture settings: with heatmaps or recordings switched off the page says so instead of linking to an empty view, every arm shows its basis ("solange nicht abgelehnt", "nur mit Zustimmung"), and arms below the project's minimum show "zu wenig Besucher".

#New versions

Hypothesis, arms, metrics, unit, targeting, runtime and rule are frozen by the pre-registration. To change one of them, or to run a test again, use Duplizieren als neue Version (experiments_duplicate, POST …/experiments/{experiment}/duplicate): a new draft under <key>-v2 (or your own key) with the same definition. The original keeps its results; both histories link the other version. The new version needs its own variant("<key>-v2") call in the code before it can start.