readout: read out an experiment's lift
Calling an experiment early, or quoting a number nobody can check, turns a test into an opinion. readout reads
both arms' counts once each has reached the Experiment's minimum sample, and writes the lift, the significance test,
and the confidence interval into the Experiment as its Result.
| Field | Value |
|---|---|
| In plain words | Reads out an experiment once it has enough people, and writes down the measured lift. |
| Journey stage | Measure and learn |
| Accountable seat | Revenue Architect (Kuber) |
| Does the work | Kuber, the Accountable seat's twin, in the spec model lane, with the seat's paired human when it has one |
| Consulted | Product Strategist (Arjun) |
| Informed | Founder (Atri), Marketer & Documenter (Sauti) |
| Layers it can touch | L08 Analytics / reporting |
| Who reasons | your agent's own model, inside the skill |
| Writes as | the bot identity (a GitHub App); without one, your identity, tagged as human fallback |
| Where it runs | the aayu command, a skill in your agent, an MCP tool (only with [mcp] expose_writes on) |
| The skill your agent reads | skills/readout/SKILL.md |
When to use it
- An Experiment has run long enough that both arms may have reached its minimum sample.
- You need the measured lift written down where anyone can reproduce it, before deciding which version stays.
Measuring a Goal against its baseline after a release is measure. No Experiment yet? Start with
experiment.
Run it
Run it once the experiment has had time to collect people. Your agent reports the Result exactly as written.
Use the readout skill to read out Experiment issue 57 and tell me the Result as written.
aayu readout <experiment>
| Input | Required | What it does |
|---|---|---|
<experiment> |
yes | Experiment to read out. |
--source |
no | Where the per-arm counts come from; default posthog. One of: posthog, growthbook. |
--metric |
no | GrowthBook metric ID (with --source growthbook); PostHog uses the Experiment's Metric. |
--from |
no | Start of the PostHog readout window (YYYY-MM-DD, UTC); default the Experiment's creation. |
With GrowthBook: aayu readout 57 --source growthbook --metric met_signup.
Needs setup: the counts come from PostHog or GrowthBook. Connect the one you use with
aayu init --integrations, then store its read-only key withaayu auth set-secret posthogoraayu auth set-secret growthbook. Connecting PostHog is in PostHog and Sentry; GrowthBook has no step-by-step guide yet, and aayu cannot check its key's role, so give it a read-only key.Honest status: the PostHog and GrowthBook readouts have not been proven live against real accounts here.
What you get back
A Result section in the Experiment issue: the lift of the treatment over the control, relative and absolute, marked
significant or not significant at the chosen alpha, with the z-test, the confidence interval, a table of users and
conversions per arm, the window, and the source query. The Experiment moves to status:measured.
You should see the Experiment labelled status:measured and its Result starting Lift of. Stuck? See
everything else.
Gates
It checks these before it acts, and stops when one fails:
- min-sample: The experiment reached its minimum sample size.
While either arm is under the minimum sample, or the stated minimum disagrees with the Experiment's own parameters, it stops with exit 4 and writes nothing. Wait, then run it again.
What only a person does
The registry reserves no step of this verb for a person.
The Result says what the data shows, "not significant" included. Deciding what it means for the product, and which version stays, is a person's call.
What's next
- Measure the Goal the Feature serves:
measure. - The whole after-release flow: did it work?
- Every skill.