A/B Test Planning Worksheet: The 7-Field Spec Every Marketing Test Needs

Most marketing A/B tests fail before the variants go live. Not because the wrong creative wins, but because the test was specified badly enough that the result is unactionable. The metric is wrong. The sample size is wrong. The decision rule does not exist. The team picks the winner by gut anyway.

The fix is a planning worksheet. Seven fields, filled in before the test runs, signed off by the person who will make the post-test decision. Tests that go through this filter ship with answers. Tests that skip it produce arguments.

Below is the worksheet I use with marketing teams running everything from email subject line tests to multi-quarter landing page experiments. Free to copy.

Why most A/B tests waste effort

Three failure modes account for almost every wasted test I have seen.

The test was underpowered. The variant moved the metric 2%, the sample size could only detect a 10% move, and the result was 'no significant difference.' The team concludes the variant did not work. The variant might have worked. The test could not see it.

The metric was wrong. The team optimized click-through rate when the question was about conversion rate. The variant won on clicks and lost on revenue. Now what?

The decision rule was missing. The result came in ambiguous. Three stakeholders had three opinions on whether to ship. The conversation took two weeks, and the test became politics instead of evidence.

The worksheet forces all three to be decided before anyone writes code.

The 7-field A/B test planning worksheet

How to use each field

Hypothesis

The hypothesis is the whole point. 'Let us try removing the second form field' is not a hypothesis. 'We believe removing the second form field will increase form conversion rate by 8% or more for first-time visitors because the field caused detectable hesitation in session recordings' is a hypothesis.

Forcing the if-true and if-false observations into the spec is what separates a test from a tweak. Without those, the team will rationalize whatever happened.

Primary metric

Pick one. Tests with three primary metrics have no primary metric. The closer the metric is to revenue, the harder the test (longer time, more sample). The trade-off is real. Document why you made it.

Common defensible trade-offs: 'We are optimizing email click rate as a proxy for downstream meeting bookings because meeting bookings have a 6-week lag and we need decisions faster.' That is honest. 'We are optimizing email click rate because revenue is hard' is not.

Minimum detectable effect

The MDE answers 'how big a change has to happen for it to matter?' If your form converts at 4% and a 0.1% absolute lift would not change your annual revenue meaningfully, the MDE should be 0.5% or higher. Designing the test to detect tiny effects that do not matter is the most expensive form of test design.

MDE is also a sanity check on whether the test is worth running. Many tests proposed at marketing meetings have an MDE so small that detecting it would require months of traffic. Knowing that upfront usually kills the test before resources get committed.

Sample size and duration

Use any standard sample size calculator (Optimizely, AB Tasty, or a Python script). The output is the per-variant sample needed to detect the MDE at 80% power and 5% significance. Translate that into duration using current traffic.

The hard stop date is what most teams miss. Without it, underperforming tests run forever because nobody wants to call them. Setting a stop date in advance lets you cut a test without an argument.

Decision rule

Pre-committed decision rules prevent the post-result negotiation. The rule should be specific enough that any reasonable person reading the result can apply it without asking for input.

Inconclusive results default to 'do not ship.' This is the most disputed part of the framework and the most important. Tests that come back ambiguous are tests where the variant did not earn the right to ship. The cost of holding control is usually low. The cost of shipping unjustified variants compounds.

Rollback plan

Especially important for tests that affect anything beyond marketing (pricing changes, signup flows, checkout). Guardrail metrics catch the cases where the variant won on the primary metric but broke something else.

If rolling back takes a deploy cycle and three days, the test is more expensive than the planning conversation suggests. Worth knowing in advance.

Learning capture

The single most-skipped field. Teams run the test, look at the result for an hour, and move on. Six months later, nobody remembers what was tested, what happened, or why the team made the decision they made. The same test gets proposed again.

A one-page writeup (hypothesis, result, decision, what we learned, next test) makes the test compound into program knowledge. Pair this with the campaign retrospective template for larger initiatives.

Running the worksheet review as AI

Faster than a peer review for early-draft specs: have AI score the worksheet against the criteria above. The prompt below works for any of the major models.

When not to use this worksheet

Two cases where the full worksheet is overhead.

Subject-line tests inside an email tool that auto-picks the winner after a small send. The platform is making the decision for you. The worksheet adds nothing.

Multivariate tests with more than four variants. The worksheet assumes A/B. Multivariate testing needs a different framework (factorial design, interaction effects) that is out of scope for this template.

For everything else (landing pages, paid creative, email content tests, CTA changes, pricing page experiments, onboarding flow tests), the worksheet is the right shape.

Operating this over time

Three rhythms make this worksheet compound rather than feel like ceremony.

Weekly: a 30-minute test review meeting where every active test gets a status check against its decision rule and hard stop date. Cuts the 'test that ran 4 months past its end date' problem.

Monthly: read the last 5 to 10 test writeups out loud. Look for patterns. Are too many tests inconclusive (sample sizes too small)? Are too many tests on tiny MDEs (going after rounding errors)? Adjust the next batch of test plans.

Quarterly: roll up the program-level results. What was the net pipeline impact of the test program? How does that compare to the team's time investment? Tests that are not changing decisions are tests that did not need to run.

Pair this worksheet with the KPI dashboard template for the metrics view and the marketing brief template for upstream planning. Together they cover the planning, execution, and measurement loop.

Which of the seven fields does your team's current test process skip most often? Start there.