---
title: Pilot Design Template
description: "A worksheet for structuring a vendor pilot that produces an answer: treatment/holdout split, frozen metric definitions, four-week baseline, integrity-bounded windows and pre-agreed decision gates."
url: https://preview.artificialpoets.com/resources/pilot-design-template/
site: Artificial Poets
type: a13s_content
date: 2026-08-09T20:37:16+00:00
modified: 2026-08-15T20:38:14+00:00
---
# Pilot Design Template

The four decisions that make a pilot measurable — split, metric freeze, baseline, decision gates — as a worksheet you attach to the agreement.

The four decisions that make a pilot measurable — as a worksheet you complete with the vendor and attach to the agreement. Any section left blank is where the post-hoc storytelling will enter.

**Free template**

## Get the template

Enter your work email and the worksheet unlocks — readable in the browser, printable for the agreement.

## Section 1 · The split

- **Treatment group.** Properties or sections where the product will be enabled.
- **Held-back group.** Comparable properties or sections that stay unchanged — the counterfactual.
- **Selection rule.** Why these — ideally "entire eligible set" or a stated criterion, never chosen after results.

*Comparability check:* the two groups' weekly trends must move together during the baseline, or the comparison is repaired before the post-period counts.

## Section 2 · Metric freeze

- **Primary metric and exact definition.** e.g. multi-page session share = sessions with >1 pageview ÷ sessions; distinct-article counting stated.
- **Secondary metrics.**
- **Counting rules.** Bot filter, de-duplication, distinct-content units.
- **Freeze date.** Before enablement, in writing.

## Section 3 · Baseline

- **Instrumentation live from (day zero).**
- **Measurement-only period (weeks).** Floor: 4.
- **Enablement date — the intervention anchor.** A named day. Every later claim hangs on it.
- **Instrumentation faults found and fixed during baseline.**

## Section 4 · Windows and integrity bounds

- **Post-period length.** Minimum: one full quarter past launch — long enough to see past the launch bump.
- **Launch window vs steady-state window.** Both will be reported.
- **Week-invalidation rule.** Tracking gap, migration or retag on either group ⇒ the week is dropped.

*Panel balance:* every property present every counted week, or the week is dropped. Analysis ends at the last clean date for the whole panel — not the calendar.

## Section 5 · Decision gates

- **Success condition (pre-agreed).** Treatment exceeds held-back group by ___ points on the primary metric over the steady-state window.
- **Failure condition — the vendor completes this sentence.** "If ___ by ___, it isn't working."
- **Off-test (optional, powerful).** A scheduled pause late in the pilot; expected: metric returns toward baseline, recovers on resume.
- **Decision date and owners.**

## You just estimated. Want it measured?

Every tool here approximates. In twenty minutes on your analytics we compute the real distribution: your single-page rate, your depth by device and channel. No deck.

[Run my real numbers](/request-a-demo/)