Turning Acceptance Criteria Into Tests: From Given-When-Then to Running Scenarios

What a Vague Acceptance Criterion Actually Costs

“The user should not be able to log in with invalid credentials.” Three people read that sentence and build three different pictures in their heads: one tests a wrong password, one tests a locked account, one tests a malformed email address — and a fourth person assumes someone else covered it and tests nothing. The feature is marked done. Nobody agrees on what “done” meant.

The real cost of ambiguity is not the bug that escapes to production. It is the rework: the story that comes back from review because the developer implemented a different rule than the product owner had in mind, the defect report that turns into a thirty-minute argument about whether it is a bug or a misunderstanding, the same scenario tested twice by two people while a third case is never touched. An acceptance criterion has exactly one job: to make the done/not-done decision answerable with a single sentence, by anyone, without a conversation.

The Four Conditions of a Testable Acceptance Criterion

A criterion becomes a test only when four things are present.

An observable output. The behaviour must be visible from outside the system — a message, a status code, a state the user or a calling client can see. “The order is saved correctly” is not observable; “the order appears in the customer’s order list with status Confirmed” is.

A specific starting state. The data the system holds before the action must be named. “A user with an active account and an empty cart” is a starting state. “A normal user” is not — normal is whatever the reader last worked on.

A single triggering action. One thing happens. “The customer submits the coupon code WELCOME10” is one action. “The customer logs in, adds products and checks out” is a sequence, and when it fails you do not know which step broke the rule.

A binary outcome. The result is either true or false with no judgement in between. “The response returns HTTP 422 with error code COUPON_EXPIRED” is binary. “The page loads fast”, “the flow is user-friendly”, “the calculation works properly” each fail at least one of the four conditions — usually observability and binarity at the same time.

If a criterion fails any of the four, it is not a criterion yet. It is a wish.

Using Given-When-Then Correctly

Given-When-Then is not a writing template; it is a thinking discipline that forces the four conditions into the open. Given is the state of the system, never an action. When is the single trigger. Then is the observable result.

Three mistakes break it in practice.

Putting user steps inside Given. “Given the user opens the login page and types their email” smuggles the action into the setup. Now the scenario has two actions and the boundary between setup and trigger disappears — which means when the test fails you cannot tell whether the precondition was never satisfied or the rule itself is broken. Given describes facts: “Given a registered user with a verified email address”.

Chaining multiple Whens. Every extra When adds a failure point that the Then cannot explain. A scenario with three triggers produces a red test whose message tells you only that something along the way went wrong. One trigger per scenario keeps the diagnosis in the failure line itself.

Asserting internal database state in Then. “Then a row is inserted into the discounts table” couples the criterion to an implementation that will be refactored. The test then fails for a schema change while the business rule still works perfectly — the cheapest way to teach a team to ignore red builds. Assert what the system exposes: response body, status code, rendered text.

Rewriting a Vague Criterion Into a Scenario: Before and After

Take a real story: As a customer, I want to apply a discount coupon to my cart so that I pay less. Typical first-draft criteria:

1. The customer can apply a coupon.
2. Invalid coupons are not accepted.
3. The discount is reflected in the total.

Three questions turn each of these into a scenario. What starting data? What single action? What exact message, status code or screen?

Criterion 1 becomes:

Given a cart containing one item priced 100.00 EUR
And an active coupon WELCOME10 with a 10% discount
When the customer submits the code WELCOME10
Then the cart shows a discount line of 10.00 EUR and a total of 90.00 EUR

Criterion 2 is not one scenario — “invalid” is a category, not a state. Asking which invalid? is exactly the refinement conversation that should happen before code is written, and it produces the boundaries that were missing from the first draft: a coupon that has expired, a coupon already used by this customer, a cart below the coupon’s minimum amount, a code that does not exist at all. Each needs its own message:

Given a cart totalling 100.00 EUR
And a coupon SUMMER20 whose validity ended yesterday
When the customer submits the code SUMMER20
Then the cart total stays 100.00 EUR and the message “This coupon has expired” is shown

Given a cart totalling 30.00 EUR
And a coupon SAVE15 requiring a minimum cart of 50.00 EUR
When the customer submits the code SAVE15
Then the cart total stays 30.00 EUR and the message “Minimum cart amount for this coupon is 50.00 EUR” is shown

Criterion 3 disappears, because it was never a separate rule — it was the Then of criterion 1. One vague line became four precise ones, and two of them exist only because somebody asked “which invalid?” out loud.

Not Every Scenario Is an E2E Test: Placing Criteria at the Right Level

Automating all of these through the browser produces a slow suite that breaks on every layout change. The decision rule is mechanical: business rule variations belong to unit or service level, data flow and integration belong to API level, and only the journey the customer actually sees belongs to E2E.

For the coupon story, five criteria split across three levels:

Three business rules run in milliseconds against the discount calculator. One integration check confirms the API stores and returns the discounted total. One end-to-end test confirms the customer can actually get from cart to a visibly reduced total. If you want the longer argument for why the same assertion gets cheaper as it moves down, start with API test automation.

From Scenario to Running Test: A Playwright Skeleton

The goal is that reading the test gives you the criterion back. Given becomes setup, When becomes a single call, Then becomes one assertion block.

// tests/coupon-apply.spec.ts
import { test, expect, type APIRequestContext } from '@playwright/test';

const API = process.env.API_URL ?? 'http://localhost:3000/api';
const APP = process.env.APP_URL ?? 'http://localhost:3000';

type Cart = { id: string; token: string };

async function createCartWithItem(
  request: APIRequestContext,
  unitPriceCents: number,
): Promise<Cart> {
  const response = await request.post(`${API}/carts`, {
    data: { items: [{ sku: 'SKU-TSHIRT', quantity: 1, unitPriceCents }] },
  });
  expect(response.ok(), 'cart setup failed').toBeTruthy();
  return response.json();
}

async function createCoupon(
  request: APIRequestContext,
  coupon: { code: string; percentOff: number; minCartCents: number; expiresAt: string },
): Promise<void> {
  const response = await request.post(`${API}/admin/coupons`, { data: coupon });
  expect(response.ok(), 'coupon setup failed').toBeTruthy();
}

test('customer sees a 10% discount on a 100 EUR cart with coupon WELCOME10', async ({
  page,
  request,
}) => {
  // Given a cart containing one item priced 100.00 EUR
  const cart = await createCartWithItem(request, 10_000);

  // And an active coupon WELCOME10 with a 10% discount
  await createCoupon(request, {
    code: 'WELCOME10',
    percentOff: 10,
    minCartCents: 0,
    expiresAt: '2099-12-31T23:59:59Z',
  });

  await page.goto(`${APP}/cart/${cart.id}?token=${cart.token}`);

  // When the customer submits the code WELCOME10
  await page.getByLabel('Coupon code').fill('WELCOME10');
  await page.getByRole('button', { name: 'Apply coupon' }).click();

  // Then the cart shows a 10.00 EUR discount and a total of 90.00 EUR
  await expect(page.getByTestId('discount-line')).toHaveText('-10.00 EUR');
  await expect(page.getByTestId('cart-total')).toHaveText('90.00 EUR');
});

Two details carry the weight. The Given is done over the API, not by clicking through the shop — browser-driven setup is the single largest source of instability in an E2E suite, and each test creating its own cart keeps scenarios independent (why every test must own its data). And the test title is the criterion, not TC-114.

Example Tables and Parameterized Scenarios

When the same rule repeats with different inputs, copying the scenario four times gives you four files to update when the rule changes. Use a table instead — readable as a specification by the business analyst, usable as a data source by the test.

Cart total	Coupon	State	Expected total	Expected message
100.00	WELCOME10	active	90.00	–
100.00	SUMMER20	expired	100.00	This coupon has expired
30.00	SAVE15	min 50.00 EUR	30.00	Minimum cart amount for this coupon is 50.00 EUR

// tests/coupon-rules.spec.ts
import { test, expect } from '@playwright/test';

const API = process.env.API_URL ?? 'http://localhost:3000/api';

type Case = {
  name: string;
  cartCents: number;
  coupon: { code: string; percentOff: number; minCartCents: number; expiresAt: string };
  expectedStatus: number;
  expectedTotalCents: number;
  expectedError?: string;
};

const cases: Case[] = [
  {
    name: 'an active coupon reduces a 100.00 EUR cart to 90.00 EUR',
    cartCents: 10_000,
    coupon: { code: 'WELCOME10', percentOff: 10, minCartCents: 0, expiresAt: '2099-12-31T23:59:59Z' },
    expectedStatus: 200,
    expectedTotalCents: 9_000,
  },
  {
    name: 'an expired coupon leaves the cart total unchanged',
    cartCents: 10_000,
    coupon: { code: 'SUMMER20', percentOff: 20, minCartCents: 0, expiresAt: '2020-01-01T00:00:00Z' },
    expectedStatus: 422,
    expectedTotalCents: 10_000,
    expectedError: 'COUPON_EXPIRED',
  },
  {
    name: 'a cart below the minimum amount is rejected',
    cartCents: 3_000,
    coupon: { code: 'SAVE15', percentOff: 15, minCartCents: 5_000, expiresAt: '2099-12-31T23:59:59Z' },
    expectedStatus: 422,
    expectedTotalCents: 3_000,
    expectedError: 'CART_BELOW_MINIMUM',
  },
];

for (const c of cases) {
  test(`coupon rules: ${c.name}`, async ({ request }) => {
    // Given a cart and a coupon in the described state
    const cartResponse = await request.post(`${API}/carts`, {
      data: { items: [{ sku: 'SKU-TSHIRT', quantity: 1, unitPriceCents: c.cartCents }] },
    });
    expect(cartResponse.ok()).toBeTruthy();
    const cart = await cartResponse.json();

    const couponResponse = await request.post(`${API}/admin/coupons`, { data: c.coupon });
    expect(couponResponse.ok()).toBeTruthy();

    // When the customer submits the coupon code
    const applyResponse = await request.post(`${API}/carts/${cart.id}/coupons`, {
      data: { code: c.coupon.code },
    });

    // Then the status, total and error code match the specification
    expect(applyResponse.status()).toBe(c.expectedStatus);

    const body = await applyResponse.json();
    if (c.expectedError) {
      expect(body.errorCode).toBe(c.expectedError);
    }

    const cartState = await request.get(`${API}/carts/${cart.id}`);
    expect((await cartState.json()).totalCents).toBe(c.expectedTotalCents);
  });
}

Each row is reported as its own test, so a failure names the rule that broke. When the table grows, prune it with one question per row: does this row exercise a different branch of the rule, or just a different number? Three expired coupons with different dates test the same branch — keep one. Rows that differ only in product name or customer id are duplicates wearing a costume.

Establishing Traceability Between Criterion and Test

You do not need a separate traceability matrix spreadsheet; it rots within two sprints because nobody updates it under deadline. Test titles and tags do the job, and they live in the same commit as the code.

Put the criterion’s sentence in the test title and the story or criterion id in a tag:

test('@SHOP-412 an expired coupon leaves the cart total unchanged', async ({ request }) => {
  // ...
});

Now two questions are answerable with grep. When a rule changes, grep -r "@SHOP-412" tests/ lists every test to update. When CI goes red, the report line reads as a broken business rule rather than TC-114 failed. Playwright runs tag subsets directly — npx playwright test --grep "@SHOP-412" during refinement of that story, --grep "@smoke" on every push.

The tool you write criteria in matters less than keeping one source of truth. Typically it looks like this: criteria written in the ticket description are easy for product people but drift from the test suite; criteria written only in test code are always current but invisible to anyone outside the repository. Whichever you pick, decide which copy wins when the two disagree, and state it.

The same choice applies to the Gherkin layer. A .feature file plus step definitions buys you an artifact non-engineers can read, at the price of an indirection layer and a glue vocabulary that has to be maintained. Keeping GWT purely as a thinking frame and writing tests straight in Playwright removes that layer, at the price of criteria living in test titles instead of a document. Pick based on who actually reads the scenarios — if nobody outside the team ever opens the .feature files, you are paying for a reader who does not exist.

The Ritual Behind Good Criteria: Three Parties, Fifteen Minutes

Criteria written alone at one desk stay ambiguous, because the author cannot see their own assumptions. The fix is structural: product owner, developer and tester write the scenarios together during refinement. Fifteen minutes spent agreeing on the exact message text removes the interpretation gap before any code exists — which is the only point where removing it is free.

The meeting has three concrete outputs: criteria written in GWT, a decided test level for each one, and an explicit list of items marked not tested. That last one matters most; an untested item is a decision when it is written down and a hole when it is not.

A working checklist to run the session against:

• Does every criterion name a starting state, one action and one observable result?
• Is every expected message, status code or label written out verbatim?
• For each “invalid” or “incorrect”, have we listed the specific cases?
• What are the boundaries — zero, empty, expired, maximum, concurrent?
• Which level does each criterion run at: unit, API or E2E?
• Which criteria are deliberately not automated, and why?
• Which existing tests change because of this story?

If a question cannot be answered in the meeting, the story is not ready — that is the signal refinement exists to produce.

Patterns to Avoid

UI steps inside the scenario. “When the user clicks the blue Apply button in the top-right corner” ties a business rule to a layout. The rule survives a redesign; the sentence does not. Write “When the customer submits the coupon code” and let the test code hold the selector.

Multiple business rules in one scenario. A scenario that checks the discount amount, the expiry message and the minimum-cart message at once fails on the first assertion and hides the other two. One rule per scenario means a red test names the broken rule.

Shared data or state between scenarios. When scenario B depends on the coupon scenario A created, order becomes a hidden contract, parallel execution breaks and failures move around between runs. This is one of the most common roots of instability — see diagnosing flaky tests. Each scenario creates what it needs.

Negative, unmeasurable outcomes. “Then the system should not throw an error” passes when the page is blank, when the coupon was silently ignored, when the total is wrong. Replace it with the positive statement: “Then the cart total stays 100.00 EUR and the message ‘This coupon has expired’ is shown.”

What to Do on Your Next Story

Open the current sprint and pick one story. Audit each criterion against the four conditions — observable output, starting state, single action, binary outcome — and mark the ones that fail. Take the vaguest one and rewrite it as Given-When-Then, writing the expected message or status code verbatim. Decide its level using the rule above: business rule variation goes down, user journey stays at E2E. Then write exactly one running test for it, with the criterion as the test title and the story id as a tag.

Bring the checklist to the next refinement and run the session with all three parties in the room. In the first week, measure one thing only: the number of clarifying questions and comments per criterion after the story enters development. If that number is falling, the criteria are doing their job.