Test Data Management: Why Every Test Must Own Its Data

The Problem Starts With Shared Test Data

Your suite passes locally and fails in CI. It passes with one worker and fails with four. It passes when you run the file alone and fails when you run the folder. Everyone calls this flakiness, opens a ticket labelled flaky, adds a retry, and moves on.

It is not flakiness. It is a design defect, and the defect is shared test data. When twenty tests read the same seeded user qa_user_01, and one of those tests changes that user’s subscription tier, every other test is now reading a record whose state depends on execution order. The failure you see — “expected Pro, received Free” — looks exactly like a real product bug. That is the expensive part. You spend a morning reading application logs for a defect that exists only in your fixtures.

Shared seed data creates hidden coupling between tests that never import each other. Order becomes an invisible dependency. Parallelism becomes impossible without a lock. And the suite gets slower over time, because the only reliable fix anyone finds is fullyParallel: false. Once you accept serial execution, your test suite stops scaling with your team, and the feedback loop that automation was supposed to shorten starts growing again.

Three checks are enough to tell them apart. Run the test alone: if it passes, the problem is isolation, not the test. Change the order (--shuffle, or rename the file): if the result changes, there is a dependency between tests. Run the same test twice in a row: if the second run fails, the test is finding the residue it left behind. If all three come back clean you may genuinely have a timing problem; if they do not, adding retries only delays the diagnosis.

The Test Data Ownership Rule

One rule fixes most of this:

Every record a test reads or mutates must have been created by that test.

Not by a migration. Not by a seed.sql file checked in three years ago. Not by a beforeAll block shared across a describe block containing fifteen tests. By that test, for that test.

Three properties fall out of the rule directly:

• Order independence. If nothing outside the test produced the data, no other test can leave it in a state you did not expect. You can shuffle the suite and get the same result.
• Parallel safety. Two workers running the same file operate on disjoint records, so they cannot race for the same row.
• Standalone execution. Any single test can be run in isolation — which is what you actually do when debugging a failure at 6 p.m. on a release day.

The exception is reference data: currency codes, country lists, tax rate tables, permission definitions, feature flag defaults. This data is read-only from the application’s point of view, it is owned by a migration, and no test mutates it. That last clause is the whole exception. The moment a test updates a VAT rate for its own scenario, that row stops being reference data and becomes test data — and it needs to be owned. If you truly need to mutate reference data, create a new row instead of editing an existing one, or move the scenario to an environment where you can isolate it.

Building Data With Fixtures: Factory, Not Setup

beforeAll is the wrong tool. It creates one dataset for many tests, which is the shared-state problem in a smaller package. beforeEach is better but still awkward: it pushes state into mutable variables in module scope and gives you no clean teardown ordering.

Use a fixture that behaves as a factory: it produces data for a single test, hands it over, and removes it afterwards.

// fixtures/user-fixture.ts
import { test as base, expect, type APIRequestContext } from '@playwright/test';

export type TestUser = {
  id: string;
  email: string;
  password: string;
};

type Fixtures = {
  api: APIRequestContext;
  testUser: TestUser;
};

export const test = base.extend<Fixtures>({
  api: async ({ playwright }, use) => {
    const context = await playwright.request.newContext({
      baseURL: process.env.API_BASE_URL ?? 'http://localhost:3000/api',
      extraHTTPHeaders: {
        Authorization: `Bearer ${process.env.API_ADMIN_TOKEN ?? ''}`,
      },
    });
    await use(context);
    await context.dispose();
  },

  testUser: async ({ api }, use, testInfo) => {
    const email = `qa-w${testInfo.workerIndex}-${Date.now().toString(36)}@example.test`;
    const password = 'Str0ng-Passw0rd!';

    const created = await api.post('/users', {
      data: { email, password, displayName: 'Fixture User' },
    });
    expect(created.ok(), `user creation failed: ${created.status()}`).toBeTruthy();
    const user = await created.json();

    await use({ id: user.id, email, password });

    await api.delete(`/users/${user.id}`);
  },
});

export { expect } from '@playwright/test';

A test that needs a user asks for testUser and gets a user nobody else can touch. A test that does not need a user never pays for creating one, because Playwright only instantiates the fixtures a test actually requests. That property alone removes a lot of dead setup time from a mature suite.

Uniqueness: Not Random, But Deterministically Unique

user${Math.random()}@test.com is a bad identifier for two reasons. It collides — rarely enough that you will not believe it, often enough to burn a day. And when you open the database to investigate a leftover record, us***********@**st.com tells you nothing about which test created it.

Build identifiers from information you already have: worker index, retry index, a timestamp, and the test title.

// support/unique.ts
import type { TestInfo } from '@playwright/test';

function slug(input: string): string {
  return input
    .toLowerCase()
    .replace(/[^a-z0-9]+/g, '-')
    .replace(/^-+|-+$/g, '')
    .slice(0, 40);
}

export function uniqueTag(testInfo: TestInfo): string {
  const worker = String(testInfo.workerIndex).padStart(2, '0');
  const retry = String(testInfo.retry);
  const stamp = Date.now().toString(36);
  return `w${worker}r${retry}-${stamp}-${slug(testInfo.title)}`;
}

export function uniqueEmail(testInfo: TestInfo): string {
  return `qa-${uniqueTag(testInfo)}@example.test`;
}

export function uniqueName(testInfo: TestInfo, prefix: string): string {
  return `${prefix}-${uniqueTag(testInfo)}`;
}

qa******************************************@*****le.test answers three questions at a glance: which worker, which retry, which test. When you find an orphaned record next week, you know exactly which spec to fix. The timestamp handles reruns; the worker and retry index handle concurrency inside a single run.

Set Up Through the API, Verify Through the UI

Creating preconditions through the UI is the most common reason suites are slow and brittle at the same time. If forty tests click through the signup form to get a user, then a change to that form breaks forty tests that have nothing to do with signup. And each of those tests pays five seconds for something an HTTP call does in eighty milliseconds.

Set up state over HTTP. Use the browser only for the behaviour under test. This is also the point where a solid API automation layer pays for itself twice — once as its own test suite and once as infrastructure for the UI suite (API test otomasyonu neden ve nasıl yapılır, in Turkish).

// tests/projects/archive.spec.ts
import { test, expect } from '../../fixtures/user-fixture';
import { uniqueName } from '../../support/unique';

test('archived projects disappear from the active list', async ({
  page,
  api,
  testUser,
}, testInfo) => {
  const projectName = uniqueName(testInfo, 'Project');

  const createProject = await api.post('/projects', {
    data: { name: projectName, ownerId: testUser.id, status: 'active' },
  });
  expect(createProject.ok(), `project creation failed: ${createProject.status()}`).toBeTruthy();

  const session = await api.post('/auth/sessions', {
    data: { email: testUser.email, password: testUser.password },
  });
  expect(session.ok(), `login failed: ${session.status()}`).toBeTruthy();
  const { token } = await session.json();

  await page.addInitScript((value: string) => {
    window.localStorage.setItem('auth_token', value);
  }, token);

  await page.goto('/projects');
  await expect(page.getByRole('link', { name: projectName })).toBeVisible();

  await page.getByRole('row', { name: projectName })
    .getByRole('button', { name: 'Archive' })
    .click();
  await page.getByRole('button', { name: 'Confirm' }).click();

  await expect(page.getByRole('link', { name: projectName })).toBeHidden();
});

The project is deleted as a side effect of deleting its owner, so ownership still holds. If your API does not cascade, add a project fixture that cleans up after itself, exactly like testUser does.

Cleanup Strategies and When to Pick Which

Delete at end of test. Cheapest to implement, works against any environment you can reach over HTTP, and keeps ownership visible in the test code. Cost: it depends on delete endpoints existing and on referential integrity being cooperative. Best default for UI and API suites running against a deployed environment.

Transaction rollback. Wrap the test in a database transaction and roll it back. Nearly free, no leftovers ever. Cost: it only works when the test and the application share a process and a connection — integration tests against an in-process app, not a browser talking to a deployed service. Best for service-level tests, unusable for end-to-end.

Namespace or tenant isolation. Each test (or each worker) gets a tenant, an organisation, or an account, and everything it creates lives inside it. Cost: your product must support multi-tenancy, and provisioning a tenant is slower than creating a row. Best for B2B SaaS, and the only workable option when records cannot be deleted.

Periodic bulk cleanup. A scheduled job removes anything matching qa-% older than N hours. Cost: it never guarantees a clean state at any given moment, so it cannot be your only strategy. Best as a safety net behind one of the other three — and if you run tests against a shared environment, you need it regardless.

One constraint applies to all four: cleanup must never influence the assertion. Do not write a test whose final assertion is “the delete succeeded”. Do not put teardown before the last expect. Teardown is housekeeping; a failed cleanup means the environment has junk in it, not that the feature is broken.

Three constraints drive the choice. If the test has direct database access, transaction rollback is the cheapest option — nothing is ever committed, so cleanup costs nothing. If it does not, and setup goes through the API, you have to collect the id of every record a test creates and delete them at the end, which assumes those delete endpoints exist. If records cannot be deleted at all — audit trails, accounting constraints, mandatory soft deletes — stop trying to delete: use a partition that makes collision impossible instead, giving each test its own tenant or its own identifier space.

What Happens When Cleanup Fails

The CI runner is killed. The API returns 502 during teardown. A record has a foreign key you did not expect. Cleanup fails, and if cleanup is written as an assertion, a passing test turns red for a reason that has nothing to do with the product.

Treat cleanup as best effort: record the failure, do not raise it.

// support/cleanup.ts
import type { APIRequestContext, TestInfo } from '@playwright/test';

export async function safeDelete(
  api: APIRequestContext,
  path: string,
  testInfo: TestInfo,
): Promise<void> {
  try {
    const response = await api.delete(path);
    if (!response.ok() && response.status() !== 404) {
      testInfo.annotations.push({
        type: 'cleanup-failed',
        description: `${path} responded ${response.status()}`,
      });
    }
  } catch (error) {
    testInfo.annotations.push({
      type: 'cleanup-error',
      description: `${path}: ${(error as Error).message}`,
    });
  }
}

Then swap await api.delete(...) in the fixture for await safeDelete(api,/users/${user.id}, testInfo). Annotations show up in the HTML report, so leaks stay visible without failing anything.

Behind that, run a sweeper on a schedule:

// scripts/sweep-test-data.ts
const BASE_URL = process.env.API_BASE_URL ?? 'http://localhost:3000/api';
const TOKEN = process.env.API_ADMIN_TOKEN ?? '';
const MAX_AGE_HOURS = Number(process.env.MAX_AGE_HOURS ?? 12);

type User = { id: string; email: string; createdAt: string };

async function main(): Promise<void> {
  const cutoff = Date.now() - MAX_AGE_HOURS * 60 * 60 * 1000;
  const headers = { Authorization: `Bearer ${TOKEN}` };

  const response = await fetch(`${BASE_URL}/users?emailPrefix=qa-&limit=500`, { headers });
  if (!response.ok) {
    throw new Error(`listing users failed: ${response.status}`);
  }

  const users: User[] = await response.json();
  const stale = users.filter((u) => Date.parse(u.createdAt) < cutoff);

  let removed = 0;
  for (const user of stale) {
    const deletion = await fetch(`${BASE_URL}/users/${user.id}`, {
      method: 'DELETE',
      headers,
    });
    if (deletion.ok || deletion.status === 404) {
      removed += 1;
    } else {
      console.warn(`could not delete ${user.email}: ${deletion.status}`);
    }
  }

  console.log(`swept ${removed} of ${stale.length} stale test users`);
}

main().catch((error) => {
  console.error(error);
  process.exit(1);
});

The qa- prefix from your unique-identifier helper is what makes this job safe to run: it can only ever touch records your tests created.

Working With Immutable Data: Production-Like Environments

Some systems will not let you delete anything. Accounting ledgers, audit trails, regulated financial records, event stores. Other times the data originates in an upstream system you do not control, and your application only reads it.

Here, isolation is not achieved by removal but by scoping. Give each test its own slice of the domain:

• A dedicated tenant or organisation per test or per worker.
• A dedicated account, ledger, or customer that no other test touches.
• A dedicated date range, so a report test asserts over a window only its own transactions fall into.

The getByRole('row', ...) style of scoped assertion has a database equivalent: never assert on a global count, always assert within a scope you created. “Total transactions = 5” is a broken assertion in a shared environment. “Transactions for account X in October 2025 = 5” is stable regardless of what other tests do.

The limits are real. Scoping consumes resources permanently — the environment grows and never shrinks, which affects query performance and eventually your test durations. Some upstream data cannot be scoped at all, in which case the honest answer is to mock the upstream boundary rather than pretend you own it. And when scoping requires provisioning a tenant per test, the setup cost may push you to worker-scoped tenants instead.

Test Data and Parallel Execution

Parallel execution is the exam your data architecture has to pass. Nothing exposes ownership violations faster than raising workers from 1 to 4 and reading the failure list. Each failure points at a record two tests believe they own.

Do this deliberately, not accidentally. Run the suite with two workers, note what fails, fix the ownership, then go to four. The failures are a map of your shared state; treat the list as a backlog, not as noise. This is also where suite-level performance work actually starts paying off, because parallelism is the only lever that scales with hardware (test otomasyonunda performans optimizasyonu, in Turkish).

The second decision is fixture scope. Test-scoped data is created and destroyed per test: use it for anything the test mutates. Worker-scoped data is created once per worker process and shared by every test that worker runs: use it only for expensive, read-only setup — an admin session, a tenant shell, a catalogue of products nobody edits.

// fixtures/tenant-fixture.ts
import { test as base, expect, type APIRequestContext } from '@playwright/test';

type Tenant = { id: string; name: string };
type Project = { id: string; name: string };

type WorkerFixtures = {
  adminApi: APIRequestContext;
  workerTenant: Tenant;
};

type TestFixtures = {
  project: Project;
};

export const test = base.extend<TestFixtures, WorkerFixtures>({
  adminApi: [
    async ({ playwright }, use) => {
      const context = await playwright.request.newContext({
        baseURL: process.env.API_BASE_URL ?? 'http://localhost:3000/api',
        extraHTTPHeaders: {
          Authorization: `Bearer ${process.env.API_ADMIN_TOKEN ?? ''}`,
        },
      });
      await use(context);
      await context.dispose();
    },
    { scope: 'worker' },
  ],

  workerTenant: [
    async ({ adminApi }, use, workerInfo) => {
      const name = `qa-tenant-w${workerInfo.workerIndex}-${Date.now().toString(36)}`;
      const response = await adminApi.post('/tenants', { data: { name } });
      expect(response.ok(), `tenant creation failed: ${response.status()}`).toBeTruthy();
      const tenant: Tenant = await response.json();

      await use(tenant);

      await adminApi.delete(`/tenants/${tenant.id}`).catch(() => undefined);
    },
    { scope: 'worker' },
  ],

  project: async ({ adminApi, workerTenant }, use, testInfo) => {
    const name = `qa-project-w${testInfo.workerIndex}r${testInfo.retry}-${Date.now().toString(36)}`;
    const response = await adminApi.post(`/tenants/${workerTenant.id}/projects`, {
      data: { name },
    });
    expect(response.ok(), `project creation failed: ${response.status()}`).toBeTruthy();
    const project: Project = await response.json();

    await use(project);

    await adminApi
      .delete(`/tenants/${workerTenant.id}/projects/${project.id}`)
      .catch(() => undefined);
  },
});

export { expect } from '@playwright/test';

Tenant creation happens once per worker; project creation happens once per test. If a test needs to change tenant-level settings, it must create its own tenant rather than mutate the worker’s — that is the ownership rule applied at a coarser grain.

Two numbers only mean something together when you raise the worker count: total runtime and the number of failing tests. If runtime does not fall roughly in proportion to workers, the bottleneck is not in the tests but in a shared resource — usually the database connection pool, or a single setup step repeated per worker. If the failure count rises, isolation is incomplete; look at which tests fail and the ones touching the same table will cluster. Do not change both at once: fix isolation at a fixed worker count first, then raise it. Otherwise you cannot tell which change fixed what.

Test Data at the Mock and Contract Boundary

The JSON you feed a mocked dependency is test data. It obeys the same rule. A shared fixtures/payment-response.json imported by thirty specs is as fragile as a shared database row: the first test that needs status: "declined" edits the file, and twenty-nine other tests silently change behaviour.

Generate mock payloads per test from a builder function with sensible defaults and per-test overrides, the same way you generate database records. Keep the defaults in one place, keep the deviations in the test that needs them (mocking ve stubbing, in Turkish).

There is a second failure mode here that cleanup will never save you from: mock data drifting away from the real schema. Your suite goes green against a payload shape the provider stopped sending two sprints ago. The fix is to bind the mock payloads to a contract test — validate the builder’s output against the same schema the provider publishes, and run that validation in CI. Mock data that is not verified against a contract is a guess with a green checkmark on it.

Migrating an Existing Suite Step by Step

You do not need to rewrite everything. Work in the order that removes the most pain per hour:

1. Find the top failures. Collect flaky and failing test names from the last few weeks of CI runs. Rank by frequency. This list is almost always shorter than people expect, and it is where shared data hurts most.
2. Map the tables they touch. For each test in that list, write down which entities it creates, reads, or updates. Two tests appearing on the list that touch the same entity is your first target.
3. Isolate writers first. Any test that mutates shared data is the one poisoning the others. Give it its own data through a fixture. Readers often become stable without being touched, because nobody is corrupting their state anymore.
4. Then convert the readers. Once writers are isolated, migrate the readers file by file, so each one produces the record it asserts on.
5. Measure with workers. After each batch, raise the worker count by one and rerun. If the suite stays green, the batch is done. If it does not, you have found the next ownership violation — and you found it in minutes rather than during a release.

This is incremental by design, and it composes with the other structural work a growing suite needs (test otomasyonunda ölçeklenebilirlik, in Turkish).

What To Do Next

This week, in this order:

• Run your suite with --workers=2 --repeat-each=1 and save the failure list to a file. Do not fix anything yet. That file is your inventory of shared state.
• Pick the single most frequent failure from that list, and write one fixture that creates the record it depends on. Migrate that one spec file, nothing more.
• Replace every Math.random() and hardcoded qa_user_01 in that file with a qa-w{worker}r{retry}-{timestamp}-{title} identifier, so the next leak is traceable to a line of code.
• Add a scheduled sweeper job to your test environment that deletes qa- prefixed records older than twelve hours, and make it log how many it removed. That number is your leak rate.
• Add a rule to your PR checklist: a test may not read a record it did not create. Enforce it in review, before the suite grows another hundred files.