Test Data Management: Why Every Test Must Own Its Data

The Problem Starts With Shared Test Data

Your suite passes locally and fails in CI. It passes with one worker and fails with four. It passes when you run the file alone and fails when you run the folder. Everyone calls this flakiness, opens a ticket labelled flaky, adds a retry, and moves on.

It is not flakiness. It is a design defect, and the defect is shared test data. When twenty tests read the same seeded user qa_user_01, and one of those tests changes that user’s subscription tier, every other test is now reading a record whose state depends on execution order. The failure you see — “expected Pro, received Free” — looks exactly like a real product bug. That is the expensive part. You spend a morning reading application logs for a defect that exists only in your fixtures.

Shared seed data creates hidden coupling between tests that never import each other. Order becomes an invisible dependency. Parallelism becomes impossible without a lock. And the suite gets slower over time, because the only reliable fix anyone finds is fullyParallel: false. Once you accept serial execution, your test suite stops scaling with your team, and the feedback loop that automation was supposed to shorten starts growing again.

Three checks are enough to tell them apart. Run the test alone: if it passes, the problem is isolation, not the test. Change the order (--shuffle, or rename the file): if the result changes, there is a dependency between tests. Run the same test twice in a row: if the second run fails, the test is finding the residue it left behind. If all three come back clean you may genuinely have a timing problem; if they do not, adding retries only delays the diagnosis.

The Test Data Ownership Rule

One rule fixes most of this:

Every record a test reads or mutates must have been created by that test.

Not by a migration. Not by a seed.sql file checked in three years ago. Not by a beforeAll block shared across a describe block containing fifteen tests. By that test, for that test.

Three properties fall out of the rule directly:

• Order independence. If nothing outside the test produced the data, no other test can leave it in a state you did not expect. You can shuffle the suite and get the same result.
• Parallel safety. Two workers running the same file operate on disjoint records, so they cannot race for the same row.
• Standalone execution. Any single test can be run in isolation — which is what you actually do when debugging a failure at 6 p.m. on a release day.

The exception is reference data: currency codes, country lists, tax rate tables, permission definitions, feature flag defaults. This data is read-only from the application’s point of view, it is owned by a migration, and no test mutates it. That last clause is the whole exception. The moment a test updates a VAT rate for its own scenario, that row stops being reference data and becomes test data — and it needs to be owned. If you truly need to mutate reference data, create a new row instead of editing an existing one, or move the scenario to an environment where you can isolate it.

Putting the Rule Into Practice

The ownership rule needs three mechanisms: a factory that produces the test’s own record, a setup step that builds preconditions through the API rather than the UI, and a cleanup strategy that reclaims what the run produced. I covered how each one is built, which is right for which environment, and what each actually costs in a separate article: Test Data Management: Fixtures, Factories and the Cost of Cloning Production Data.

The rest of this article covers the two hard corners of the rule: environments where you cannot change the data, and parallel execution.

Working With Immutable Data: Production-Like Environments

Some systems will not let you delete anything. Accounting ledgers, audit trails, regulated financial records, event stores. Other times the data originates in an upstream system you do not control, and your application only reads it.

Here, isolation is not achieved by removal but by scoping. Give each test its own slice of the domain:

• A dedicated tenant or organisation per test or per worker.
• A dedicated account, ledger, or customer that no other test touches.
• A dedicated date range, so a report test asserts over a window only its own transactions fall into.

The getByRole('row', ...) style of scoped assertion has a database equivalent: never assert on a global count, always assert within a scope you created. “Total transactions = 5” is a broken assertion in a shared environment. “Transactions for account X in October 2025 = 5” is stable regardless of what other tests do.

The limits are real. Scoping consumes resources permanently — the environment grows and never shrinks, which affects query performance and eventually your test durations. Some upstream data cannot be scoped at all, in which case the honest answer is to mock the upstream boundary rather than pretend you own it. And when scoping requires provisioning a tenant per test, the setup cost may push you to worker-scoped tenants instead.

Test Data and Parallel Execution

Parallel execution is the exam your data architecture has to pass. Nothing exposes ownership violations faster than raising workers from 1 to 4 and reading the failure list. Each failure points at a record two tests believe they own.

Do this deliberately, not accidentally. Run the suite with two workers, note what fails, fix the ownership, then go to four. The failures are a map of your shared state; treat the list as a backlog, not as noise. This is also where suite-level performance work actually starts paying off, because parallelism is the only lever that scales with hardware (test otomasyonunda performans optimizasyonu, in Turkish).

The second decision is fixture scope. Test-scoped data is created and destroyed per test: use it for anything the test mutates. Worker-scoped data is created once per worker process and shared by every test that worker runs: use it only for expensive, read-only setup — an admin session, a tenant shell, a catalogue of products nobody edits.

// fixtures/tenant-fixture.ts
import { test as base, expect, type APIRequestContext } from '@playwright/test';

type Tenant = { id: string; name: string };
type Project = { id: string; name: string };

type WorkerFixtures = {
  adminApi: APIRequestContext;
  workerTenant: Tenant;
};

type TestFixtures = {
  project: Project;
};

export const test = base.extend<TestFixtures, WorkerFixtures>({
  adminApi: [
    async ({ playwright }, use) => {
      const context = await playwright.request.newContext({
        baseURL: process.env.API_BASE_URL ?? 'http://localhost:3000/api',
        extraHTTPHeaders: {
          Authorization: `Bearer ${process.env.API_ADMIN_TOKEN ?? ''}`,
        },
      });
      await use(context);
      await context.dispose();
    },
    { scope: 'worker' },
  ],

  workerTenant: [
    async ({ adminApi }, use, workerInfo) => {
      const name = `qa-tenant-w${workerInfo.workerIndex}-${Date.now().toString(36)}`;
      const response = await adminApi.post('/tenants', { data: { name } });
      expect(response.ok(), `tenant creation failed: ${response.status()}`).toBeTruthy();
      const tenant: Tenant = await response.json();

      await use(tenant);

      await adminApi.delete(`/tenants/${tenant.id}`).catch(() => undefined);
    },
    { scope: 'worker' },
  ],

  project: async ({ adminApi, workerTenant }, use, testInfo) => {
    const name = `qa-project-w${testInfo.workerIndex}r${testInfo.retry}-${Date.now().toString(36)}`;
    const response = await adminApi.post(`/tenants/${workerTenant.id}/projects`, {
      data: { name },
    });
    expect(response.ok(), `project creation failed: ${response.status()}`).toBeTruthy();
    const project: Project = await response.json();

    await use(project);

    await adminApi
      .delete(`/tenants/${workerTenant.id}/projects/${project.id}`)
      .catch(() => undefined);
  },
});

export { expect } from '@playwright/test';

Tenant creation happens once per worker; project creation happens once per test. If a test needs to change tenant-level settings, it must create its own tenant rather than mutate the worker’s — that is the ownership rule applied at a coarser grain.

Two numbers only mean something together when you raise the worker count: total runtime and the number of failing tests. If runtime does not fall roughly in proportion to workers, the bottleneck is not in the tests but in a shared resource — usually the database connection pool, or a single setup step repeated per worker. If the failure count rises, isolation is incomplete; look at which tests fail and the ones touching the same table will cluster. Do not change both at once: fix isolation at a fixed worker count first, then raise it. Otherwise you cannot tell which change fixed what.

Test Data at the Mock and Contract Boundary

The JSON you feed a mocked dependency is test data. It obeys the same rule. A shared fixtures/payment-response.json imported by thirty specs is as fragile as a shared database row: the first test that needs status: "declined" edits the file, and twenty-nine other tests silently change behaviour.

Generate mock payloads per test from a builder function with sensible defaults and per-test overrides, the same way you generate database records. Keep the defaults in one place, keep the deviations in the test that needs them (mocking ve stubbing, in Turkish).

There is a second failure mode here that cleanup will never save you from: mock data drifting away from the real schema. Your suite goes green against a payload shape the provider stopped sending two sprints ago. The fix is to bind the mock payloads to a contract test — validate the builder’s output against the same schema the provider publishes, and run that validation in CI. Mock data that is not verified against a contract is a guess with a green checkmark on it.

Migrating an Existing Suite Step by Step

You do not need to rewrite everything. Work in the order that removes the most pain per hour:

1. Find the top failures. Collect flaky and failing test names from the last few weeks of CI runs. Rank by frequency. This list is almost always shorter than people expect, and it is where shared data hurts most.
2. Map the tables they touch. For each test in that list, write down which entities it creates, reads, or updates. Two tests appearing on the list that touch the same entity is your first target.
3. Isolate writers first. Any test that mutates shared data is the one poisoning the others. Give it its own data through a fixture. Readers often become stable without being touched, because nobody is corrupting their state anymore.
4. Then convert the readers. Once writers are isolated, migrate the readers file by file, so each one produces the record it asserts on.
5. Measure with workers. After each batch, raise the worker count by one and rerun. If the suite stays green, the batch is done. If it does not, you have found the next ownership violation — and you found it in minutes rather than during a release.

This is incremental by design, and it composes with the other structural work a growing suite needs (test otomasyonunda ölçeklenebilirlik, in Turkish).

What To Do Next

This week, in this order:

• Run your suite with --workers=2 --repeat-each=1 and save the failure list to a file. Do not fix anything yet. That file is your inventory of shared state.
• Pick the single most frequent failure from that list, and write one fixture that creates the record it depends on. Migrate that one spec file, nothing more.
• Replace every Math.random() and hardcoded qa_user_01 in that file with a qa-w{worker}r{retry}-{timestamp}-{title} identifier, so the next leak is traceable to a line of code.
• Add a scheduled sweeper job to your test environment that deletes qa- prefixed records older than twelve hours, and make it log how many it removed. That number is your leak rate.
• Add a rule to your PR checklist: a test may not read a record it did not create. Enforce it in review, before the suite grows another hundred files.