Guillaume Lamanda

Stable and fast end-to-end tests with Playwright

The rules we used on a client project to keep a Playwright suite fast and free of flaky tests, from test scope to fixtures and debugging timeouts.

Published testing, playwrightLire en français

This article comes from the guide I wrote for a team I worked with. Our end-to-end suite had two goals: be stable 🧱 and fast ⚡. Here are the rules that got us there.

📖 Before you start

If you’ve never read the Playwright docs, read at least these three sections first:

🚷 Global rules

Random data, to avoid flakiness

Tests must never conflict with each other.

One way to make sure of that is to generate random data with Faker instead of using hard-coded values.

Limited scope

We want to test value for the user, so it’s tempting to test a whole big workflow.

Inside that workflow, look for independent chunks that still bring value to the user.

Sometimes it isn’t possible. Onboarding ends when the user gets into the app, and there’s no value before that 😅. That test will be bigger.

Why it matters: testing a big workflow gives you either one enormous test or many tests that depend on each other.

Take invoices. You might write a test that:

  • creates an invoice,
  • then updates it,
  • then deletes it.

It makes sense. But the second step depends on the first, and the third on the second.

These are three tests, one per action, ideally one file each.

Hey, but I need an invoice to update or delete it!

True. That’s the next rule.

💽 Prepare your test data

A test needs data for one of two things: reading it (a query) or changing it (a mutation).

Data you read

Data you only read, to check something on screen, belongs in the account’s base dataset. We provide it with Playwright fixtures.

Fixtures extend Playwright’s test object. More precisely, they extend the test environment: Playwright makes sure each test has what it needs, nothing more, nothing less.

We created fixtures for basic data, like a bank account. To use one, take it from the test arguments:

test.describe('homepage', () => {
  //                                       take it from here 👇
  test('shows the bank account name', async ({ page, bankAccount }) => {
    //                                   👇 the whole object is available 🎉
    await expect(page.getByText(bankAccount.alias)).toBeVisible();
  });
});

To change the dataset, change the fixture, or the code it calls.

Data you change

Say you want to test updating an invoice.

You need an invoice. You might be tempted to create it through the app, clicking everywhere, or even to reuse the creation test.

That only slows the test down, and we want it fast ⚡.

Instead, create the invoice before the test by calling the API directly. Use the app’s API when you can. If you can’t, write a helper, or ask the backend team for an endpoint.

import { test as baseTest } from './fixtures/test';

const test = baseTest.extend<object, { invoice: Invoice }>({
  invoice: [
    //        👇 fixtures can use other fixtures
    async ({ organization }, use) => {
      // create the object through the API
      const invoice = createInvoiceObject({
        number: faker.string.alphanumeric(8),
        totalAmount: 20_000,
        direction: 'INBOUND',
        status: 'IMPORTED',
      });
      await createInvoices(organization.id, [invoice]);
      // hand it to the test
      await use(invoice);
    },
    { scope: 'worker' },
  ],
});

test('shows the new invoice', async ({ page, invoice }) => {
  //                                       👇 use it
  await expect(page.getByText(invoice.number)).toBeVisible();
});

Because the test is part of the feature, this involves the backend too.

ℹ️ This could get simpler. Playwright’s mergeTests combines fixtures from several modules, so one day it could look like mergeTests(baseTest, invoiceFixture({ number: faker.string.alphanumeric(8) })).

You might be tempted to reuse data from the read fixtures. It can work, but if concurrent tests share data and one of them changes it, you get flakiness. Fresh, dedicated data for each mutation test is the simplest.

Helper structure

We split the helpers in two folders:

  • api: functions that call the API,
  • objects: functions that build a domain object, like an invoice.

The objects helpers infer the type of what you pass. In the invoice type, supplierName is string | null. The object returned by createInvoiceObject({ supplierName: 'Superman' }) has a supplierName typed string.

Most of the time, you build an object, then pass it to an API function:

import { createInvoice } from './utils/api/createInvoice';
import { createInvoiceObject } from './utils/objects/createInvoiceObject';

const invoice = createInvoiceObject({ number: faker.string.alphanumeric(8) });

test.beforeAll(async () => {
  await createInvoice(invoice);
});

test('...', async ({ page }) => {
  // ...
  await expect(page.getByText(invoice.number)).toBeVisible();
});

😄 Focus on the happy path

To start, focus on the happy path. Edge cases belong in unit tests, at least for now.

The goal is still a fast ⚡ suite.

📹 Record videos for QA

To help QA review a new end-to-end test, turn on Playwright’s video recording in playwright.config.ts:

export default defineConfig({
  use: {
    video: 'on',
    launchOptions: {
      slowMo: 1000, // slows each action down so the video is easy to follow
    },
  },
});

After a run, the videos are in the test-results folder.

🤖 Commands

npx playwright test                    # run everything
npx playwright test invoices           # only tests with `invoices` in their path
npx playwright test invoices --debug   # step through with the inspector
npx playwright test --ui               # full-featured UI mode

🦠 Hunting flakiness

The HTML report is your best friend. Every CI run should publish one, with traces. (On that project, we also wrote a small custom reporter that sent flaky tests to a shared page, so we could see the trend.)

Look at why the first attempt failed. Was it a timeout? Why? Did the API take long to respond, and is that normal? Was it a navigation that took too long? Or did the test itself time out?

Those are the three big families of failures.

Test timeout

If the test itself timed out, first ask: does it contain parts that could be tested separately? If yes, split it.

Then ask: what really matters in this test? Can I skip some steps with API calls? If so, do it. The API will always be faster than clicking through the interface.

Only when none of that helps, raise the timeout. A slow test that passes beats a slow test that runs twice because it’s flaky.

There are two ways to get a navigation timeout: the locator you expect never shows up, or the API takes too long to respond.

If the locator never shows up, the app probably behaves inconsistently. The best fix is always to fix the app.

If the page is waiting on a slow API, raise the timeout of that expect:

await expect(myLocator).toBeVisible({ timeout: 20_000 });

And tell the backend team: they have a slow endpoint.

To get a clearer error, assert the URL before looking for elements on the page:

await expect(page).toHaveURL(`/inbound-invoices/${invoice.id}`);
// with a timeout
await expect(page).toHaveURL(`/inbound-invoices/${invoice.id}`, { timeout: 20_000 });