Avoiding Flaky Tests

Common causes of flakiness

A flaky test passes and fails on the same code. Typical causes: fixed sleeps (page.waitForTimeout, cy.wait(2000)) that are sometimes too short; brittle selectors tied to CSS classes or DOM structure that change with styling; shared state where one test depends on data or login left by another, or on run order; uncontrolled time and randomness such as dates, animations and random IDs; and third-party dependencies like analytics or payment widgets. Fixes: rely on auto-waiting and retrying assertions, prefer role, label or dedicated data-testid/data-cy attributes, make every test set up its own state, control clocks and stub external services.

A flaky test is worse than no test

Reliable suites wait for conditions, use stable selectors, isolate state and make failures easy to diagnose.

Flaky versus robust

The same steps written two ways.

// flaky (Playwright)
await page.click('.btn.btn-primary:nth-child(2)');
await page.waitForTimeout(3000);
expect(await page.locator('.toast').textContent()).toBe('Saved');

// robust (Playwright)
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByRole('status')).toHaveText('Saved');

// flaky (Cypress)
cy.get('.btn-primary').eq(1).click();
cy.wait(3000);
cy.get('.toast').then(($t) => expect($t.text()).to.eq('Saved'));

// robust (Cypress)
cy.get('[data-cy=save]').click();
cy.get('[data-cy=toast]').should('have.text', 'Saved');

Retries hide flakiness, they do not fix it

Test retries in CI keep pipelines green, but track tests that only pass on retry (both tools report them as flaky) and fix the root cause.

Waiting for the kettle

A fixed sleep is like walking away for exactly three minutes and hoping the kettle has boiled. A retrying assertion is listening for the click.

Quick check: Which change most directly reduces flakiness?

• Selecting elements by nth-child CSS
• Increasing every sleep to ten seconds
• Replacing fixed sleeps with retrying assertions on the expected state
• Making tests depend on the previous test's data