Test Smells and Flaky Tests

When tests cause pain

Bad tests slow teams down. Common test smells: brittle tests that break on harmless refactors because they verify internal calls or exact log messages; over-mocking, where tests replicate the implementation line by line; logic in tests (loops, conditionals) that can itself be wrong; mystery guests, where tests depend on external files or data you cannot see; giant tests checking many behaviours at once; assertion-free tests that only check that no exception occurred; and slow tests that start Spring or databases for logic that needs neither. Flaky tests pass and fail without code changes, destroying trust in the suite. Typical causes: Thread.sleep and timing assumptions (use await libraries such as Awaitility to poll for conditions), shared state between tests, dependence on test order, real time and time zones, randomness without seeds, network calls, and parallel execution on shared resources. Track flaky tests, fix or quarantine them quickly, and never simply re-run CI until it turns green.

Replacing sleep with Awaitility

Polling for a condition is faster and more reliable than a fixed sleep.

// flaky: guesses how long the async work takes
@Test
void emailEventuallySent_flaky() throws Exception {
    service.placeOrderAsync(order);
    Thread.sleep(2_000);                       // too short on a slow CI machine, too long locally
    assertThat(outbox.sentEmails()).hasSize(1);
}

// reliable: wait until the condition holds, up to a timeout
@Test
void emailEventuallySent() {
    service.placeOrderAsync(order);
    await().atMost(Duration.ofSeconds(5))
           .pollInterval(Duration.ofMillis(50))
           .untilAsserted(() -> assertThat(outbox.sentEmails()).hasSize(1));
}

A smoke alarm that goes off randomly

A flaky test is a smoke alarm that sometimes beeps for no reason. After a few false alarms, people start ignoring it, and then it is useless when there really is a fire.

Quick check: What is the most reliable replacement for Thread.sleep when waiting for asynchronous work in a test?

• A longer sleep
• Polling for the expected condition with a timeout (for example Awaitility)
• Removing the assertion
• Running the test twice