Automated test cases can fail intermittently due to transient issues such as service deployments, network instability, or temporary resource contention. Introducing a retry mechanism helps filter out flaky failures and improves overall test reliability.

The pytest-rerunfailures plugin integrates with pytest to automatically re-execute failed tests based on configurable criteria.

Plugin Overview

pytest-rerunfailures is a pytest extension that detects failing test executions and reruns them up to a specified count. If a test passes within the allowed attempts, it is marked successful; otherwise, the last failure is reported.

Installation

Install via pip:

pip install pytest-rerunfailures

Once installed, the plugin activates when running pytest without extra steps.

Applying Retry Logic

Method 1: Using a Marker Decorator

Mark individual tests with @pytest.mark.flaky to enable retries:

import pytest

@pytest.mark.flaky(max_attempts=4, pause_seconds=2)
def verify_response():
    assert fetch_data() == expected_value

Here, max_attempts defines how many times to retry (including the first run), and pause_seconds sets the wait time between attempts. Running the test yields output indicating each rerun until success or exhaustion.

Example command:

pytest -s -v path/to/test_module.py::verify_response

Direct execution from certain IDEs may bypass plugin hooks; use terminal invocation for guaranteed behavior.

Method 2: Command-Line or Programmatic Configuration

Specify retry parameters globally via CLI flags:

pytest -s -v --reruns 4 --reruns-delay 2 path/to/test_module.py::verify_response

Or configure programmatically:

import pytest

pytest.main([
    "-s", "-v",
    "--reruns", "4",
    "--reruns-delay", "2",
    "path/to/test_module.py::verify_response"
])

Execution Flow

During test collection, pytest loads pytest-rerunfailures. For each test item:

1. The plugin checks if retry parameters exist either via marker or global settings.
2. If a test fails, it captures the failure and evaluates whether another attempt should occur.
3. When reetrying, it respects the configured delay before invoking the test again.
4. Intermediate results are logged as rerun outcomes.
5. Cached fixture results and setup state are cleared before subsequent attempts to avoid state leakage.
6. The loop exits when the test passes or the maximum retry count is hit, returning the final status.

This design ensures isolation between runs and accurate reporting of both original and retry outcomes.

Key Implementation Details

Registration of the custom marker occurs in the pytest_configure hook:

def pytest_configure(config):
    config.addinivalue_line(
        "markers",
        "flaky(max_attempts=1, pause_seconds=0): rerun test up to 'max_attempts' times with 'pause_seconds' between runs."
    )

The core retry loop resides in a modified test protocol handler:

def pytest_runtest_protocol(item, nextitem):
    attempts_allowed = get_attempts_limit(item)
    if attempts_allowed is None:
        return

    check_options(item.session.config)
    delay_time = get_pause_duration(item)
    parallel_mode = not is_master(item.config)
    failures_tracker = item.session.config.failures_db
    item.run_counter = failures_tracker.get_test_failures(item.nodeid)
    failures_tracker.set_test_attempts(item.nodeid, attempts_allowed)

    if item.run_counter >= attempts_allowed:
        return True

    should_retry = True
    while should_retry:
        item.run_counter += 1
        item.ihook.pytest_runtest_logstart(nodeid=item.nodeid, location=item.location)
        test_reports = runtestprotocol(item, nextitem=nextitem, log=False)

        for rpt in test_reports:
            rpt.rerun = item.run_counter - 1
            if not should_stop_retry(item, rpt, attempts_allowed):
                item.ihook.pytest_runtest_logreport(report=rpt)
            else:
                rpt.outcome = "rerun"
                time.sleep(delay_time)

                if not parallel_mode or xdist_compatible(item.config):
                    item.ihook.pytest_runtest_logreport(report=rpt)

                _clear_fixture_cache(item)
                _reset_setup_state(item)
                break
        else:
            should_retry = False

        item.ihook.pytest_runtest_logfinish(nodeid=item.nodeid, location=item.location)

    return True

Functions like get_attempts_limit, get_pause_duration, and should_stop_retry encapsulate configuration retrieval and decision-making, ensuring separation of concersn and easier maintenance.

Retry behavior is deterministic: it isolates state, enforces delays, and produces detailed logs for each attempt, aiding debugging of flaky scenarios.