Introduction

The Allure Framework is a lightweight, flexible, multi-language test reporting tool. It presents test results in a clean web-based format and helps everyone involved in the development process extract meaningful information from daily test executions.

From a development and testing perspective:

Allure reports enable quick identification of failure points, allowing you to categorize test failures as bugs or broken tests. You can configure logging, steps, fixtures, attachments, timing, history, and integrate with test management systems and bug tracking tools to maintain complete visibility.

From a management perspective:

Allure provides a clear overview covering feature coverage, failure clustering, execution schedules, and other useful metrics. Its modular and extensible architecture allows fine-tuning to match your specific requirements.

Setup

Pytest is a highly extensible and powerful automation testing framwork, but its native output is relatively basic. For comprehensive test reporting, additional plugins are required.

For basic reporting needs, the pytest-html plugin covers standard requirements. However, for detailed test process visualization, multi-dimensional reports, custom output options, and integration with test cases and bug systems, allure-python is the ideal choice.

Note: The allure-pytest package was deprecated after version 1.7 and migrated to the allure-python project (Allure 2). Java runtime is also required for the Allure command-line tool.

Installation

Step 1: Install the Allure Pytest Plugin

pip install -U allure-pytest

This installs the allure-pytest and allure-python-commons packages to generate Allure 2 compatible report data.

Step 2: Install the Allure Command-Line Tool

Download the latest release from: Allure Releases on GitHub

Extract the archive (recommended to a location like your Python installation folder), then add the bin directory to your system PATH. Verify the installation with:

allure --version

Basic Usage

Collecting Results

To enable Allure listeners during test execution, add the --alluredir option with a target directory:

pytest --alluredir=<directory-with-results>

To clear previous results before running, include the --clean-alluredir flag:

pytest --alluredir=<directory-with-results> --clean-alluredir

Generating Reports

After test execution, generate the actual report using the Allure CLI:

Display report in default browser:

allure serve <my-allure-results>

Generate report from existing results:

allure generate <directory-with-results>

By default, the report generates to the allure-report folder. Use the -o flag to specify a different output directory:

allure generate <directory-with-results> -o <directory-with-report>

Open generated report in browser:

allure open <directory-with-report>

Alternatively, open the index.html file directly in your browser. If you encounter missing data or encoding issues when opening locally, use PyCharm's browser preview or run the allure serve command instead.

Clean generated reports:

allure report clean

Allure commands search for results in the allure-results folder by default. Use the -o option to specify alternative locations.

Additional help:

allure help

Test Report Overview

Allure reports display all standard Pytest statuses. Tests failing due to assertion errors are marked as failed, while other exceptions result in a broken status.

Example Test Suite

# test_calculator.py

import pytest


def calculate_sum(a, b):
    return a + b


class TestCalculator:

    def test_skipped_case(self):
        pytest.skip('Skipping this test')
        assert calculate_sum(3, 4) == 7

    def test_exception_case(self):
        assert calculate_sum(-3, 4) == 1
        raise RuntimeError('Something went wrong')

    def test_passed_case(self):
        assert calculate_sum(3, -4) == -1

    def test_failed_case(self):
        assert calculate_sum(-3, -4) == 7

# conftest.py

import pytest


@pytest.fixture(scope='session', autouse=True)
def database_connection():
    print('Connecting to database')
    yield
    print('Closing database connection')

Execution Output

$ pytest test_calculator.py --alluredir=report --clean-alluredir

====== test session starts ======
platform win32 -- Python 3.7.3, pytest-6.0.2
collected 4 items

test_calculator.py sF.F [100%]

====== FAILURES ======

---- TestCalculator.test_exception_case ----

    def test_exception_case(self):
        assert calculate_sum(-3, 4) == 1
        raise RuntimeError('Something went wrong')
        E RuntimeError: Something went wrong

---- TestCalculator.test_failed_case ----

    def test_failed_case(self):
        assert calculate_sum(-3, -4) == 7
        E assert -7 == 7

====== short test summary info ======
FAILED test_calculator.py::TestCalculator::test_exception_case
FAILED test_calculator.py::TestCalculator::test_failed_case
====== 2 failed, 1 passed, 1 skipped in 0.14s ======

Generating and Viewing Reports

$ allure generate --clean report
Report successfully generated to allure-report

Report Directory Structure

.
├── allure-report/
│   ├── data/
│   │   ├── attachments/
│   │   └── test-cases/
│   ├── export/
│   ├── history/
│   ├── plugins/
│   │   ├── behaviors/
│   │   ├── jira/
│   │   ├── packages/
│   │   ├── screen-diff/
│   │   └── xunit-xml/
│   └── widgets/
└── report/

Report Sections

Overview: Executive summary showing test execution status, severity distribution, and environment information.

Categories: Tests grouped by execution result, distinguishing between broken tests and failed assertions.

Suites: Hierarchical view organized as directory → test file → test class → test method.

Graphs: Visual representations including status distribution, severity breakdown, duration analysis, and trend metrics.

Timeline: Chronological view of test execution with precise timestamps for each test case.

Behaviors: Tests grouped by behavior patterns (requires Allure decorators on test cases).

Packages: Directory-based organization showing test execution across different package structures.

Advanced Features: Test Case Details

Allure captures more than test results. It can display parameters from parameterized tests, error details, and fixture information.

# test_parameterized.py

import pytest
import allure


def multiply(x, y):
    return x * y


@allure.feature("Arithmetic Operations")
class TestMathOperations:

    test_data = [
        [2, 3, 6],
        [-2, 3, -6],
        [2, -3, -6],
        [-2, -3, 6],
    ]

    @allure.story("Multiplication Tests")
    @allure.severity(allure.severity_level.NORMAL)
    @pytest.mark.parametrize("data", test_data)
    def test_multiplication(self, data):
        assert multiply(data[0], data[1]) == data[2]

Key decorators explained:

• @allure.feature: Groups tests by feature/module
• @allure.story: Groups tests by user story or specific scenario
• @allure.severity: Marks test priority levels (blocker, critical, normal, minor, trivial)