Skip to content

Introduce foundation for unit tests (pytest + Zuul CI integration) #2192

@berendt

Description

@berendt

Background

The osism/python-osism repository currently has no unit tests at all:

  • No tests/ directory exists
  • No pytest configuration (neither in setup.cfg, pyproject.toml, nor as pytest.ini)
  • No test dependencies in Pipfile or requirements.txt
  • The Zuul job python-osism-test-setup (see playbooks/test-setup.yml) only installs the package via pipenv install && pipenv run pip install . but does not run any tests

The goal of this issue is to establish the foundation so that unit tests can be written and automatically executed in Zuul CI. Actual test coverage of the modules (osism/commands/, osism/utils/, osism/tasks/, osism/services/, osism/api.py, …) will be addressed in follow-up issues.

Scope

In scope (foundation only):

  • Set up the test framework (pytest)
  • Establish directory structure and configuration
  • One or two minimal smoke tests so the CI pipeline passes and the infrastructure is validated
  • Zuul integration: new job python-osism-unit-tests running in the check pipeline
  • Documentation on how to run tests locally

Out of scope:

  • Full test coverage of existing modules
  • Integration / end-to-end tests (the existing pytest-testinfra-based scenarios remain untouched)
  • Coverage gates / thresholds (can be added later)

Tasks

1. Test dependencies

Add a new [dev-packages] section in Pipfile (or alternatively move them into a separate requirements.test.txt):

  • pytest
  • pytest-cov
  • pytest-mock

Versions should be pinned, consistent with the existing style in Pipfile.

2. Directory structure

tests/
├── __init__.py
├── conftest.py
└── unit/
    ├── __init__.py
    └── test_smoke.py
  • conftest.py initially only contains a placeholder for shared fixtures to be added later.
  • tests/unit/test_smoke.py contains a trivial test (e.g. import osism / version check) so that a failure in the test infrastructure is immediately visible.

3. pytest configuration

Add a [tool:pytest] section to setup.cfg (consistent with the existing setup):

[tool:pytest]
testpaths = tests/unit
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts = -ra --strict-markers

Do not set a coverage threshold — only enable optional reporting via --cov=osism.

4. Zuul CI integration

In .zuul.yaml:

  • Define a new job python-osism-unit-tests (analogous to python-osism-test-setup):

    - job:
        name: python-osism-unit-tests
        pre-run: playbooks/pre.yml
        run: playbooks/test-unit.yml
  • Add the job to the check pipeline and to periodic-daily.

Create a new playbook playbooks/test-unit.yml that:

  1. Runs pipenv install --dev
  2. Installs the package via pipenv run pip install .
  3. Executes pipenv run pytest and uses the exit code as the job result.

5. Local usage & documentation

  • Short section in README.md (or a new CONTRIBUTING.md) describing:
    • pipenv install --dev
    • pipenv run pytest for the full run
    • pipenv run pytest tests/unit/test_smoke.py to run a single test

6. Verification

  • pipenv run pytest runs green locally
  • The new Zuul job appears in the check pipeline and succeeds
  • The existing python-osism-test-setup job remains functional (or is explicitly replaced by the new job — to be decided in review)
  • flake8, mypy, python-black remain green (new files must comply with the rules in .flake8)

Open questions for review

  1. Should the existing python-osism-test-setup job be replaced by python-osism-unit-tests or run in parallel? Recommendation: replace, since the existing job performs no verification that is not already covered by the build.
  2. Test dependencies as [dev-packages] in Pipfile or as a separate requirements.test.txt? Recommendation: [dev-packages] in Pipfile, to preserve the existing workflow.
  3. Whether the Python version for the test job should be pinned (e.g. only 3.11) or multiple versions in a matrix — in the scope of this issue: only 3.11, additional versions in a follow-up issue.

Definition of Done

  • All tasks in sections 1–5 implemented
  • Zuul check pipeline contains and passes the new python-osism-unit-tests job
  • A minimal smoke test runs successfully in CI
  • Developers can run tests locally with two commands (pipenv install --dev, pipenv run pytest)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions