Background
The osism/python-osism repository currently has no unit tests at all:
- No
tests/ directory exists
- No pytest configuration (neither in
setup.cfg, pyproject.toml, nor as pytest.ini)
- No test dependencies in
Pipfile or requirements.txt
- The Zuul job
python-osism-test-setup (see playbooks/test-setup.yml) only installs the package via pipenv install && pipenv run pip install . but does not run any tests
The goal of this issue is to establish the foundation so that unit tests can be written and automatically executed in Zuul CI. Actual test coverage of the modules (osism/commands/, osism/utils/, osism/tasks/, osism/services/, osism/api.py, …) will be addressed in follow-up issues.
Scope
In scope (foundation only):
- Set up the test framework (pytest)
- Establish directory structure and configuration
- One or two minimal smoke tests so the CI pipeline passes and the infrastructure is validated
- Zuul integration: new job
python-osism-unit-tests running in the check pipeline
- Documentation on how to run tests locally
Out of scope:
- Full test coverage of existing modules
- Integration / end-to-end tests (the existing
pytest-testinfra-based scenarios remain untouched)
- Coverage gates / thresholds (can be added later)
Tasks
1. Test dependencies
Add a new [dev-packages] section in Pipfile (or alternatively move them into a separate requirements.test.txt):
pytest
pytest-cov
pytest-mock
Versions should be pinned, consistent with the existing style in Pipfile.
2. Directory structure
tests/
├── __init__.py
├── conftest.py
└── unit/
├── __init__.py
└── test_smoke.py
conftest.py initially only contains a placeholder for shared fixtures to be added later.
tests/unit/test_smoke.py contains a trivial test (e.g. import osism / version check) so that a failure in the test infrastructure is immediately visible.
3. pytest configuration
Add a [tool:pytest] section to setup.cfg (consistent with the existing setup):
[tool:pytest]
testpaths = tests/unit
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts = -ra --strict-markers
Do not set a coverage threshold — only enable optional reporting via --cov=osism.
4. Zuul CI integration
In .zuul.yaml:
Create a new playbook playbooks/test-unit.yml that:
- Runs
pipenv install --dev
- Installs the package via
pipenv run pip install .
- Executes
pipenv run pytest and uses the exit code as the job result.
5. Local usage & documentation
- Short section in
README.md (or a new CONTRIBUTING.md) describing:
pipenv install --dev
pipenv run pytest for the full run
pipenv run pytest tests/unit/test_smoke.py to run a single test
6. Verification
Open questions for review
- Should the existing
python-osism-test-setup job be replaced by python-osism-unit-tests or run in parallel? Recommendation: replace, since the existing job performs no verification that is not already covered by the build.
- Test dependencies as
[dev-packages] in Pipfile or as a separate requirements.test.txt? Recommendation: [dev-packages] in Pipfile, to preserve the existing workflow.
- Whether the Python version for the test job should be pinned (e.g. only 3.11) or multiple versions in a matrix — in the scope of this issue: only 3.11, additional versions in a follow-up issue.
Definition of Done
- All tasks in sections 1–5 implemented
- Zuul
check pipeline contains and passes the new python-osism-unit-tests job
- A minimal smoke test runs successfully in CI
- Developers can run tests locally with two commands (
pipenv install --dev, pipenv run pytest)
Background
The
osism/python-osismrepository currently has no unit tests at all:tests/directory existssetup.cfg,pyproject.toml, nor aspytest.ini)Pipfileorrequirements.txtpython-osism-test-setup(seeplaybooks/test-setup.yml) only installs the package viapipenv install && pipenv run pip install .but does not run any testsThe goal of this issue is to establish the foundation so that unit tests can be written and automatically executed in Zuul CI. Actual test coverage of the modules (
osism/commands/,osism/utils/,osism/tasks/,osism/services/,osism/api.py, …) will be addressed in follow-up issues.Scope
In scope (foundation only):
python-osism-unit-testsrunning in thecheckpipelineOut of scope:
pytest-testinfra-based scenarios remain untouched)Tasks
1. Test dependencies
Add a new
[dev-packages]section inPipfile(or alternatively move them into a separaterequirements.test.txt):pytestpytest-covpytest-mockVersions should be pinned, consistent with the existing style in
Pipfile.2. Directory structure
conftest.pyinitially only contains a placeholder for shared fixtures to be added later.tests/unit/test_smoke.pycontains a trivial test (e.g.import osism/ version check) so that a failure in the test infrastructure is immediately visible.3. pytest configuration
Add a
[tool:pytest]section tosetup.cfg(consistent with the existing setup):Do not set a coverage threshold — only enable optional reporting via
--cov=osism.4. Zuul CI integration
In
.zuul.yaml:Define a new job
python-osism-unit-tests(analogous topython-osism-test-setup):Add the job to the
checkpipeline and toperiodic-daily.Create a new playbook
playbooks/test-unit.ymlthat:pipenv install --devpipenv run pip install .pipenv run pytestand uses the exit code as the job result.5. Local usage & documentation
README.md(or a newCONTRIBUTING.md) describing:pipenv install --devpipenv run pytestfor the full runpipenv run pytest tests/unit/test_smoke.pyto run a single test6. Verification
pipenv run pytestruns green locallycheckpipeline and succeedspython-osism-test-setupjob remains functional (or is explicitly replaced by the new job — to be decided in review)flake8,mypy,python-blackremain green (new files must comply with the rules in.flake8)Open questions for review
python-osism-test-setupjob be replaced bypython-osism-unit-testsor run in parallel? Recommendation: replace, since the existing job performs no verification that is not already covered by the build.[dev-packages]inPipfileor as a separaterequirements.test.txt? Recommendation:[dev-packages]inPipfile, to preserve the existing workflow.Definition of Done
checkpipeline contains and passes the newpython-osism-unit-testsjobpipenv install --dev,pipenv run pytest)