Python(feat): pytest graceful handle missing connection#569
Draft
alexluck-sift wants to merge 12 commits into
Draft
Python(feat): pytest graceful handle missing connection#569alexluck-sift wants to merge 12 commits into
alexluck-sift wants to merge 12 commits into
Conversation
Contributor
|
Python docs preview: https://sift-stack.github.io/sift/python/pr-569/ Deployed from |
alexluck-sift
changed the base branch from
main
to
al/python/feat/pytest-dev-branch
alexluck-sift
changed the base branch from
al/python/feat/pytest-dev-branch
to
al/python/pytest-plugin-idiomatic
alexluck-sift
force-pushed
the
al/python/graceful-handle-missing-connection
branch
4 times, most recently
from
67afc3b to
2483944
Compare
Base automatically changed from
al/python/pytest-plugin-idiomatic
to
al/python/feat/pytest-dev-branch
May 19, 2026 13:25
…ffline Adds ``value_passes_bounds``, ``to_numpy_array``, ``out_of_bounds_mask``, and ``all_within_bounds`` to ``bounds`` so consumers that need pass/fail semantics without recording a measurement (e.g. a stub plugin mode) can share the same evaluation logic. ``evaluate_measurement_bounds`` now delegates to ``value_passes_bounds`` to keep the two paths in sync. Adds an ``offline`` flag to ``ReportContext``: when set, the context auto-creates a temp log file if none is provided and skips spawning the import-test-result-log replay subprocess at session end. The log file is the sole sink in offline mode. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…t-offline and --sift-disabled Replaces the per-fixture connection gate from #567 (which made `report_context` / `step` / `module_substep` yield None on a failed ping, breaking user code that calls `step.measure(...)`) with two explicit modes: * `--sift-offline` writes every create/update to the JSONL log file for later replay via `import-test-result-log`. The session-start ping is skipped and the replay subprocess is not spawned at session end. Combining with `--sift-log-file=none` is rejected as a usage error since the log file is the sole sink. * `--sift-disabled` skips Sift entirely. Autouse fixtures yield stub objects (`_NoopStep`, `_NoopReportContext`) that share the bounds helpers from `bounds`, so `step.measure(...)` still returns real pass/fail booleans without any Sift configuration. Also honored via the `SIFT_DISABLED` env var. The two modes are mutually exclusive. Online mode now fail-fasts at session start: a failed ping aborts the session with `pytest.UsageError` naming both flags as escape hatches, instead of letting fixtures silently no-op. Preserves #567's marker system (`sift_include` / `sift_exclude` + `sift_test_results_autouse` ini key), lazy `report_context`, per-module gate, URI ini fallbacks, and the `_Option` CLI+ini plumbing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrites the "Running offline" section into a unified three-mode table (online / offline / disabled) and replaces the old check-connection + None-handling guidance with the new fail-fast and stub-yielding designs. Repo conftest defaults to --sift-disabled for unit-test runs and stays online (log-file inline) for integration runs against a real backend. addopts now loads the plugin globally so the conftest can flip modes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
alexluck-sift
force-pushed
the
al/python/graceful-handle-missing-connection
branch
from
3ae545c to
c403740
Compare
…d of erroring Disabled mode is the "skip Sift entirely" hammer; combining it with offline (or anything else) shouldn't be a usage error since disabled already skips the work the other flag describes. Drops the mutex check in `pytest_configure`. The existing fixture ordering already makes disabled win: `report_context` and `sift_client` short-circuit on `_is_disabled` before consulting `_is_offline`. Updates the help text, docs, and the test that previously asserted the error. The only remaining incompatibility is `--sift-offline` + `--sift-test-results-log-file=none`, which still errors because offline mode genuinely needs the log file as its sole sink. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…test The previous conditional ``is_integration_run and have_real_backend`` silently fell back to ``--sift-disabled`` when someone ran ``-m integration`` without ``SIFT_*`` env vars set, which both swallowed the missing-credentials usage error and ran integration tests against the disabled-mode stubs instead of the real backend. The credential check belongs to the plugin's ``sift_client`` fixture, which already raises a ``pytest.UsageError`` naming the missing env vars. ``-m integration`` is a commitment; let that error fire. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests that probe what each mode actually does, not just whether the session exits 0: * Disabled: ``step`` / ``report_context`` / ``module_substep`` are ``_NoopStep`` / ``_NoopReportContext`` instances; no log file is written even when a path is pinned; ``sift_client`` and ``client_has_connection`` are never resolved (raising overrides stay un-triggered). * Offline: ``step`` / ``report_context`` are the real ``NewStep`` / ``ReportContext``; the pinned JSONL log file contains ``[CreateTestReport:...]`` and ``[CreateTestStep:...]`` lines after the run; ``client_has_connection`` is never resolved. * Online: ``client_has_connection`` is resolved exactly once at session start (counter file written by the override). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Target handled missing connection via
--sift-test-results-check-connection, which yieldedNonefromstep/report_context/module_substepon ping failure. Any test callingstep.measure(...)then crashed withAttributeError, forcingif step is None: pytest.skip(...)guards throughout user code. The flag name also didn't convey what it actually did.Changes
--sift-offline: skips the ping, routes create/update calls to a JSONL log file for replay viaimport-test-result-log, tolerates missingSIFT_*env vars.--sift-disabled(alsoSIFT_DISABLED=1): yields stub objects that still evaluate bounds locally, sostep.measure(...)keeps returning a real pass/fail boolean. No network, log file, or credentials.offline(write to log) vsdisabled(skip entirely).pytest.UsageErrornaming both flags as escape hatches.bounds.pyso the realNewStepand the stub share the same pass/fail logic.sift_test_results_*→sift_*).