Coverage mask

[martinkilbinger]

Summary

Added library files and classes to create healsparse coverage mask files.
The following steps can be performed:

• Download exposure FITS headers from VOSpace
• Extract CCD corner coordinates
• Create hsp coverage map

Wait for #702 .

Closes #700 .

Reviewer Checklist

Reviewers should tick the following boxes before approving and merging the PR.

• The PR targets the develop branch
• The PR is assigned to the developer
• The PR has appropriate labels
• The PR is included in appropriate projects and/or milestones
• The PR includes a clear description of the proposed changes
• If the PR addresses an open issue the description includes "closes #"
• The code and documentation style match the current standards
• Documentation has been added/updated consistently with the code
• All CI tests are passing
• API docs have been built and checked at least once (if relevant)
• All changed files have been checked and comments provided to the developer
• All of the reviewer's comments have been satisfactorily addressed by the developer

[cailmdaley]

Merged latest develop into this branch (no force-push — just a merge commit). This absorbs #702, shrinking the effective PR diff from 43 files / ~29.6k lines to 11 files / +2036 / -165, which makes the genuinely new coverage-mask code reviewable.

Conflicts resolved:

• pyproject.toml — kept both sides: fitsio extra from develop + plot in dev from this branch
• docs/source/pipeline_canfar.md — kept develop's new "Create matched star catalogue" section; kept this branch's richer "Create coverage mask" docs (including the -d headers_v1.3 symlink-reuse notes and full build_coverage_map command)

Nothing structural was touched. Full review to follow.

[cailmdaley]

Review of coverage-mask work

Good overall — the division of labor between this PR (builds coverage from exposure polygons) and sp_validation (consumes/applies/validates masks) is clean, and using cs_util.plots.FootprintPlotter for plotting is the right factoring.

Note on scope: after merging develop into this branch, the effective diff is 11 files / +2036 / −165 — concentrated on the 8 genuinely new files.

Blocking

• coverage_run.py:97–139 — run_pipeline can't work as written. All three stages (run_download_headers, run_extract_corners, run_build_coverage) are called with the same args, but each uses -i/-o/etc. with different meanings. The coverage_pipeline entry point in pyproject.toml therefore runs only the first stage correctly. Either prefix options per stage, or drive the pipeline from a config file.
• coverage_map_builder.py:19 vs :321 contradict. Module-level from shapepipe.utilities.coverage_plotter import CoveragePlotter runs at import time — if the plot stack is missing, the whole module fails before reaching the except ImportError at L321. That try/except is unreachable. Either move the import into the try block, or drop the try entirely.
• coverage_map_builder.py:323 — wrong dependency in error message. Says "Install sp_validation package for plotting support". Should say cs_util — CoveragePlotter's hard dep is cs_util.plots.FootprintPlotter, not sp_validation.
• field_corners_extractor.py:135 — expnum = int(path[end - 6 : end]) hardcodes 6-digit exposure numbers. Silently corrupts if an exposure number ever exceeds 999999. Parse by the - separator in the CCD filename, or use a regex.

Non-blocking but worth fixing

• field_corners_extractor.py:149–152 — the hardcoded MegaCam HDU indices (0, 8, 35, 27) and pixel coords (2079, 0, 32) work but are magic. At minimum a short comment naming the MegaCam corner convention.
• field_corners_extractor.py has two copies of the same WCS-parsing logic — process_single_header (static, L113–162, exists only so multiprocessing can pickle it) and get_wcs_from_header (L164–195). Extract to a module-level helper.
• coverage_plotter.py:237 — FootprintPlotter._regions[region] reaches into a single-underscore private attribute. Fragile. Either cs_util should expose a public accessor for region lookup, or duplicate the dict locally.
• coverage_map_builder.py:175 — new_hsp = hsp_map + 0 is an opaque copy idiom. Use .copy() if healsparse has it, otherwise a one-line comment explaining why +0.
• coverage_map_builder.py:224 — verbose = self._params["verbose"] but verbose is never declared in params_default. Relies on cs_util.args.parse_options injecting it. Declare it explicitly in params_default so the contract is local.
• header_downloader.py:36 — vospace_path = "vos:cfis/pitcairn" is a fine default but CFIS-specific. Worth a prominent note in help string or docs.
• build_and_plot_coverage_maps.sh — BUILD_NSIDE=131072 (~0.1″ resolution). The Python class default in coverage_map_builder.py:39 is nside=2048. The docs say 131072 matches the bit-mask resolution — could you confirm that's the intended pairing, and maybe note it in a comment at the top of the bash script?

Tests

No tests for any of the new modules. Same gap as #702. A small test at least for CoverageMapBuilder.median_filter and FieldCornersExtractor.process_single_header would catch a lot. Not a blocker for merge, but worth tracking.

Side note on FootprintPlotter

Not a concern for this PR (you correctly import from cs_util), but flagging: sp_validation has a stale copy of FootprintPlotter at src/sp_validation/plots.py:402 that has diverged from cs_util's. Opening a separate cleanup PR on sp_validation to delete the local copy and import from cs_util.

[martinkilbinger]

I fixed some of the problems:

• coverage_run.py:97–139 — run_pipeline: Fixed, removed this non-functioning pipeline runner.
• coverage_map_builder.py: Fixed.

And added comments for the hard-coded MegaCAM/UNIONS stuff.

For the rest I think claude can do a better job than me to fix.

[cailmdaley]

Thanks @martinkilbinger — your three fixes close out the blocking items (the bogus run_pipeline, the unreachable ImportError guard, the sp_validation→cs_util error string). I walked through the rest; current state:

Addressed on this branch:

• field_corners_extractor.py — regex bug fixed; process_single_header and get_wcs_from_header consolidated into one helper; MegaCam HDU/pixel conventions commented (bbe53c5)
• coverage_map_builder.py — hsp_map + 0 → .copy() (134a79f)
• coverage_map_builder.py — verbose declared explicitly in params_default; header_downloader.py vospace default noted as CFIS-specific; build_and_plot_coverage_maps.sh has a comment explaining BUILD_NSIDE=131072 matches UNIONS bit-mask resolution vs the class default of 2048 for lighter-weight offline use (9a574e9)

Left as-is (with comment):

• coverage_plotter.py:166, 241 still reaches into FootprintPlotter._regions. I added a comment at L161–163 flagging it and pointing at cs_util; the real fix is upstream (expose regions as a public accessor). Not worth duplicating the 3-region dict locally given cs_util should be the single source of truth.

Non-blocking gap to track (not in this PR):

• No tests for the new coverage modules. Targets worth covering later: CoverageMapBuilder.median_filter and FieldCornersExtractor.process_single_header — a small synthetic header + tiny hsp map would lock in the load-bearing numerics. Happy to file an issue or pick it up after this merges, whichever you prefer.

From my side the PR is ready for a final review / merge. LMK if you want me to touch anything else.