feat: Phase 3f — comprehensive end-to-end integration test

[g-talbot]

Summary

Stacked on #6354 (Phase 3d+3e). Comprehensive integration test for the Parquet merge pipeline with thorough output verification.

Integration test

Exercises the full merge actor chain in-process with RamStorage and mock metastore using diverse inputs (different metrics, timestamps, tags per split):

1. Creates 2 real sorted Parquet files with different data (cpu.usage vs mem.usage, disjoint timestamps, different service/host tags)
2. Uploads to RamStorage, seeds ParquetMergePipeline with split metadata (merge_factor=2)
3. Verifies the pipeline plans a merge, downloads, merges, uploads, and publishes

What the test verifies

Staged metadata (captured from mock metastore):

• num_rows == 100 (MC-1 row preservation)
• time_range == (100, 250) (union of inputs)
• metric_names == {cpu.usage, mem.usage}
• num_merge_ops == 1, sort_fields preserved, row_keys_proto present
• zonemap_regexes present with metric_name entry
• service tags == {web, api}

Parquet file contents (read back from RamStorage):

• Row count matches metadata
• All metric names, timestamps, service/host values present
• sorted_series monotonically non-decreasing (sort invariant)
• cpu.usage rows precede mem.usage rows (sort schema semantics)

KV metadata headers: qh.sort_fields, qh.num_merge_ops, qh.row_keys, qh.zonemap_regexes

Cross-validation: metadata agrees with file contents (timestamps, row count, zonemap regexes)

Known limitation documented

Only "service" tags are extracted in both the ingest and merge paths. Host/env/datacenter/region columns exist in the data but are not yet tracked in split metadata.

Test plan

• Integration test passes (full merge pipeline e2e with data verification)
• cargo clippy clean
• cargo +nightly fmt clean
• Compiles with and without metrics feature

🤖 Generated with Claude Code

[mattmkim]

couple comments