Skip to content

feat(cli): forward perf breakdown + tmpPeakBytes to PostHog telemetry#454

Merged
jrusso1020 merged 1 commit intomainfrom
perf/v2/cli-telemetry-breakdown
Apr 24, 2026
Merged

feat(cli): forward perf breakdown + tmpPeakBytes to PostHog telemetry#454
jrusso1020 merged 1 commit intomainfrom
perf/v2/cli-telemetry-breakdown

Conversation

@jrusso1020
Copy link
Copy Markdown
Collaborator

@jrusso1020 jrusso1020 commented Apr 23, 2026

What

Forwards the new per-phase extraction breakdown and tmpPeakBytes fields from RenderPerfSummary (added in #444 and #446) to PostHog via the CLI's existing render_complete telemetry event.

Why

The CLI already ships render_complete events to PostHog (packages/cli/src/telemetry/client.ts), but events.ts:trackRenderComplete only carried a subset of RenderPerfSummary — top-level timings, composition dims, and memory snapshots. After #444 added per-phase extraction breakdown (videoExtractBreakdown) and #446 added cache hit/miss counters, the data lives on job.perfSummary at render-complete but never reaches PostHog dashboards.

Without this, any PostHog insight built around "how often are we hitting the cache?", "what's the median HDR preflight cost?", or "where in the extract phase do compositions spend time?" has to be answered by Datadog log scraping instead.

How

  • packages/cli/src/telemetry/events.ts — extend trackRenderComplete props with 17 new optional fields: tmpPeakBytes, the six named stage timings, and the ten videoExtractBreakdown fields. All sent as flat properties (extract_cache_hits, stage_capture_ms, etc.) — PostHog insights query flat keys more ergonomically than nested objects.
  • packages/cli/src/commands/render.ts — wire job.perfSummary.videoExtractBreakdown / stages / tmpPeakBytes into the trackRenderMetricstrackRenderComplete hand-off.
  • Naming: extract_phase3_ms deliberately disambiguates from stage_video_extract_ms — the former is just the parallel ffmpeg extract inside Phase 3; the latter is the full stage (resolve + probe + preflight + extract).
  • All new fields are optional. The Docker-subprocess branch of render.ts that doesn't have a local perfSummary still compiles and ships events without them.

Test plan

  • bun run --cwd packages/cli test — 161/161 pass
  • bunx tsc -p packages/cli/tsconfig.json --noEmit — no errors
  • bunx oxlint + bunx oxfmt — clean
  • Once merged, verify PostHog receives the new properties on a real render event (run hyperframes render against a fixture and watch PostHog ingestion — telemetry auto-disables in CI, so this requires a local dev render with HYPERFRAMES_NO_TELEMETRY unset).

Stack

Depends on #444 (adds the videoExtractBreakdown + tmpPeakBytes fields to RenderPerfSummary) and transitively on #445#446.

Future work (not in this PR)

  • The HeyGen internal producer server (hyperframes-internal/packages/producer/src/server.ts) logs perfSummary to Datadog via log.info but has no PostHog integration. Production renders are the bulk of the traffic — separate PR to either ship perfSummary to PostHog from the internal server, or materialize Datadog log-based metrics for per-phase timings.

Copy link
Copy Markdown
Collaborator Author

jrusso1020 commented Apr 23, 2026

Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: feat(cli): forward perf breakdown + tmpPeakBytes to PostHog telemetry

Clean, mechanical wiring PR. 2 files, +56/-0, all additive. No issues.

What it does

Forwards 17 new optional fields from RenderPerfSummary to PostHog's render_complete event: tmpPeakBytes, 6 stage timings, and 10 videoExtractBreakdown fields (including cache hit/miss counters from #446).

What I checked

  • Naming consistency — snake_case PostHog keys match the existing pattern. extract_phase3_ms vs stage_video_extract_ms disambiguation is correct and documented in the PR body.
  • All fields optional — Docker-subprocess branch without a local perfSummary still compiles and ships events without them. Confirmed by the nullish chaining on perf?.stages, perf?.videoExtractBreakdown, perf?.tmpPeakBytes.
  • No naming conflicts — the ...getMemorySnapshot() spread at the end uses peak_memory_mb / memory_free_mb, no overlap with the new stage_* / extract_* keys.
  • Flat property layout — correct for PostHog. Nested objects would require JSON stringification and make insights queries harder.
  • No cardinality concerns — all values are numeric (timings in ms, byte counts, integer counters). PostHog handles this fine.

One note (not blocking)

trackEvent will receive undefined for all 17 new properties when perf is null (the Docker-subprocess path). Most PostHog SDKs strip undefined values before sending, so this is fine — but if the SDK sends them as explicit nulls, it could clutter the event schema. Worth a quick check on the first real ingestion event (already in the test plan).

Verdict: Approve

No issues. Ship it.

@jrusso1020 jrusso1020 force-pushed the perf/v2/extraction-cache branch from 8a31b2c to 3c36324 Compare April 23, 2026 22:10
@jrusso1020 jrusso1020 force-pushed the perf/v2/cli-telemetry-breakdown branch from e4ef982 to 406e281 Compare April 23, 2026 22:11
@jrusso1020
Copy link
Copy Markdown
Collaborator Author

Rebased the stack onto current main + onto the updated #446. No code changes — <https://github.com/heygen-com/hyperframes/pull/454#pullrequestreview-4165039347|your review> was clean approval, and the downstream fixes on #444/#445/#446 don't touch any of the 17 new trackRenderComplete fields, so the wiring stays identical. bun run --cwd packages/cli test → 161/161 pass, typecheck clean.

On your non-blocking observation about undefined values on the Docker-subprocess path: still in the test plan to eyeball the first real ingestion event. PostHog's node SDK strips undefined before sending (confirmed via the SDK's normalize path), so worst case the schema shows the keys only on renders that carried a real perfSummary. Will spot-check once this lands.

@jrusso1020 jrusso1020 force-pushed the perf/v2/extraction-cache branch from 3c36324 to a0f49f2 Compare April 23, 2026 22:43
@jrusso1020 jrusso1020 force-pushed the perf/v2/cli-telemetry-breakdown branch from 406e281 to 6d11a66 Compare April 23, 2026 22:43
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked the live stacked head. The telemetry wiring remains clean on top of the updated engine stack, local CLI validation is clean, and the latest CI is green.

@jrusso1020 jrusso1020 force-pushed the perf/v2/extraction-cache branch from a0f49f2 to b231f61 Compare April 24, 2026 03:55
@jrusso1020 jrusso1020 force-pushed the perf/v2/cli-telemetry-breakdown branch from 6d11a66 to 7085490 Compare April 24, 2026 03:56
@jrusso1020 jrusso1020 force-pushed the perf/v2/extraction-cache branch 2 times, most recently from 1a669be to ba05796 Compare April 24, 2026 04:15
@jrusso1020 jrusso1020 force-pushed the perf/v2/cli-telemetry-breakdown branch from 7085490 to 74112a4 Compare April 24, 2026 04:15
@jrusso1020 jrusso1020 changed the base branch from perf/v2/extraction-cache to graphite-base/454 April 24, 2026 04:35
@jrusso1020 jrusso1020 force-pushed the perf/v2/cli-telemetry-breakdown branch from 74112a4 to 330b112 Compare April 24, 2026 04:35
@graphite-app graphite-app Bot changed the base branch from graphite-base/454 to main April 24, 2026 04:36
The CLI already ships `render_complete` events to PostHog but only carries
a subset of `RenderPerfSummary` (duration_ms, capture_avg_ms, etc.). With
#444 adding per-phase breakdown and #446 adding cache counters, the data
is sitting in `job.perfSummary.videoExtractBreakdown` and
`job.perfSummary.tmpPeakBytes` but never reaching PostHog.

This forwards the new fields as flat properties (flat is easier to query
with PostHog insights than nested objects):

- tmp_peak_bytes
- stage_compile_ms, stage_video_extract_ms, stage_audio_process_ms,
  stage_capture_ms, stage_encode_ms, stage_assemble_ms
- extract_resolve_ms, extract_hdr_probe_ms, extract_hdr_preflight_ms,
  extract_hdr_preflight_count, extract_vfr_probe_ms, extract_vfr_preflight_ms,
  extract_vfr_preflight_count, extract_phase3_ms, extract_cache_hits,
  extract_cache_misses

`extract_phase3_ms` disambiguates from `stage_video_extract_ms`: the former
is just the parallel ffmpeg extract inside Phase 3, the latter is the full
stage (resolve + probe + preflight + extract).

All fields optional — the Docker subprocess call site (no perfSummary
available) still compiles and ships events without them.
@jrusso1020 jrusso1020 force-pushed the perf/v2/cli-telemetry-breakdown branch from 330b112 to 3760bae Compare April 24, 2026 04:36
@jrusso1020 jrusso1020 merged commit b42a8e4 into main Apr 24, 2026
25 checks passed
Copy link
Copy Markdown
Collaborator Author

Merge activity

@jrusso1020 jrusso1020 deleted the perf/v2/cli-telemetry-breakdown branch April 24, 2026 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants