Skip to content

feature(telemetry): Add Telemetry Reporting for Hooks#29

Open
rishiy15 wants to merge 3 commits into
mainfrom
feature/add-hook-telemetry
Open

feature(telemetry): Add Telemetry Reporting for Hooks#29
rishiy15 wants to merge 3 commits into
mainfrom
feature/add-hook-telemetry

Conversation

@rishiy15
Copy link
Copy Markdown
Collaborator

@rishiy15 rishiy15 commented May 13, 2026

What's Changed

Adds opt-in telemetry for agent hook usage so we can understand adoption, plugin-vs-hooks usage, and hook outcomes across agentic IDEs. Events are written locally as JSONL into the 1Password app's hook-events directory and ingested out-of-band by the desktop app — the hook itself never makes network calls.

High-level summary

Area Change
lib/telemetry.sh (new) Consent-gated JSONL writer, schema-aligned event builders (write_execution_event, write_install_event), millisecond clock helper, bucketing, install-method detection, idempotent install sentinel, 1MB file cap.
bin/run-hook.sh Times the hook run, derives telemetry metadata from the canonical output, emits an agent_hook_execution event per invocation, and emits a one-time agent_hook_install event per (client, hook, install_method) combo. All telemetry is wrapped in a fail-open subshell so the hook decision is never affected.
hooks/1password-validate-mounted-env-files/hook.sh Extends canonical output with telemetry metadata (mode, mount_count, deny_reason). Populates resolved_mode, increments total_mount_count per validated mount, and derives deny_reason from the failure mode.
install.sh Copies VERSION into the bundle and writes one agent_hook_install event per installed hook with install_method=install_script.
lib/json.sh Adds extract_json_integer so the runner can pull mount_count out of canonical output.
schemas/hook-output.schema.json Adds optional mode, mount_count, and deny_reason properties — strictly additive, existing hooks remain valid.
VERSION (new) Source of truth for hook_version in telemetry. Set to 1.0.0.

Telemetry event schemas

agent_hook_execution

Field Type Description
hook_name string e.g. 1password-validate-mounted-env-files
hook_version string major.minor.patch from VERSION
client string cursor, claude-code, github-copilot, windsurf, unknown
event_type string e.g. before_shell_execution
decision string (enum) allow, deny
deny_reason string | null file_missing, file_disabled, or null
duration_ms string (enum) Bucketed range: 0-50, 50-100, 100-200, 200-500, 500-1000, 1000-5000, 5000+
mode string (enum) configured (explicit TOML mount_paths) or default (auto-discovery)
mount_count integer Number of env files validated this run

agent_hook_install

Field Type Description
client string IDE the hook was installed for
hook_name string Which hook was installed
install_method string (enum) plugin_marketplace, install_script, unknown

Privacy & safety

  • Opt-in only. Events are written only when ~/.config/1Password/telemetry-enabled exists; the 1Password app creates this file based on the user's choice.
  • Fail open. Telemetry writes are wrapped in a subshell with || true; any failure (permission, disk, parse) is silently swallowed and the hook decision proceeds normally.
  • Bounded disk usage. Writer enforces a 1MB cap on events.jsonl (~4100 events) before any append.
  • No PII. Events contain hook name/version, client, event type, decision, bucketed duration, mode, mount count, deny reason, and install method — no paths, payloads, or environment names.
  • Install idempotency. A per-(client, hook, install_method) sentinel file prevents duplicate install events across runs (e.g. plugin marketplace deployments that bypass install.sh).

Testing Done

  • New tests/lib/telemetry.bats (27 cases) covers bucketing, consent gating, JSONL writes, file-size cap, JSON shape of execution/install events, install sentinel idempotency, and install-method detection (CURSOR_PLUGIN_ROOT, CLAUDE_PLUGIN_ROOT, bundle-path marker, unknown).
  • Extended tests/lib/json.bats with extract_json_integer cases.
  • Updated tests/hooks/1password-validate-mounted-env-files.bats to account for the extended canonical output.
  • Full suite: 192/192 bats tests passing (bats tests/adapters/*.bats tests/hooks/*.bats tests/install/*.bats tests/lib/*.bats).
  • Manually verified telemetry writes by toggling the consent signal file and tailing ~/.config/1Password/data/hook-events/events.jsonl.
  • Manually verified end-to-end telemetry ingestion via a local dev build of the 1Password desktop

@rishiy15 rishiy15 changed the title Add Telemetry Reporting for Hooks feature(telemetry): Add Telemetry Reporting for Hooks May 14, 2026
@rishiy15 rishiy15 marked this pull request as ready for review May 22, 2026 21:09
@rishiy15 rishiy15 requested review from JillRegan and bertrmz May 22, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant