Skip to content

feat(reward): sentinel-based exchange-count bypass for cron episodes#1848

Open
chiefmojo wants to merge 1 commit into
MemTensor:mainfrom
chiefmojo:pr/cron-episode-scoring-bypass
Open

feat(reward): sentinel-based exchange-count bypass for cron episodes#1848
chiefmojo wants to merge 1 commit into
MemTensor:mainfrom
chiefmojo:pr/cron-episode-scoring-bypass

Conversation

@chiefmojo
Copy link
Copy Markdown

Summary

Adds a config-driven bypass so cron-initiated episodes can be scored even when they fall below the minExchangesForCompletion threshold.

Problem

Scheduled agent jobs produce single-exchange episodes (the cron prompt + agent response). With the default minExchangesForCompletion: 1 floor these score fine, but tighter settings — or multi-turn flows that cron fires as one composite turn — hit the "too few exchanges" skip condition. The cron session produces real work worth scoring but the exchange-count gate silently discards it.

Solution

RewardConfig.cronSentinels: string[] — an array of substrings. If any user message in the episode starts with one of these strings, the exchange-count check (gate 1) is bypassed. All other quality gates (content length, triviality, tool-heavy ratio) remain active.

The default is an empty array ([]), so behavior is unchanged for existing deployments. Operators who run scheduled agent jobs add their sentinel string to config.yaml:

algorithm:
  reward:
    cronSentinels: ["[IMPORTANT: You are running as a scheduled cron job"]

Design notes

  • Only gate 1 (exchange count) is bypassed — content and triviality gates still apply.
  • Sentinel match is prefix-based (message.startsWith(sentinel)).
  • The bypass is transparent in logs: skipped-by-cron episodes show a distinct log tag.

Test plan

  • Episode with 1 exchange and matching sentinel: passes exchange-count gate, other gates still apply
  • Episode with 1 exchange and no matching sentinel: still skipped by exchange-count gate (default behavior)
  • Empty cronSentinels array: no change to existing behavior
  • Trivial cron episode (matching sentinel but trivial content): still skipped by content gate

🤖 Generated with Claude Code

Cron jobs always produce exactly 1 user↔agent exchange — the task prompt
plus one reply — so minExchangesForCompletion: 2 zero-scores every cron
episode before content is even evaluated. This starves L2 induction of
signal after the bridge stabilises.

Adds `cronSentinels` to RewardConfig (schema, defaults, types). When
the first user turn starts with a sentinel prefix, check 1 (exchange
count) is skipped; content/triviality checks still apply. Default
sentinel covers the Hermes cron prompt.

The `snapshot.meta?.initialUserText` fallback handles episodes scored
during recovery when turns aren't materialised. If the field is absent
the episode falls back to the old skip behaviour — no false positives.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant