feat(reward): sentinel-based exchange-count bypass for cron episodes#1848
Open
chiefmojo wants to merge 1 commit into
Open
feat(reward): sentinel-based exchange-count bypass for cron episodes#1848chiefmojo wants to merge 1 commit into
chiefmojo wants to merge 1 commit into
Conversation
Cron jobs always produce exactly 1 user↔agent exchange — the task prompt plus one reply — so minExchangesForCompletion: 2 zero-scores every cron episode before content is even evaluated. This starves L2 induction of signal after the bridge stabilises. Adds `cronSentinels` to RewardConfig (schema, defaults, types). When the first user turn starts with a sentinel prefix, check 1 (exchange count) is skipped; content/triviality checks still apply. Default sentinel covers the Hermes cron prompt. The `snapshot.meta?.initialUserText` fallback handles episodes scored during recovery when turns aren't materialised. If the field is absent the episode falls back to the old skip behaviour — no false positives. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a config-driven bypass so cron-initiated episodes can be scored even when they fall below the
minExchangesForCompletionthreshold.Problem
Scheduled agent jobs produce single-exchange episodes (the cron prompt + agent response). With the default
minExchangesForCompletion: 1floor these score fine, but tighter settings — or multi-turn flows that cron fires as one composite turn — hit the "too few exchanges" skip condition. The cron session produces real work worth scoring but the exchange-count gate silently discards it.Solution
RewardConfig.cronSentinels: string[]— an array of substrings. If any user message in the episode starts with one of these strings, the exchange-count check (gate 1) is bypassed. All other quality gates (content length, triviality, tool-heavy ratio) remain active.The default is an empty array (
[]), so behavior is unchanged for existing deployments. Operators who run scheduled agent jobs add their sentinel string toconfig.yaml:Design notes
message.startsWith(sentinel)).Test plan
cronSentinelsarray: no change to existing behavior🤖 Generated with Claude Code