fix: handle null base64_image in MCP screenshot responses

[dzorlu]

Summary

• Add null check for parsed["base64_image"] in _extract_tool_result
• When Fleet MCP returns {"base64_image": null} (failed screenshot capture), return a text error message instead of creating an image_url block with url: None
• Prevents downstream AttributeError in SkyRL's extract_images_from_conversation

Root cause

_extract_tool_result unconditionally wraps parsed["base64_image"] in an OpenAI image_url block. When the value is None, this creates {"url": None} which crashes .startswith() in the consumer.

Test plan

• Verify VL training handles failed screenshots gracefully (text error instead of crash)
• Confirm working screenshots still produce valid image_url blocks

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR introduces the fleet_env integration module for OpenEnv — FleetEnvClient, FleetMCPTools, FleetTaskEnv, and supporting utilities — and applies a targeted bug fix: _extract_tool_result now null-checks parsed["base64_image"] before constructing an OpenAI image_url block, preventing a downstream AttributeError in SkyRL's extract_images_from_conversation when Fleet MCP returns {"base64_image": null} for a failed screenshot.

Key findings:

• The null-guard fix itself is correct (lines 149–154 of fleet_mcp_client.py). The new if data_url is not None branch returns a descriptive string instead of {"url": None}, cleanly breaking the crash path.
• Three tests will fail at runtime (test_fleet_env_from_fleet_returns_orchestrator_and_tools, test_fleet_env_reset_uses_http_manager_base_url, test_fleet_env_step_rejects_tool_actions in test_fleet_env.py). All three call FleetEnvClient.from_fleet() without the three required positional arguments data_key, data_version, and image_type. Python raises TypeError before the test body executes. The TestFleetEnvClientDbQuery block in the same file shows the correct call pattern.
• No regression test for the fixed code path. The TestFleetMCPClientExtractToolResult class covers eight scenarios but not the null base64_image case that motivated this PR.

Alignment notes:

• Client/server separation is maintained: FleetEnvClient.step() actively rejects ListToolsAction/CallToolAction, enforcing the MCP-only channel for agent actions.
• Rewards are computed entirely inside the environment (_compute_reward via the Fleet SDK verifier), consistent with the "rewards inside environment" principle.
• No RFC-breaking changes detected; this is additive integration code.

Confidence Score: 4/5

Safe to merge after fixing the three broken test call-sites; the core logic is sound.

One P1 issue remains: three tests in test_fleet_env.py call from_fleet() without required arguments and will raise TypeError, meaning those tests never validate their intended behavior. The core bug fix in fleet_mcp_client.py is correct and all other code is well-structured. Score is 4 rather than 5 because the failing tests should be fixed before merge.

tests/envs/test_fleet_env.py lines 79–109 — three tests use an incomplete from_fleet() signature.

Important Files Changed

Filename	Overview
src/envs/fleet_env/fleet_mcp_client.py	Core bug fix: adds null guard for base64_image in _extract_tool_result, correctly returning a text error string instead of {"url": None} that would crash downstream .startswith() calls.
tests/envs/test_fleet_env.py	Three tests call FleetEnvClient.from_fleet() without required positional args data_key, data_version, image_type — these will fail with TypeError. Also missing a regression test for the null base64_image fix.
src/envs/fleet_env/mcp_tools.py	New file: FleetMCPTools agent-facing wrapper with retry logic for list_tools and call_tool over multiple MCP endpoints with deduplication.
src/envs/fleet_env/client.py	New file: FleetEnvClient HTTP orchestrator with sync/async provisioning, DB query helpers, and clean separation that rejects MCP tool actions via step().
src/envs/fleet_env/task_env.py	New file: FleetTaskEnv Gymnasium-compatible task wrapper with async provisioning, modality-based tool filtering, verifier execution, and rollout telemetry.
tests/envs/test_fleet_task_env.py	New test file for FleetTaskEnv; tests use proper mocking and cover init, specs, and rollout lifecycle.

Sequence Diagram

sequenceDiagram
    participant Agent
    participant FleetMCPTools
    participant FleetMCPClient
    participant FleetMCP as Fleet MCP Server

    Agent->>FleetMCPTools: call_tool("computer", {action: "screenshot"})
    FleetMCPTools->>FleetMCPClient: call_tool("computer", args)
    FleetMCPClient->>FleetMCP: session.call_tool(name, arguments)
    FleetMCP-->>FleetMCPClient: CallToolResult(content=[TextContent(text='{"base64_image": null}')])
    FleetMCPClient->>FleetMCPClient: _extract_tool_result(result)
    Note over FleetMCPClient: json.loads → parsed["base64_image"] is None<br/>NEW: returns "Screenshot capture failed (null image)"<br/>OLD: returned [{"type":"image_url","image_url":{"url":None}}]
    FleetMCPClient-->>FleetMCPTools: "Screenshot capture failed (null image)"
    FleetMCPTools-->>Agent: str (safe — no crash downstream)

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: tests/envs/test_fleet_env.py
Line: 79-85

Comment:
**`from_fleet` called without required positional arguments**

`FleetEnvClient.from_fleet()` requires three additional positional parameters — `data_key: str`, `data_version: str`, and `image_type: str` — beyond `api_key` and `env_key`. None of these three carry default values. Calling the method while omitting them will raise `TypeError` before any assertion runs, failing the test.

The same problem exists on lines 88–98 and 101–109.

The fix is to pass placeholder values for all three required parameters. The `TestFleetEnvClientDbQuery` block (lines ~488–494) already shows the correct calling convention with `data_key`, `data_version`, and `image_type` filled in.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: tests/envs/test_fleet_env.py
Line: 343-391

Comment:
**No regression test for the null `base64_image` fix**

The PR's stated purpose is to prevent the crash when `{"base64_image": null}` is returned by Fleet MCP, but the test class only exercises the happy path (a valid data URL in `test_extract_base64_image_json_format`). There is no test asserting that `_extract_tool_result` returns a plain string error when `base64_image` is `null`.

A companion test alongside the existing JSON-format tests would guard against this exact regression. The test should construct a `TextContent` whose `.text` is `'{"base64_image": null}'`, call `_extract_tool_result`, and assert the result is a `str` (not a `list` or `dict` with `url: None`).

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "fix: handle null base64_image from Fleet..." | Re-trigger Greptile}

[greptile-apps]

from_fleet called without required positional arguments

FleetEnvClient.from_fleet() requires three additional positional parameters — data_key: str, data_version: str, and image_type: str — beyond api_key and env_key. None of these three carry default values. Calling the method while omitting them will raise TypeError before any assertion runs, failing the test.

The same problem exists on lines 88–98 and 101–109.

The fix is to pass placeholder values for all three required parameters. The TestFleetEnvClientDbQuery block (lines ~488–494) already shows the correct calling convention with data_key, data_version, and image_type filled in.

Prompt To Fix With AI

This is a comment left during a code review.
Path: tests/envs/test_fleet_env.py
Line: 79-85

Comment:
**`from_fleet` called without required positional arguments**

`FleetEnvClient.from_fleet()` requires three additional positional parameters — `data_key: str`, `data_version: str`, and `image_type: str` — beyond `api_key` and `env_key`. None of these three carry default values. Calling the method while omitting them will raise `TypeError` before any assertion runs, failing the test.

The same problem exists on lines 88–98 and 101–109.

The fix is to pass placeholder values for all three required parameters. The `TestFleetEnvClientDbQuery` block (lines ~488–494) already shows the correct calling convention with `data_key`, `data_version`, and `image_type` filled in.

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

No regression test for the null base64_image fix

The PR's stated purpose is to prevent the crash when {"base64_image": null} is returned by Fleet MCP, but the test class only exercises the happy path (a valid data URL in test_extract_base64_image_json_format). There is no test asserting that _extract_tool_result returns a plain string error when base64_image is null.

A companion test alongside the existing JSON-format tests would guard against this exact regression. The test should construct a TextContent whose .text is '{"base64_image": null}', call _extract_tool_result, and assert the result is a str (not a list or dict with url: None).

Prompt To Fix With AI

This is a comment left during a code review.
Path: tests/envs/test_fleet_env.py
Line: 343-391

Comment:
**No regression test for the null `base64_image` fix**

The PR's stated purpose is to prevent the crash when `{"base64_image": null}` is returned by Fleet MCP, but the test class only exercises the happy path (a valid data URL in `test_extract_base64_image_json_format`). There is no test asserting that `_extract_tool_result` returns a plain string error when `base64_image` is `null`.

A companion test alongside the existing JSON-format tests would guard against this exact regression. The test should construct a `TextContent` whose `.text` is `'{"base64_image": null}'`, call `_extract_tool_result`, and assert the result is a `str` (not a `list` or `dict` with `url: None`).

How can I resolve this? If you propose a fix, please make it concise.