The handle_messages function in core/connectors/sdk/src/source.rs discards the actual error returned by source.poll():
let Ok(messages) = messages else {
error!("Failed to poll messages for source connector with ID: {plugin_id}");
continue;
};The underlying error is never logged. Any connector failure produces an infinite loop of "Failed to poll messages" with zero context about the root cause. During the Postgres-to-Iceberg benchmark, this cost significant debugging time because the real error (InvalidRecord from a NUMERIC column type mismatch) was invisible until a temporary log line was added inside the connector's own poll() method.
Fix: Log the error: error!("Failed to poll messages for source connector with ID: {plugin_id}: {err:?}");
The same pattern should be checked in the sink SDK path.
The
handle_messagesfunction incore/connectors/sdk/src/source.rsdiscards the actual error returned bysource.poll():The underlying error is never logged. Any connector failure produces an infinite loop of "Failed to poll messages" with zero context about the root cause. During the Postgres-to-Iceberg benchmark, this cost significant debugging time because the real error (
InvalidRecordfrom a NUMERIC column type mismatch) was invisible until a temporary log line was added inside the connector's ownpoll()method.Fix: Log the error:
error!("Failed to poll messages for source connector with ID: {plugin_id}: {err:?}");The same pattern should be checked in the sink SDK path.