Skip to content

[codex] Use non-view registry for Python register#470

Draft
jprafael wants to merge 1 commit into
duckdb:mainfrom
jprafael:codex/register-registry-non-view-pr
Draft

[codex] Use non-view registry for Python register#470
jprafael wants to merge 1 commit into
duckdb:mainfrom
jprafael:codex/register-registry-non-view-pr

Conversation

@jprafael
Copy link
Copy Markdown

Summary

  • replace the Python register() implementation with a connection-local registry resolved through replacement scans instead of creating a temporary view
  • make unregister() erase the registered Python dependency immediately, including inside an open transaction
  • keep existing temp object conflict behavior and add a pandas regression test that verifies the DataFrame can be collected after unregistering inside a transaction

Root Cause

register() created a temporary view with an external dependency on the Python object. When unregister() ran inside a transaction, DROP VIEW did not release that dependency until the transaction completed, so large DataFrames stayed alive across the transaction.

Validation

  • SKBUILD_EDITABLE_SKIP=/home/ubuntu/duckdb-sqlite-worktrees/duckdb-python-v1.5.3/build/debug python -m pytest tests/fast/pandas/test_pandas_unregister.py tests/fast/arrow/test_unregister.py tests/fast/api/test_duckdb_connection.py::TestDuckDBConnection::test_register_relation tests/fast/api/test_duckdb_connection.py::TestDuckDBConnection::test_unregister_problematic_behavior tests/fast/pandas/test_same_name.py::TestMultipleColumnsSameName::test_multiple_columns_with_same_name -q
  • manual weakref check: unregister() inside a transaction releases the DataFrame immediately and rollback does not resurrect the registered name
  • full-size chunk sanity check: RSS growth still reproduces through the insert path without register(), while each registered DataFrame weakref is released after unregister()

@jprafael jprafael force-pushed the codex/register-registry-non-view-pr branch from efe1f8b to 32b37c1 Compare May 25, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant