Skip to content

Support async cache clients in FlagDefinitionCacheProvider #567

@nachogarcia

Description

@nachogarcia

Problem Statement

Context

We use posthog-python for server-side local evaluation behind a FastAPI app
running on Azure Container Apps. Multiple worker processes share a Redis
instance and we use a FlagDefinitionCacheProvider (modeled on
examples/redis_flag_cache.py) so only one worker polls PostHog at a time.

The rest of our codebase is async-first — every Redis call is redis.asyncio.
The FlagDefinitionCacheProvider protocol is sync, so to plug into it we had
to build a sync facade that owns a dedicated daemon thread and its own event
loop, then submit coroutines via run_coroutine_threadsafe for every call.
That's ~80 lines of plumbing solely to bridge redis.asyncio into a sync
contract whose only caller is itself a background thread.

Request

Allow async cache providers — i.e. let get_flag_definitions,
should_fetch_flag_definitions, on_flag_definitions_received, and shutdown
return awaitables that the SDK runs to completion before continuing.

Because the provider is only invoked from _load_feature_flags on the
Poller daemon thread (not from any synchronous evaluation path), the SDK
can transparently support this by running an event loop on that thread —
no API break, no opt-in flag needed.

Solution Brainstorm

Suggested shape

Option A — single protocol, awaitable-aware:

class FlagDefinitionCacheProvider(Protocol):
    def get_flag_definitions(self) -> (
        Optional[FlagDefinitionCacheData] | Awaitable[Optional[FlagDefinitionCacheData]]
    ): ...
    # ... etc

In _load_feature_flags, wrap each call:

result = provider.get_flag_definitions()
if inspect.isawaitable(result):
    result = _run_on_poller_loop(result)

with _run_on_poller_loop lazily creating a single asyncio event loop bound
to the polling thread (created once, reused across ticks).

Option B — separate AsyncFlagDefinitionCacheProvider protocol, accepted by
the same flag_definition_cache_provider= kwarg with isinstance dispatch.
Less elegant but explicit.

Option A keeps the public surface unchanged and lets users pass async
providers without thinking about it.

What we'd contribute

Happy to send a PR implementing Option A with:

Want to confirm the direction before opening the PR — happy to do Option B
instead if you'd rather keep the sync protocol strictly sync.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions