feat: python asyncio connection pooling#1689
Conversation
Greptile SummaryAdds unified asynchronous Python SDK methods and persistent HTTP connection pools.
Confidence Score: 2/5The PR is not yet safe to merge because synchronous cleanup can still finalize a loop-bound async transport on a fresh event loop. The original-loop path only applies when that loop remains active in another thread; cleanup on the owning thread or after loop termination still uses a newly created event loop after discarding the retained client reference, so the previously reported transport-cleanup failure remains. Files Needing Attention: templates/python/package/client.py.twig Important Files Changed
Reviews (16): Last reviewed commit: "fix(python): close async client on its o..." | Re-trigger Greptile |
ede92a8 to
59e98e2
Compare
…cture Add async method generation (*_async) alongside synchronous methods, introduce api_async and file_async request dispatch templates, and handle TypeEnum import aliasing for parameter name collisions.
…llel chunked upload - Implement persistent httpx.Client and httpx.AsyncClient connection pooling - Add close(), aclose(), and context managers (__enter__/__exit__, __aenter__/__aexit__) - Track pending async cleanup tasks in Client._cleanup_tasks set and await them in aclose() - Schedule async client cleanup safely during close() and set_self_signed() - Configure 30s client timeouts (httpx.Timeout(30.0, connect=10.0)) - Implement parallel chunked_upload_async using asyncio.Semaphore(8) + asyncio.gather with non-blocking asyncio.to_thread(_read) file I/O matching sync ThreadPoolExecutor(8)
Replace requests with httpx>=0.27 across package manifests (pyproject.toml, setup.py, requirements.txt) and switch mock engine to respx.
…urrency unit tests - Create test_client.py template testing context managers, close/aclose, mixed-pool cleanup, tracked cleanup tasks under active loop, and Semaphore(8) concurrency cap - Update test_service.py template for respx and add dual-mode *_async test methods with location guards - Register test_client.py in Python.php file list and add TypeEnum collision guards
59e98e2 to
472dadb
Compare
|
@ChiragAgg5k can you review this when you have bandwidth |
|
Thanks for the review @greptile-apps This design is intentional due to Python's asyncio event loop constraints: In asynchronous contexts, developers should use async with Client() or await client.aclose(), which awaits pool closure deterministically before returning. |
|
That explanation is technically sound and aligns well with Python's asyncio constraints. The three-path approach is a reasonable best-effort design:
One thing worth documenting in the generated code (in the docstring or a comment near # If called from a running event loop, async pool closure is scheduled
# as a background task and is not guaranteed to complete before this
# method returns. Use `await aclose()` or `async with Client()` for
# deterministic cleanup in async contexts.Happy to withdraw the original concern given this context. |
…get sync close inside event loop
…ync close in event loop
|
@greptile-apps review |
|
@greptile-apps The reference handling in close() and set_self_signed() is intentionally structured to preserve old_async_client until cleanup completes: Setting self._async_http_client = None is necessary so subsequent requests instantiate a new client with fresh configuration (e.g. updated SSL/TLS settings). |
|
You're right, and I was wrong on both counts. Looking at the actual code again:
And for the no-loop case in The |
|
@greptile-apps review |
… guarantee immediate synchronous cleanup
… cleanup_tasks tracking
…hort event loop termination
…thread cleanup The previous _schedule_async_client_cleanup used loop.create_task() + asyncio.shield, which had two failure modes: 1. Short-lived loop shutdown: asyncio.run() cancels all pending tasks on exit, including our shielded cleanup task, leaving the HTTP transport and pooled sockets open. 2. Wrong-loop fallback: the no-loop path called asyncio.run(aclose()), creating a brand-new event loop that is different from the one the AsyncClient transport was used on. Fix: replace with _close_async_client_sync() which: - Spawns a daemon thread with its own fresh event loop - Calls loop.run_until_complete(client.aclose()) in that thread - Joins with t.join(timeout=5), making close() and set_self_signed() fully deterministic - the async pool is guaranteed closed on return Also removes _cleanup_tasks set (no longer needed) and updates tests to assert async_client.is_closed immediately after close() returns, rather than asserting internal task-set bookkeeping.
… closed t.join(timeout=5) could return while the daemon thread was still running aclose(), leaving pooled sockets open with no way for the caller to retry or await the remaining cleanup. Removing the timeout makes close() and set_self_signed() block until aclose() has fully completed on the dedicated thread, giving an unconditional guarantee: the function returns if and only if the async connection pool is closed. httpx.AsyncClient uses anyio internally and is not bound to its creation event loop, so running aclose() on a fresh loop in a dedicated thread is safe and well-defined.
…routine_threadsafe Previous approach always used a fresh event loop in a worker thread, which Greptile flagged as a cross-event-loop defect: connections established on loop A were being closed on loop B. Fix: - Track _async_client_loop: the event loop the AsyncClient was created on. - In _close_async_client_sync, when that loop is still running in a different thread, use asyncio.run_coroutine_threadsafe to schedule aclose() on the original loop and block with future.result() — no cross-loop close, fully deterministic. - Detect whether close() is being called from within the loop's own thread (asyncio.get_running_loop() is loop) to avoid deadlock; fall back to the dedicated thread+fresh-loop path in that case. - The thread-based fallback (for pure-sync callers or after asyncio.run() exits) remains and t.join() has no timeout. Tests: - Add test_sync_close_after_active_pool: makes real mocked requests via respx before calling close(), covering the used-pool code path. - Add test_sync_close_from_async_context_after_requests: same but close() is called synchronously from within a running event loop.
What does this PR do?
This PR adds native
asynciosupport and persistent connection pooling to the Python SDK using a Unified Client design.Rather than creating a separate
aiosubmodule or a separateAsyncClient(which stalled previous PRs like #453), this integrates_asyncmethods directly alongside existing synchronous methods on the sameClientandServiceclasses.What changed:
account.get()andawait account.get_async()). All synchronous methods are 100% untouched and backward compatible.httpx.Clientandhttpx.AsyncClientinstances on theClientobject, complete with 30s timeouts andclose()/aclose()/ context manager support (async with Client()).chunked_upload_asyncusesasyncio.Semaphore(8)+asyncio.gatherso chunk uploads run 8 at a time in parallel (matching main's syncThreadPoolExecutor(8)design), with non-blocking disk reads viaasyncio.to_thread.requeststohttpx(andrespxfor mock testing).Test Plan
pytest.test_client.pycovering:with Client()andasync with Client()context managersclose()andaclose()callsSemaphore(8)concurrency cap validation on async uploads(Note: Unit tests cover all
_asyncmethods viarespx. The existing E2E scripttests.pyruns sync methods against the mock server container; let me know if you want_asynccalls added to E2E mock testing as well.)Related PRs and Issues
Have you read the Contributing Guidelines on issues?
Yes