Repository navigation
Release 3.4.2: direct mode keeps its engine connection alive - #34
Merged
Merged
Conversation
DirectEngine opened a new connection (urllib.request.urlopen) for every protocol call. It now keeps one http.client connection per thread and reuses it (HTTPS when the engine URL says so, verified as before). - A connection the engine closed while idle is detected before reuse and replaced. - If a reused connection fails before the engine answers, a read (describe, query other than a statement) is sent once more on a new connection; a write is not and raises RushDBError, as before. - Read timeouts still raise TimeoutError; DirectEngine takes `timeout` (default 120 s, unchanged). - Connections of ended threads are closed; close() closes all; a forked child does not touch the parent's sockets. - When HTTP_PROXY/HTTPS_PROXY applies to the engine URL, calls go through urllib and the proxy, as before. - No new dependency. count(*) p50 through db.engine.execute() against a local rushdb 3.0.12 container (20K records, 3 x 3000 calls per run, ABBA): via the published port: urlopen 0.155-0.164 ms, keep-alive 0.077-0.079 ms via the container IP: urlopen 0.121-0.123 ms, keep-alive 0.066 ms raw http.client keep-alive floor: 0.070 / 0.061 ms
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Direct mode now keeps its connection to the engine alive. Before,
DirectEngineopened a new connection (urllib.request.urlopen) for every protocol call.Change: each thread keeps one
http.clientconnection to the engine and reuses it (HTTPS when the engine URL says so, certificate verified as before). No new dependency.describe, or aqueryother than a Cypher/SQL statement) is sent once more on a new connection. A write is not sent again; it raisesRushDBError, as before.TimeoutError.DirectEnginetakestimeout(default 120 s, unchanged) and hasclose().HTTP_PROXY/HTTPS_PROXYapplies to the engine URL, calls still go throughurlliband the proxy, without keep-alive.Measured:
count(*)p50 throughdb.engine.execute()against a local rushdb 3.0.12 container (20K records, 3 × 3000 calls per run, runs alternated old/new/old/new):http.clientkeep-aliveAbout 0.06–0.08 ms saved per call; the SDK now adds ~0.006 ms over a bare
http.clientconnection.Tests: new
tests/test_direct_engine_transport.py(18 tests against a real local HTTP and HTTPS server: reuse, idle close,Connection: close, read resent at most once, write never resent, timeout, threads,close(), proxy fallback, TLS verified and reused, untrusted certificate refused). Existing direct-mode tests stubDirectEngine._sendinstead ofurlopen. A live test checks that calls share one connection. Lint and mypy clean; 244 tests pass on Python 3.10, 3.12 and 3.13 withResourceWarningas an error; the live suite passes against rushdb 3.0.12.Patch release (3.4.2), no API change beyond the additive
timeoutargument andclose().