fix: thread conn= into failover timeout paths; invalidate pooled connections in tracker#1255
Merged
karenc-bq merged 1 commit intoJul 2, 2026
Conversation
…ections in tracker Follow-up to the sync foundation PR aws#1252 (phase 1 of aws#1251): sync-side hardening of the failover paths, intended to land before the async phase (phase 2) is introduced. Pass conn= to driver_dialect.execute() in the failover plugins' rollback/close paths so a timed-out operation can abort its socket and drain its worker. PR aws#1252 introduced this abort-and-drain mechanism and applied it to the host-monitoring and limitless paths; this completes the failover paths. In the connection tracker, prefer PoolProxiedConnection.invalidate() over close() for pool-proxied connections (wrapper-internal connection pools): close() checks the connection back into the pool, which runs an unbounded rollback-on-return against the failed writer and re-pools the connection if that rollback succeeds. invalidate() skips the rollback and discards the connection so the pool opens a fresh one on next checkout.
44baad8 to
2a1f20d
Compare
Open
2 tasks
karenc-bq
approved these changes
Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #1252 (sync foundation for asyncio support, phase 1 of #1251). This is sync-side hardening of the failover paths, intended to land before the async phase (phase 2) is introduced.
Two changes:
Thread
conn=into the failover plugins' timeout paths. feat: sync foundation for asyncio support (phase 1 of #1251) #1252 introduced the abort-and-drain mechanism for timed-out offloaded operations (on timeout: shut down the operation's socket viadriver_dialect.abort_connectionand wait for its worker thread, so a later close/reuse of the connection cannot race the still-running operation) and applied it to the host-monitoring and limitless paths. This PR passesconn=todriver_dialect.execute()in the failover plugins' rollback/close call sites (failover_plugin.py,failover_v2_plugin.py), completing the mechanism for the failover paths.Prefer
PoolProxiedConnection.invalidate()overclose()in the connection tracker. When wrapper-internal connection pools (SqlAlchemyPooledConnectionProvider) are enabled, the connections tracked byOpenedConnectionTrackerare SQLAlchemy pool proxies. Callingclose()on a proxy checks it back into the pool, which first runs rollback-on-return against the failed writer — an unbounded blocking call on the tracker thread when the host is unreachable — and re-pools the connection if that rollback happens to succeed (e.g. the old writer came back as a reader).invalidate()(public SQLAlchemy API) skips the rollback and discards the connection, so the pool opens a fresh connection on the next checkout. Plain driver connections keep the existingclose()behavior.Unit tests cover both the invalidate-vs-close dispatch and the end-to-end pool behavior against a real
QueuePool(connection discarded without rollback-on-return, fresh connection on next checkout).Proposed CHANGELOG entry (under
### :bug: Fixed):By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.