Skip to content

gh-151593: Fix dead lock in PyDict insert_split_key()#152200

Merged
vstinner merged 2 commits into
python:mainfrom
vstinner:dict_deadlock
Jun 25, 2026
Merged

gh-151593: Fix dead lock in PyDict insert_split_key()#152200
vstinner merged 2 commits into
python:mainfrom
vstinner:dict_deadlock

Conversation

@vstinner

@vstinner vstinner commented Jun 25, 2026

Copy link
Copy Markdown
Member

Do not hold LOCK_KEYS() lock when calling PyType_Modified() to avoid a deadlock.

Do not hold LOCK_KEYS() lock when calling PyType_Modified() to avoid
a deadlock.

Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
@vstinner

Copy link
Copy Markdown
Member Author

On Python built with ./configure --with-pydebug --with-system-expat --with-thread-sanitizer --disable-gil, I can easily reproduce the issue with:

./python -m test test_abc --parallel-threads=10 -v -F

It only takes 1 or 2 iterations to trigger the deadlock.

With this change, I can no longer reproduce the deadlock: the test is running for 5 minutes (test_abc passed 35 times in a row).

@vstinner

Copy link
Copy Markdown
Member Author

@dpdani @colesbury @nascheme: Would you mind to review this fix?

I just converted @nascheme's comment into an actual PR.

@dpdani dpdani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add a comment, but otherwise LGTM 👍

Comment thread Objects/dictobject.c
@vstinner

Copy link
Copy Markdown
Member Author

@dpdani: I added a comment on LOCK_KEYS() based on previous comments on pull requests and issues. Does it look good to you?

@colesbury colesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this

@vstinner vstinner merged commit bef5706 into python:main Jun 25, 2026
55 checks passed
@vstinner vstinner deleted the dict_deadlock branch June 25, 2026 17:58
@vstinner

Copy link
Copy Markdown
Member Author

Merged. Thanks for reviews!

nascheme added a commit that referenced this pull request Jun 26, 2026
Revert commits:
    gh-151593: Fix dead lock in PyDict insert_split_key() (#152200)
    gh-150490: Raise PyType_Modified for insertion into split dictionary (#150489)

For gh-150489, it violates locking discipline and results in deadlocks,
gh-151593 is an example of it being hit in CI.  The attempted fix 
gh-152200 avoids the deadlock but introduces a data-race.  The race
window is small but can be triggered with pure Python code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants