Skip to content

gh-141930: Use the regular IO stack to write .pyc files for a better error message on failure#141931

Merged
gpshead merged 9 commits into
python:mainfrom
stefanor:retry-pyc-write
Nov 27, 2025
Merged

gh-141930: Use the regular IO stack to write .pyc files for a better error message on failure#141931
gpshead merged 9 commits into
python:mainfrom
stefanor:retry-pyc-write

Conversation

@stefanor

@stefanor stefanor commented Nov 25, 2025

Copy link
Copy Markdown
Contributor

Writes are not guaranteed to complete by the operating system. Use the normal file object to do the necessary retries and raise appropriate errors, if we fail to completely write a .pyc file in one syscall.

Comment thread Lib/importlib/_bootstrap_external.py Outdated
Comment thread Lib/importlib/_bootstrap_external.py Outdated
Comment thread Lib/importlib/_bootstrap_external.py Outdated
@python-cla-bot

python-cla-bot Bot commented Nov 25, 2025

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

Interruption by a system call or something is not completely unexpected.
Retry the write with the remaining data.
@stefanor

Copy link
Copy Markdown
Contributor Author

Squashed in that suggestion to avoid the CLA bot.

Comment thread Lib/importlib/_bootstrap_external.py Outdated
Comment thread Lib/test/test_importlib/test_util.py Outdated
Comment thread Lib/test/test_importlib/test_util.py Outdated
Comment thread Lib/test/test_importlib/test_util.py Outdated
Comment thread Misc/NEWS.d/next/Core_and_Builtins/2025-11-24-21-09-30.gh-issue-141930.hIIzSd.rst Outdated

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread Lib/test/test_importlib/test_util.py Outdated
@gpshead gpshead changed the title gh-141930: Retry writing .pyc files if incomplete gh-141930: Use the regular IO stack to write .pyc files for a better error message on failure Nov 27, 2025

@gpshead gpshead left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reworded the news and title because "retry" and "retried" imply the wrong thing. The important thing is that the write will be "complete" so that's the word I put in NEWS or we'll get the actual underlying error.

@gpshead gpshead added the needs backport to 3.13 bugs and security fixes label Nov 27, 2025
@gpshead gpshead enabled auto-merge (squash) November 27, 2025 18:58
@gpshead gpshead merged commit 656a64b into python:main Nov 27, 2025
52 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @stefanor for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 27, 2025
…etter error message on failure (pythonGH-141931)

* Use open() to write the bytecode
* Convert to unittest style asserts
* Tweak news, thanks @vstinner
* Tidy
* reword NEWS, avoid word "retried"
(cherry picked from commit 656a64b)

Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
@miss-islington-app

Copy link
Copy Markdown

Sorry, @stefanor and @gpshead, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 656a64b37f817cc8fe36ee17f332100482185cce 3.13

@bedevere-app

bedevere-app Bot commented Nov 27, 2025

Copy link
Copy Markdown

GH-142021 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 27, 2025
@bedevere-app

bedevere-app Bot commented Nov 27, 2025

Copy link
Copy Markdown

GH-142023 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Nov 27, 2025
@stefanor stefanor deleted the retry-pyc-write branch November 28, 2025 01:02
hugovk pushed a commit that referenced this pull request Dec 1, 2025
…better error message on failure (GH-141931) (#142021)

Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
Yhg1s pushed a commit that referenced this pull request Dec 1, 2025
…better error message on failure (GH-141931) (#142023)

* Use open() to write the bytecode
* Convert to unittest style asserts
* Tweak news, thanks @vstinner
* Tidy
* reword NEWS, avoid word "retried"
(cherry picked from commit 656a64b)

Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
…etter error message on failure (pythonGH-141931)

* Use open() to write the bytecode
* Convert to unittest style asserts
* Tweak news, thanks @vstinner
* Tidy
* reword NEWS, avoid word "retried"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants