gh-153333: Read tkinter profile scripts with the source file's encoding#153334
Conversation
…encoding Tk.readprofile execed the user's ~/.CLASSNAME.py and ~/.BASENAME.py profile scripts read via open() with no encoding argument, i.e. the locale default encoding. That emits an EncodingWarning under -X warn_default_encoding and mis-decodes a profile whose source carries a PEP 263 coding cookie. Read them with tokenize.open() instead, which decodes using the encoding detected from the file (its coding cookie, else UTF-8), matching how Python reads source. As an incidental effect of using the context-manager form, each script is now closed promptly after reading instead of at garbage collection, removing a ResourceWarning.
serhiy-storchaka
left a comment
serhiy-storchaka
left a comment
There was a problem hiding this comment.
Simply open them in binary mode.
Pass the raw bytes to exec() so the compiler applies the source file's encoding declaration, instead of detecting it with tokenize.open().
serhiy-storchaka
left a comment
serhiy-storchaka
left a comment
There was a problem hiding this comment.
I left some comments to the test.
If you use AI to generate this PR, please tell it to simplify and trim the PR message and the commit message. Several times if needed. Nobody want to read such large boring text. Comments also can be shortened.
serhiy-storchaka
added
the
needs backport to 3.15
|
I consider this a bug fix, but backporting it to released Python versions can break user code. So I only backport it to 3.15, which has not yet been released. |
|
Thanks @tonghuaroot for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15. |
|
Sorry, @tonghuaroot and @serhiy-storchaka, I could not cleanly backport this to |
|
GH-153541 is a backport of this pull request to the 3.15 branch. |
bedevere-app
Bot
removed
the
needs backport to 3.15
… encoding (GH-153334) (GH-153541) Tk.readprofile ran the user's ~/.CLASS.py and ~/.BASE.py scripts with exec(open(path).read()), decoding them with the locale encoding. Read them in binary mode so exec() honors each script's own coding cookie. (cherry picked from commit bac73b0) Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
2 participants
Tk.readprofileread the~/.CLASS.py/~/.BASE.pyscripts with a plainopen().read(), so they were decoded with the locale encoding and left open until GC. Read them in binary mode soexecuses each file's own coding cookie.