test_unittest failure triggered by test_import + test_importlib

[Yhg1s]

Bug report

Bug description:

test_unittest fails if (and only if) test_import and test_importlib are run (in the same process) before it:

./python -m test test_import test_importlib test_unittest
0:00:00 load avg: 3.83 Run tests sequentially
0:00:00 load avg: 3.83 [1/3] test_import
0:00:02 load avg: 3.52 [2/3] test_importlib
0:00:06 load avg: 3.24 [3/3] test_unittest
test test_unittest failed -- Traceback (most recent call last):
  File "/home/thomas/python/python/cpython/Lib/test/test_unittest/test_discovery.py", line 840, in test_discovery_failed_discovery
    with test.test_importlib.util.uncache('package'):
         ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'test.test_importlib' has no attribute 'util'

test_unittest failed (1 error)

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-108927: Fix test_import + test_importlib + test_unittest problem #108929
• gh-108927: Fix removing testing modules from sys.modules #108952
• [3.12] gh-108927: Fix test_import + test_importlib + test_unittest problem (GH-108929) #108954
• [3.11] gh-108927: Fix test_import + test_importlib + test_unittest problem (GH-108929) #110347
• [3.12] gh-108927: Fix removing testing modules from sys.modules (GH-108952) #112711
• [3.11] gh-108927: Fix removing testing modules from sys.modules (GH-108952) #112712
• gh-108927: Include new dir test/regrtestdata in the installation #112765
• [3.11] gh-108927: Include new dir test/regrtestdata in the installati… #112784
• [3.12] gh-108927: Include new dir test/regrtestdata in the installati… #112785

[serhiy-storchaka]

1. test_import imports test.test_importlib.util. 'test.test_importlib' and 'test.test_importlib.util' are now in sys.modules.
2. Somebody removes 'test.test_importlib' from sys.modules but leaves 'test.test_importlib.util'.
3. libregrtest imports test.test_importlib. test now has attribute test_importlib.
4. test_unittest.test_discovery requests to import test.test_importlib.util at import time, and since it is found in sys.modules, it simply sets module global test.
5. It now tries to access test.test_importlib.util. Global test has attribute test_importlib, but test.test_importlib has no attribute util.

[serhiy-storchaka]

2. Somebody removes 'test.test_importlib' from sys.modules but leaves 'test.test_importlib.util'.

libregrtest does this for every tested module. Now this sequence is clear:

1. test_import imports test.test_importlib.util.
2. libregrtest messes with import machinery by removing and re-importing test_importlib.
3. test_unittest imports and use test.test_importlib.util.

You can use other tests which import test.test_importlib.util, just add test_importlib between two of such tests.

[serhiy-storchaka]

test_compileall yet one test which can fail after test_importlib.

Details

$ ./python -m test -v test_import test_importlib test_compileall
...
---------------------------------------------------------
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_py_compile.py", line 22, in wrapper
    return fxn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/test/test_compileall.py", line 331, in test_ddir_multiple_workers
    return self._test_ddir_only(ddir="<a prefix>", parallel=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/test/test_compileall.py", line 320, in _test_ddir_only
    mod_code_obj = test.test_importlib.util.get_code_from_pyc(modcode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'test.test_importlib' has no attribute 'util'

======================================================================
ERROR: test_ddir_only_one_worker (test.test_compileall.CompileallTestsWithoutSourceEpoch.test_ddir_only_one_worker)
Recursive compile_dir ddir= contains package paths; bpo39769.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_py_compile.py", line 22, in wrapper
    return fxn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/test/test_compileall.py", line 326, in test_ddir_only_one_worker
    return self._test_ddir_only(ddir="<a prefix>", parallel=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/test/test_compileall.py", line 320, in _test_ddir_only
    mod_code_obj = test.test_importlib.util.get_code_from_pyc(modcode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'test.test_importlib' has no attribute 'util'

----------------------------------------------------------------------

#108952 fixes all such cases.

[lordmauve]

I am also seeing this in the 3.11.6 release and can confirm that #108929 fixes it.

[lordmauve]

Actually when I run just ./python -m test test_import test_importlib test_unittest I see this in 3.11.5 and 3.11.4 as well. But 3.11.4's test suite is succeeding on our CI. 3.11.6 is not.