pyright gives types found in `site-packages` higher priority than types found in typeshed

[Avasam]

See this comment for context and example: #10058 (comment)

I think this only affects pyyaml and setuptools at the moment, as they are they only type-* packages installed during tests.

Can probably be fixed by specifying the --typeshed-path cli argument. Note that this would have to be done in pyright_test.py and all Github actions. I have not tested this potential solution myself yet, and I didn't feel like doing this right now, hence this issue to remember.

[AlexWaygood]

Interesting, would specifying the --typeshed-path argument potentially also unblock #9989 (comment)?

[Avasam]

I just remembered that the setting typeshedPath is set in the configurations.
So the question is: Should that setting imply that the stubs found in the typeshed path have priority over everything else (not just the bundled types)? (in which case, open an issue at pyright). Or should typeshed change how it installs dependencies for tests?

[AlexWaygood]

So the question is: Should that setting imply that the stubs found in the typeshed path have priority over everything else (not just the bundled types)?

There might be something in PEP 561 about this, but I don't know what the answer is off the top of my head. But it would certainly be convenient for us if the answer was "yes" :)

(This also came up in #10090 (comment))

[AlexWaygood]

There might be something in PEP 561 about this

Typeshed is listed 5th in PEP 561's "Type checker module resolution order" section here. So the answer PEP 561 gives is that pyright is probably correct here in giving type hints found in site-packages/ priority over type hints found in typeshed.

(And that makes sense from the perspective of end users. It would be annoying if you'd installed an awesome-package-stubs package, but found that type checkers weren't using those stubs when type-checking your code, because they were prioritising the stubs they found for awesome-package that were in the type checker's bundled copy of typeshed, which you can't control at all.)

[Avasam]

Can we consider this issue closed by #10121 ? The linked issue comments are a different problem that can be resolved with a different solution or just by waiting.

[AlexWaygood]

Okay sure. Though I suspect the other problem will keep coming up, and it has a similar cause (insufficient isolation of the testing environment when we run pyright in CI).