Relative include path of generated frozen module in getpath.c

[itamaro]

Feature or enhancement

Proposal:

in getpath.c, the generated frozen getpath.h is [included using relative path](

cpython/Modules/getpath.c

Line 25 in c6e614f

#include "../Python/frozen_modules/getpath.h"

I'd like to consider changing this to use absolute include path.
my primary motivation is to make it possible to build python using alternative build systems (buck2 in my case), where frozen modules are generated as intermediate build artifacts, and it's not straight forward (read: I couldn't figure it out yet) to make the generated header "look like" it's in that relative path.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-113039: Don't use relative include path in getpath.c #113022

[itamaro]

I verified that using absolute path works for in-tree and out-of-tree builds on MacOS and Linux.

CI on gh-113022 indicates that Windows doesn't like it, but I don't have access to a Windows machine to debug. @zooba do you think this would be an easy fix on Windows?

[zooba]

I'd rather refer to it as just #include "getpath.h" and provide an extra include directory to that file when it's being built (see PCbuild/pythoncore.vcxproj which has a few file-specific PreprocessorDefinitions elements - you can add a file-specific AdditionalIncludeDirectories in this case).

I'm not sure where the relative path came from, but I don't like it. I'm also not a huge fan of generated files going back into the source tree (I prefer them in $(IntDir) so that read-only source trees are supported), but that seems a big and unnecessary change right now.

[erlend-aasland]

[...] I'm also not a huge fan of generated files going back into the source tree (I prefer them in $(IntDir) so that read-only source trees are supported), but that seems a big and unnecessary change right now.

+1 to all you said, but especially this part. We should create an issue for this (unless there already is one).

[itamaro]

I'm also not a huge fan of generated files going back into the source tree

completely agree about the source tree. on Linux it seems this is already the case (in OOT builds, the frozen headers go in the build tree).
does Windows support OOT builds? how would I set this up if I wanted to test?

I'd rather refer to it as just #include "getpath.h" and provide an extra include directory to that file when it's being built

I think I'd prefer seeing the full path there -- seeing just getpath.h would be surprising ("where am I supposed to see this file on disk?") and probably require a comment pointing at the Python/frozen_modules directory - so why not have that bit in the include?

in any case, I was able to cheat and get CI on gh-113022 passing by keeping the relative include only on Windows.
I can try messing with AdditionalIncludeDirectories and see if I can make it work - might take a while as I am quite clueless about Windows :)

[itamaro]

figured out AdditionalIncludeDirectories and published the PR

[zooba]

does Windows support OOT builds? how would I set this up if I wanted to test?

It's already the default - build files go into PCbuild/obj and outputs into PCbuild/amd64 (or win32/arm64). You can set Py_IntDir and/or Py_OutDir environment variables before building in order to move them.

I think I'd prefer seeing the full path there -- seeing just getpath.h would be surprising ("where am I supposed to see this file on disk?") and probably require a comment pointing at the Python/frozen_modules directory - so why not have that bit in the include?

Because it's still going to be relative to an implied include dir, it's just going to make it harder to guess which one because the file is so much further away from any of the directories.

Adding the comment is fine. You can give a better explanation that way.

[zooba]

The PR was fine though, so I merged it. If you want to do more work to avoid writing files back into the source tree at build time, feel free, but I didn't see any reason to hold up that first change.

[itamaro]

Thanks @zooba !

I'll close this issue now, as the relative include issue has been resolved. I opened gh-113258 to follow-up on moving the frozen modules out of the source tree.