[SYCL][Driver] Add option to specify the location of ocloc - #22912
[SYCL][Driver] Add option to specify the location of ocloc#22912mdtoguchi wants to merge 16 commits into
Conversation
Some new offload model tests within sycl-offload-jit.cpp will fail when the corresponding device libraries are not built. In those cases, the test needs to pick up the internal variants so the driver does not error due to not finding any.
The ocloc tool used for ahead of time compilation targeting Intel GPUs is acquired externally, so it is not guaranteed to be placed in the PATH. Add --ocloc-path=<dir>, allowing a user to specify the directory in which ocloc resides. The user provided location always wins over any ocloc that is otherwise visible via the program paths or the PATH environment variable. The option is honored for both the old and the new offloading model: - Old model: the driver's Intel GPU backend compile job (SYCL.cpp) and the ocloc help emitted for -fsycl-help=gen. - New model: forwarded to clang-linker-wrapper, and to clang-sycl-linker for the --sycl-link path. Both tools also accept --ocloc-path= directly for standalone use.
tahonermann
left a comment
There was a problem hiding this comment.
This looks ok to me. I'd like to see more comments explaining what happens if ocloc is not at the specified path or not found elsewhere.
It's unfortunate that the logic for finding the right path is repeated in so many places, but that is a pre-existing issue that probably isn't easy to fix.
The help text for each of the three cases of the new option differs for each, but not in a meaningful way as far as I can tell. I suggest consolidating to one phrasing.
| // Returns the full path of the ocloc tool to be used for AOT compilation. A | ||
| // user provided --ocloc-path= is honored above all other lookup locations. | ||
| const char *getOclocPath(Compilation &C, const ToolChain &TC, | ||
| const llvm::opt::ArgList &Args); |
There was a problem hiding this comment.
What happens when ocloc is not found? Is a null pointer returned? Is a diagnostic issued? It would be helpful for the comment to state what should happen.
There was a problem hiding this comment.
If not found, the fallback is just the tool name with no path information. I'll update the comment.
YuriPlyakhin
left a comment
There was a problem hiding this comment.
Does it make sense to submit it upstream first?
Probably not - the only tool where ocloc is even mentioned is the |
| } | ||
|
|
||
| /// Locate the 'ocloc' tool used for Intel GPU AOT compilation. | ||
| Expected<std::string> findOcloc(const ArgList &Args) { |
There was a problem hiding this comment.
Implementation in ClangLinkerWrapper.cpp and ClangSYCLLinker.cpp - near-identical, but differ in some subtle things:
- The two copies check
DryRunon opposite sides of thefindProgramByNamecall. In the wrapper, a--dry-runin an environment whereoclocdoes exist in the given directory hits the filesystem and returns the resolved path; in clang-sycl-linker it never does. - Diagnostic capitalization is different (
Unable to findvsunable to find).
Could you please align implementations?
YuriPlyakhin
left a comment
There was a problem hiding this comment.
Question: can we use -B instead of introducing new per-tool option?
| SmallString<128> ExecPath; | ||
| // A user provided --ocloc-path= overrides the usual tool lookup for ocloc. | ||
| if (Arg *A = C.getArgs().getLastArg(options::OPT_ocloc_path_EQ); | ||
| A && std::get<1>(HA) == "ocloc") { |
There was a problem hiding this comment.
nit
std::get<1>(HA) is now repeated three times in five lines - a bit difficult to read the code.
Could you please consider a small named struct ({Triple, ToolName, HelpFlag, ExtraArg}) instead of tuple (std::tuple<llvm::Triple, StringRef, StringRef, StringRef>)? Maybe you can do a small NFC PR with this refactoring as a prerequisite to this PR.
There was a problem hiding this comment.
I'll look into restructuring this for readability. Current changes at least clean up the std::get<1>(HA) usage.
| C.getDefaultToolChain().GetProgramPath(std::get<1>(HA).data())); | ||
| SmallString<128> ExecPath; | ||
| // A user provided --ocloc-path= overrides the usual tool lookup for ocloc. | ||
| if (Arg *A = C.getArgs().getLastArg(options::OPT_ocloc_path_EQ); |
There was a problem hiding this comment.
getLastArg(OPT_ocloc_path_EQ) is re-queried on every loop iteration for every tool, please, consider hoisting it above the loop or maybe change the order of conditions, to first check if the tool is ocloc.
There was a problem hiding this comment.
should no longer be an issue with the change to use a common ocloc location query.
| def cuda_path_EQ : Joined<["--"], "cuda-path=">, | ||
| Flags<[WrapperOnlyOption]>, MetaVarName<"<dir>">, | ||
| HelpText<"Set the system CUDA path">; | ||
| def ocloc_path_EQ : Joined<["--"], "ocloc-path=">, |
There was a problem hiding this comment.
Please, update documentation for both: clang-linker-wrapper and clang-sycl-linker to document new options (also driver maybe?)
ClangLinkerWrapper.rst
ClangSYCLLinker.rst
sycl/doc/UsersManual.md
Also Options WG???
There was a problem hiding this comment.
Documentation additions here: cb367d5
Options WG can be queried, but is currently more geared towards xmain and new options that require additional scrutiny. If this new option naming is somewhat controversial, a proposal can be submitted.
Fixes a few items from review - Improved error handling with -fsycl-help=gen - Make getOclocPath a static helper - make --ocloc-path= usage consistent when no value provided
6e0734d to
fff04f4
Compare
Use of |
sarnex
left a comment
There was a problem hiding this comment.
mostly lgtm, just some minor questions
| // Forward the user provided location for ocloc. | ||
| if (Arg *A = Args.getLastArg(options::OPT_ocloc_path_EQ)) | ||
| CmdArgs.push_back( | ||
| Args.MakeArgString(Twine("--ocloc-path=") + A->getValue())); |
There was a problem hiding this comment.
can we have a test where the path contains spaces just to make sure we don't get any nonsense
| // An empty --ocloc-path= is rejected here for consistent usage for areas | ||
| // that consume it. | ||
| if (A->getOption().matches(options::OPT_ocloc_path_EQ) && | ||
| A->containsValue("")) { |
There was a problem hiding this comment.
will this fail if the path contains a space?
There was a problem hiding this comment.
It's only checking for no argument and doesn't care if the string itself is malformed or contains a space.
There was a problem hiding this comment.
ah sorry i misread the code thanks
| } | ||
| // Run the Tool. | ||
| llvm::sys::ExecuteAndWait(ToolBinary.get(), ToolArgs); | ||
| if (llvm::sys::ExecuteAndWait(ToolBinary.get(), ToolArgs) < 0) |
There was a problem hiding this comment.
probably a nit:
the doc for ExecuteAndWait says
Returns
an integer result code indicating the status of the program. A zero or positive value indicates the result code of the program. -1 indicates failure to execute -2 indicates a crash during execution or timeout
So it will definitely return something < 0 if it failed to run/crash, but I think it will be positive if it ran but ocloc returns non-zero. Do we want to throw the error in that case or is that case handled elsewhere?
There was a problem hiding this comment.
I will update to cover all non-zero. The call here is just for emitting tool help information
| } | ||
| // Run the Tool. | ||
| llvm::sys::ExecuteAndWait(ToolBinary.get(), ToolArgs); | ||
| if (!llvm::sys::ExecuteAndWait(ToolBinary.get(), ToolArgs)) |
There was a problem hiding this comment.
i might be overthinking this but we only want to go into the if if the result is not exacltly 0 right? either < 0 or >0 we want to error right?, if so i dont think this condition does that
There was a problem hiding this comment.
Hmm... I think my brain is fried.
The ocloc tool used for ahead of time compilation targeting Intel GPUs is
acquired externally, so it is not guaranteed to be placed in the PATH.
Add --ocloc-path=dir, allowing a user to specify the directory in which
ocloc resides. The user provided location always wins over any ocloc that
is otherwise visible via the program paths or the PATH environment
variable.
The option is honored for both the old and the new offloading model
the ocloc help emitted for -fsycl-help=gen.
for the --sycl-link path. Both tools also accept --ocloc-path=
directly for standalone use.