build the bundled onetbb on windows when rtools has no onetbb - #269
Merged
Conversation
tbbLdFlags() returned only '-L<libs> -lRcppParallel' on Windows, leaving downstream packages to resolve TBB symbols against whichever objects happened to be pulled out of the static TBB when RcppParallel.dll was linked. That export surface is incidental rather than declared, and packages taking all of their PKG_LIBS from RcppParallelLibs() (e.g. rstan) have no other way to ask for TBB. Offer the stub library as well, after '-lRcppParallel' so the package's own exports still take precedence and no dependency on the stub is recorded unless something actually needs it. RcppParallel 5.1.11 and earlier appended '-ltbb -ltbbmalloc' here for the same reason.
Rtools42 ships Intel TBB 2017, and configure adopted it purely on the presence of a libtbb* archive. Downstream packages then compiled against those headers, which is not viable: StanHeaders uses tbb::this_task_arena::isolate unconditionally, TBB 2017 still gates it behind TBB_PREVIEW_TASK_ISOLATION, and its library doesn't export isolate_within_arena, so forcing the declaration into view wouldn't link either. rstan 2.32.7 consequently stopped building on R 4.2 / Windows. Only adopt the Rtools TBB when it is oneTBB, and otherwise build the bundled copy from sources, as we already do elsewhere. Rtools42 provides cmake 3.24.1 in the same directory as gcc, so this needs no new tooling. It is built as a static library so that RcppParallel.dll takes exactly the shape it does with an Rtools oneTBB -- oneTBB marks its entry points __declspec(dllexport) on mingw, so the whole-archive re-export of tbbmalloc carries over unchanged. Toolchains that can build neither (e.g. Rtools40, which has no TBB and no cmake) still fall back to tinythread: a missing or too-old cmake is only an error off Windows, where the bundled build is the sole option. Also add CI covering this. R CMD check gains an R 4.2 Windows job, and a new 'downstream' job compiles and loads a translation unit against the flags RcppParallel advertises, exercising both this_task_arena::isolate and task_scheduler_observer. R CMD check alone would not have caught the rstan breakage: RcppParallel installed perfectly well throughout.
kevinushey
force-pushed
the
feature/windows-bundled-onetbb
branch
from
July 27, 2026 23:34
438257d to
e13d475
Compare
The default shell on Windows runners is PowerShell, where 'r' is an alias for Invoke-History; 'R CMD INSTALL' was parsed as 'Invoke-History CMD INSTALL' and failed before the job did anything useful.
oneTBB probes the GNU assembler version by compiling /dev/null, which does
not exist when a native Windows cmake drives mingw gcc. The compiler fails
with 'no input files', the version regex doesn't match, and math() then
chokes on the error text -- taking the whole configure down:
CMake Error at cmake/compilers/GNU.cmake:65 (math):
math cannot parse the expression: "g++.exe: error: /dev/null: No such
file or directory ... * 1000 + ..."
MXE doesn't hit this because it cross-builds oneTBB from a POSIX host.
Use NUL on Windows, and treat an unparseable probe as an assembler too old
for waitpkg rather than a fatal error -- it only guards an optimization, and
Rtools42's GCC 10.4 is below the GCC 11 floor for -mwaitpkg regardless.
GCC 10.4 aborts compiling oneTBB's task_dispatcher.h with the flag on:
internal compiler error: in seh_emit_stackalloc, at config/i386/winnt.c:1056
370 | do_throw_noexcept([] { throw; });
a known GCC defect on mingw's SEH targets. It is a hardening flag rather
than a correctness one, so leave it off there.
Also fold the earlier /dev/null fix into a single patches/mingw_gnu_cmake.diff,
as both make oneTBB's GNU.cmake usable for a native mingw build.
kevinushey
force-pushed
the
feature/windows-bundled-onetbb
branch
from
July 27, 2026 23:49
961cd0f to
00076d9
Compare
The bundled oneTBB carries a local 'provided for backwards compatibility' block declaring tbb::internal::task_scheduler_observer_v3 and tbb::interface6::task_scheduler_observer (added in c276b03; not upstream). tbb-compat.cpp declares them too, which was fine while the stub was only ever built against the pristine headers Rtools ships -- but building the bundled oneTBB on Windows makes the two collide: tbb-compat.cpp:25:7: error: redefinition of 'class tbb::internal::task_scheduler_observer_v3' Have the bundled header advertise what it provides, and skip the stub's own declarations when it does. Either way the stub still exports the out-of-line observe(), which is the whole reason it exists. Also spell that definition __TBB_EXPORTED_METHOD, matching its declaration here and in the old TBB ABI; the two attributes are both no-ops on the Windows targets we build for, so this is consistency rather than a fix.
The bundled oneTBB carries a local (non-upstream) definition of
tbb::internal::task_scheduler_observer_v3::observe in observer_proxy.cpp,
alongside the declarations in task_scheduler_observer.h. Once the bundled
copy is actually built on Windows, libtbb12.a and tbb-compat.o both define
it and the stub fails to link:
libtbb12.a(observer_proxy.cpp.obj): multiple definition of
'tbb::internal::task_scheduler_observer_v3::observe(bool)'
The stub has to own that definition on Windows, because R CMD SHLIB builds
the DLL's export list from the objects it compiles -- a definition sitting
in the archive would never make it into tmp.def. So leave it out of the
library there; other platforms keep it, where it serves binaries built
against RcppParallel 5.x.
Also stop trusting R CMD SHLIB's exit code: it reported success through the
link failure above, so the install completed and shipped a package with no
tbb.dll -- precisely the failure mode that packages linking '-ltbb' only
hit at load time. Check for the artifact, and for the copy, instead.
The bundled oneTBB carries local declarations and a definition of the old ABI's task_scheduler_observer, added in c276b03 so that binaries built against RcppParallel 5.x keep working. Neither had a file in patches/, and tools/tbb/update-tbb.R wipes src/tbb wholesale -- so the next oneTBB bump would have dropped them silently, breaking the Windows tbb.dll stub in a way that is hard to trace back. Verified by applying the patch to a pristine oneTBB 2022.0.0 tree and confirming the result matches what we ship.
The 'downstream' job stands in for rstan in the normal CI run: it compiles a translation unit using the two TBB APIs StanHeaders needs, in seconds rather than the best part of an hour. This runs the real thing, on demand, for when that's worth waiting for -- before a release, or after touching the Windows TBB setup. workflow_dispatch inputs choose the R version (4.2 by default, the one whose Rtools has no oneTBB), the runner, and whether to additionally compile and fit a tiny Stan model. The last exercises the TBB runtime at run time, not just the headers and link line, so it's opt-in.
Drop V8 and withr: rstan 2.32 replaced V8 with QuickJSR, and V8 failing to install on an old toolchain would have failed the job for reasons that have nothing to do with RcppParallel. The list now mirrors rstan's Imports plus LinkingTo. Install StanHeaders and rstan with dependencies = FALSE, and assert afterwards that RcppParallel is still the .9000 build from this branch -- otherwise dependency resolution could quietly replace it with CRAN's, and the run would report on the released package instead of this one.
Comparing src/tbb against a pristine oneTBB 2022.0.0 turns up 41 modified
files. Most are workarounds for over-zealous CRAN checks -- commented-out
'#pragma warning' / '#pragma GCC diagnostic' directives and the like -- and
are not worth carrying. Seven are not:
- src/tbb/def/{lin32,lin64,mac64}-tbb.def add
tbb::internal::task_scheduler_observer_v3::observe to the export list,
under a comment reading 'Needed by rstan', along with std:: exception
typeinfo and destructors for backwards compatibility. These are the
Linux/macOS counterpart of the Windows tbb.dll stub, so losing them would
break rstan on exactly the platforms where it currently works.
- The four CMakeLists.txt files drop VERSION / SOVERSION, so the build emits
unversioned libraries. install.libs.R's versionBundledTbbLibraries() then
re-creates the 'libtbb.so.2' layout (#260); the two have to stay in step.
Both patches verified by applying them to a pristine tree and confirming
every touched file matches what we ship.
This was referenced Jul 28, 2026
kevinushey
added a commit
that referenced
this pull request
Jul 28, 2026
* build the bundled onetbb as a shared library on windows Windows was the only platform where TBB was not a shared library loaded at runtime, and nearly every Windows-specific wart traced back to that. Use the bundled oneTBB there too, built shared, shipped as tbb.dll / tbbmalloc.dll and linked exactly as on other platforms. Rtools ships static libraries only, so adopting its TBB meant linking it into RcppParallel.dll. That made the TBB version -- and ABI -- a property of the user's toolchain (Intel TBB 2017 on Rtools42, oneTBB later), and left downstream packages with no TBB library to link against. The workarounds accumulated: re-exporting the whole tbbmalloc archive so the allocator could be resolved (#262), a tbb.dll stub so packages linking -ltbb could load (#249, #250), building that stub differently for legacy toolchains (#258), offering it from RcppParallelLibs() so rstan could link at all (#269), and a library lookup that searched for archives never installed with the package (#270). Worse, the stub linked the TBB archives itself, so it was a second complete copy of the oneTBB scheduler. Both it and RcppParallel.dll export the same 81 tbb::detail::r1:: entry points, both are loaded in every session, and which one a downstream package binds to is up to the linker. Two schedulers with separate thread pools and separate global state is a correctness problem, not a linking inconvenience. This removes: - the Rtools TBB detection in configure.R, and with it the TBB_NAME derivation from archive names and the tbb12 special case - the --whole-archive tbbmalloc re-export, which only existed because there was no DLL to link - BUILD_SHARED_LIBS=0 for Windows - src/tbb-compat/ and buildTbbStub() entirely, along with the #ifndef _WIN32 guard in observer_proxy.cpp and the __TBB_LEGACY_..._PROVIDED macro that let the stub tell which headers it was compiling against - the Windows branches in .install.libs(), loadTbbLibrary(), tbbLdFlags() and tbbLibraryPath(), and the Windows-first load order in .onLoad() Notes on the pieces that are not just deletions: - mingw CMake would name a shared build libtbb.dll; patch PREFIX to "" so the runtime is tbb.dll, the name binaries built against 5.1.11 and earlier import. The import library keeps its lib prefix, so -ltbb still resolves. - the legacy task_scheduler_observer_v3::observe definition now applies on Windows too. mingw has no .def file, so its export comes from a dllexport directive: the declaration gains TBB_EXPORT, which is empty when consuming the headers. - .onLoad loads tbb before RcppParallel on every platform now, since RcppParallel.dll imports from tbb.dll and the package library directory is not on the DLL search path. - tbbLdFlags() keeps -lRcppParallel on Windows: isProcessForkedChild is compiled into RcppParallel.dll, and Windows has no lazy binding to resolve it at load time. The downstream CI check now asserts the property this is all for: the downstream library's TBB symbols come from exactly one module, that module is tbb.dll, and RcppParallel.dll imports from it rather than carrying its own copy. * fix the shared tbb link on rtools42 and aarch64 windows Two failures from the first CI run, both only reachable now that TBB is linked as a shared library on Windows -- a static build has no link step. Rtools42 (gcc 10): the TBB link failed with undefined references to __stack_chk_fail, __strncpy_chk and __strncat_chk. TBB compiles with -fstack-protector-strong and _FORTIFY_SOURCE, and on mingw those helpers live in libssp rather than libgcc; newer toolchains pull it in implicitly, Rtools42 does not. Pass -lssp via CMAKE_SHARED_LINKER_FLAGS, mirroring the -lssp that configure.R already adds for RcppParallel's own link. aarch64 Windows: 'lld: error: unknown argument: -z'. Rtools45-aarch64 drives clang with lld, and Clang.cmake adds -Wl,-z,relro,-z,now for any non-Apple target. GNU.cmake already guards this with NOT MINGW; give Clang.cmake the equivalent guard. windows-latest (release) passed on the first run, so the shared mingw build, the PREFIX "" naming and the reordered load in .onLoad() all work. * link libssp via TBB_COMMON_LINK_LIBS rather than the shared linker flags TBB compiles with -fstack-protector-strong and _FORTIFY_SOURCE=2, and on mingw the helpers those need (__stack_chk_fail, __stack_chk_guard, __strncpy_chk, ...) live in libssp rather than libgcc. Rtools42 (gcc 10) does not pull it in implicitly, so the link fails with undefined references to them. Passing -lssp through CMAKE_SHARED_LINKER_FLAGS was not enough: those flags are emitted before the objects, and GNU ld only resolves symbols that are already undefined when it reaches a library, so it was discarded. tbb and tbbmalloc happened to survive that because tbb_handle_ipo gives them LTO; tbbmalloc_proxy has no such call and failed to link. Putting ssp in TBB_COMMON_LINK_LIBS routes it through target_link_libraries instead, which lands after the objects, and covers all three targets. * fix the RcppParallel.dll half of the downstream import check Two reasons it was never actually running, both visible in the CI log even though the job passed: - the path was built with paste0("libs", .Platform$r_arch), giving 'libsx64' rather than 'libs/x64'. system.file() returned "" for that, so objdump was handed '/RcppParallel.dll' and reported "No such file". Use archSystemFile(), which the package already has for this, and fail loudly if the library still can't be found. - on aarch64 the runner has an x86_64 objdump from C:/mingw64 ahead of Rtools' own on the PATH, and it cannot read an aarch64 PE, so both halves of the check skipped there. Try the objdump sitting beside the compiler first, and fall back through the remaining candidates. * locate objdump via the compiler R actually builds with The previous attempt looked beside Sys.which("gcc"/"clang"), but on the aarch64 runner those resolve to C:/mingw64 -- the same x86_64 toolchain that leads the PATH -- so both candidates were the one objdump that cannot read an aarch64 PE, and the check still skipped there. They were also not deduplicated, differing only in slash direction. 'R CMD config CC' names the compiler that produced the library, so its objdump can read it by construction. Try that first, and normalize the candidates so one objdump is not tried under two spellings. * don't emit -ltbb when no tbb was installed The Windows branch of tbbLdFlags() named the TBB libraries unconditionally, dropping the file.exists() guard the previous stub-based code had. On a build that fell back to tinythread -- configure.R sets TBB_ENABLED = FALSE only there, when cmake is missing and TBB_LIB is unset -- RcppParallelLibs() then emitted -L<libs> -lRcppParallel -L<lib> -ltbb -ltbbmalloc for libraries that were never built or shipped, so a downstream package would fail to link with "cannot find -ltbb". CxxFlags() correctly reports -DRCPP_PARALLEL_USE_TBB=0 in that configuration, so the package is not even using TBB. Name them only when TBB is enabled and the library actually resolves. Also drop an unverified aside from NEWS: Rtools42 and Rtools45 are confirmed to ship cmake, Rtools40 was not checked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes two Windows linking problems, reported against 6.1.0 on R 4.2 / Rtools42
after rstan 2.32.7 stopped building there. It built fine against 5.1.11-2.
Background
Rtools42 ships Intel TBB 2017 (
TBB_INTERFACE_VERSION 9107) inx86_64-w64-mingw32.static.posix: static libraries only, nooneapi/, notbb/version.h. Since 6.0.0, configure adopts an Rtools TBB purely on thepresence of a
libtbb*archive, so that tree gets put on downstream includepaths. Two things then break:
Compile. StanHeaders uses
tbb::this_task_arena::isolateunconditionally (
map_rect_concurrent.hpp:54,reduce_sum.hpp:257). TBB2017 defines
__TBB_TASK_ISOLATIONas(__TBB_CPF_BUILD||TBB_PREVIEW_TASK_ISOLATION), i.e. 0, and gatesisolateon it. Forcing the declaration into view would not help either:
nmonlibtbb_static.ashows noisolate_within_arena.Link.
tbbLdFlags()returned only-L<libs> -lRcppParallelon Windows,so downstream packages could only resolve TBB symbols that
RcppParallel.dllhappened to re-export. rstan takes all of itsPKG_LIBSfrom
RcppParallelLibs()and never spells-ltbbitself, henceundefined reference to tbb::internal::task_scheduler_observer_v3::observe(bool).For contrast, 5.1.11-2 bundled TBB 2019, which enables
__TBB_TASK_ISOLATIONby default. Neither release hastbb/version.h, so-DTBB_INTERFACE_NEWwas absent in both cases -- the 2019 -> 2017 downgrade iswhat actually broke Stan, not the interface macro.
Changes
oneTBB from sources. Rtools42 provides cmake 3.24.1 in the same directory as
gcc, so this needs no new tooling. It is built static, so
RcppParallel.dlltakes exactly the shape it does with an Rtools oneTBB -- oneTBB marks its
entry points
__declspec(dllexport)on mingw (detail/_export.h), so thewhole-archive re-export of tbbmalloc carries over unchanged.
back to tinythread. A missing or too-old cmake is only an error off Windows.
RcppParallelLibs()offers the stub library again, after-lRcppParallelsothe package's own exports still take precedence.
Verified locally
is byte-identical to before this PR.
What this PR needs CI to answer
None of the Windows behaviour could be tested locally:
-mwaitpkgis gated to-G "MSYS Makefiles"the right generator under Rtools? Least confidentchoice here.
libtbb12.a/libtbbmalloc.a?RcppParallel.dllthe way the Rtools prebuilt one does?
The CI additions target exactly these: an R 4.2 Windows job, plus a
downstreamjob on R 4.2 and release that compiles and loads a translationunit using
this_task_arena::isolateandtask_scheduler_observer. PlainR CMD checkwould not have caught the rstan breakage -- RcppParallelinstalled perfectly well throughout.
A full rstan build is the real acceptance test and has not been run.