ship versioned tbb libraries on linux again - #260
Merged
Conversation
The oneTBB cmake build only versions its output on Windows, so on Linux RcppParallel 6.0.x installed unversioned libraries (e.g. 'libtbb.so' with SONAME 'libtbb.so'). RcppParallel 5.1.11 and earlier shipped a real 'libtbb.so.2' plus a 'libtbb.so' redirect, inherited from Intel TBB's make-based build (which set the SONAME from TBB_COMPATIBLE_INTERFACE_VERSION). Binaries compiled against those releases recorded a load-time dependency on 'libtbb.so.2', and so fail to load after an upgrade with "libtbb.so.2: cannot open shared object file". Restore the versioned layout for the bundled TBB on Linux by renaming 'libtbb*.so' to 'libtbb*.so.2' and re-creating 'libtbb*.so' as a symlink. A symlink (rather than a linker script) is used so the name resolves at runtime as well as at link time.
kevinushey
added a commit
that referenced
this pull request
Jul 28, 2026
Comparing src/tbb against a pristine oneTBB 2022.0.0 turns up 41 modified
files. Most are workarounds for over-zealous CRAN checks -- commented-out
'#pragma warning' / '#pragma GCC diagnostic' directives and the like -- and
are not worth carrying. Seven are not:
- src/tbb/def/{lin32,lin64,mac64}-tbb.def add
tbb::internal::task_scheduler_observer_v3::observe to the export list,
under a comment reading 'Needed by rstan', along with std:: exception
typeinfo and destructors for backwards compatibility. These are the
Linux/macOS counterpart of the Windows tbb.dll stub, so losing them would
break rstan on exactly the platforms where it currently works.
- The four CMakeLists.txt files drop VERSION / SOVERSION, so the build emits
unversioned libraries. install.libs.R's versionBundledTbbLibraries() then
re-creates the 'libtbb.so.2' layout (#260); the two have to stay in step.
Both patches verified by applying them to a pristine tree and confirming
every touched file matches what we ship.
kevinushey
added a commit
that referenced
this pull request
Jul 28, 2026
* link downstream packages against the tbb stub on windows
tbbLdFlags() returned only '-L<libs> -lRcppParallel' on Windows, leaving
downstream packages to resolve TBB symbols against whichever objects
happened to be pulled out of the static TBB when RcppParallel.dll was
linked. That export surface is incidental rather than declared, and
packages taking all of their PKG_LIBS from RcppParallelLibs() (e.g. rstan)
have no other way to ask for TBB.
Offer the stub library as well, after '-lRcppParallel' so the package's own
exports still take precedence and no dependency on the stub is recorded
unless something actually needs it. RcppParallel 5.1.11 and earlier appended
'-ltbb -ltbbmalloc' here for the same reason.
* build the bundled onetbb on windows when rtools has no onetbb
Rtools42 ships Intel TBB 2017, and configure adopted it purely on the
presence of a libtbb* archive. Downstream packages then compiled against
those headers, which is not viable: StanHeaders uses
tbb::this_task_arena::isolate unconditionally, TBB 2017 still gates it
behind TBB_PREVIEW_TASK_ISOLATION, and its library doesn't export
isolate_within_arena, so forcing the declaration into view wouldn't link
either. rstan 2.32.7 consequently stopped building on R 4.2 / Windows.
Only adopt the Rtools TBB when it is oneTBB, and otherwise build the
bundled copy from sources, as we already do elsewhere. Rtools42 provides
cmake 3.24.1 in the same directory as gcc, so this needs no new tooling.
It is built as a static library so that RcppParallel.dll takes exactly the
shape it does with an Rtools oneTBB -- oneTBB marks its entry points
__declspec(dllexport) on mingw, so the whole-archive re-export of tbbmalloc
carries over unchanged.
Toolchains that can build neither (e.g. Rtools40, which has no TBB and no
cmake) still fall back to tinythread: a missing or too-old cmake is only an
error off Windows, where the bundled build is the sole option.
Also add CI covering this. R CMD check gains an R 4.2 Windows job, and a new
'downstream' job compiles and loads a translation unit against the flags
RcppParallel advertises, exercising both this_task_arena::isolate and
task_scheduler_observer. R CMD check alone would not have caught the rstan
breakage: RcppParallel installed perfectly well throughout.
* add news entries for the windows tbb changes
* run the downstream ci job under bash
The default shell on Windows runners is PowerShell, where 'r' is an alias
for Invoke-History; 'R CMD INSTALL' was parsed as 'Invoke-History CMD
INSTALL' and failed before the job did anything useful.
* patch onetbb's assembler probe for native windows builds
oneTBB probes the GNU assembler version by compiling /dev/null, which does
not exist when a native Windows cmake drives mingw gcc. The compiler fails
with 'no input files', the version regex doesn't match, and math() then
chokes on the error text -- taking the whole configure down:
CMake Error at cmake/compilers/GNU.cmake:65 (math):
math cannot parse the expression: "g++.exe: error: /dev/null: No such
file or directory ... * 1000 + ..."
MXE doesn't hit this because it cross-builds oneTBB from a POSIX host.
Use NUL on Windows, and treat an unparseable probe as an assembler too old
for waitpkg rather than a fatal error -- it only guards an optimization, and
Rtools42's GCC 10.4 is below the GCC 11 floor for -mwaitpkg regardless.
* don't pass -fstack-clash-protection to mingw when building onetbb
GCC 10.4 aborts compiling oneTBB's task_dispatcher.h with the flag on:
internal compiler error: in seh_emit_stackalloc, at config/i386/winnt.c:1056
370 | do_throw_noexcept([] { throw; });
a known GCC defect on mingw's SEH targets. It is a hardening flag rather
than a correctness one, so leave it off there.
Also fold the earlier /dev/null fix into a single patches/mingw_gnu_cmake.diff,
as both make oneTBB's GNU.cmake usable for a native mingw build.
* don't redeclare the legacy observer classes in the tbb stub
The bundled oneTBB carries a local 'provided for backwards compatibility'
block declaring tbb::internal::task_scheduler_observer_v3 and
tbb::interface6::task_scheduler_observer (added in c276b03; not upstream).
tbb-compat.cpp declares them too, which was fine while the stub was only
ever built against the pristine headers Rtools ships -- but building the
bundled oneTBB on Windows makes the two collide:
tbb-compat.cpp:25:7: error: redefinition of
'class tbb::internal::task_scheduler_observer_v3'
Have the bundled header advertise what it provides, and skip the stub's own
declarations when it does. Either way the stub still exports the out-of-line
observe(), which is the whole reason it exists.
Also spell that definition __TBB_EXPORTED_METHOD, matching its declaration
here and in the old TBB ABI; the two attributes are both no-ops on the
Windows targets we build for, so this is consistency rather than a fix.
* keep the legacy observe definition out of the bundled tbb on windows
The bundled oneTBB carries a local (non-upstream) definition of
tbb::internal::task_scheduler_observer_v3::observe in observer_proxy.cpp,
alongside the declarations in task_scheduler_observer.h. Once the bundled
copy is actually built on Windows, libtbb12.a and tbb-compat.o both define
it and the stub fails to link:
libtbb12.a(observer_proxy.cpp.obj): multiple definition of
'tbb::internal::task_scheduler_observer_v3::observe(bool)'
The stub has to own that definition on Windows, because R CMD SHLIB builds
the DLL's export list from the objects it compiles -- a definition sitting
in the archive would never make it into tmp.def. So leave it out of the
library there; other platforms keep it, where it serves binaries built
against RcppParallel 5.x.
Also stop trusting R CMD SHLIB's exit code: it reported success through the
link failure above, so the install completed and shipped a package with no
tbb.dll -- precisely the failure mode that packages linking '-ltbb' only
hit at load time. Check for the artifact, and for the copy, instead.
* capture the legacy tbb abi modifications as a patch
The bundled oneTBB carries local declarations and a definition of the old
ABI's task_scheduler_observer, added in c276b03 so that binaries built
against RcppParallel 5.x keep working. Neither had a file in patches/, and
tools/tbb/update-tbb.R wipes src/tbb wholesale -- so the next oneTBB bump
would have dropped them silently, breaking the Windows tbb.dll stub in a
way that is hard to trace back.
Verified by applying the patch to a pristine oneTBB 2022.0.0 tree and
confirming the result matches what we ship.
* add a manually-triggered rstan build workflow
The 'downstream' job stands in for rstan in the normal CI run: it compiles
a translation unit using the two TBB APIs StanHeaders needs, in seconds
rather than the best part of an hour. This runs the real thing, on demand,
for when that's worth waiting for -- before a release, or after touching
the Windows TBB setup.
workflow_dispatch inputs choose the R version (4.2 by default, the one whose
Rtools has no oneTBB), the runner, and whether to additionally compile and
fit a tiny Stan model. The last exercises the TBB runtime at run time, not
just the headers and link line, so it's opt-in.
* correct rstan's dependency list and pin the package under test
Drop V8 and withr: rstan 2.32 replaced V8 with QuickJSR, and V8 failing to
install on an old toolchain would have failed the job for reasons that have
nothing to do with RcppParallel. The list now mirrors rstan's Imports plus
LinkingTo.
Install StanHeaders and rstan with dependencies = FALSE, and assert
afterwards that RcppParallel is still the .9000 build from this branch --
otherwise dependency resolution could quietly replace it with CRAN's, and
the run would report on the released package instead of this one.
* capture the functional oneTBB modifications as patches
Comparing src/tbb against a pristine oneTBB 2022.0.0 turns up 41 modified
files. Most are workarounds for over-zealous CRAN checks -- commented-out
'#pragma warning' / '#pragma GCC diagnostic' directives and the like -- and
are not worth carrying. Seven are not:
- src/tbb/def/{lin32,lin64,mac64}-tbb.def add
tbb::internal::task_scheduler_observer_v3::observe to the export list,
under a comment reading 'Needed by rstan', along with std:: exception
typeinfo and destructors for backwards compatibility. These are the
Linux/macOS counterpart of the Windows tbb.dll stub, so losing them would
break rstan on exactly the platforms where it currently works.
- The four CMakeLists.txt files drop VERSION / SOVERSION, so the build emits
unversioned libraries. install.libs.R's versionBundledTbbLibraries() then
re-creates the 'libtbb.so.2' layout (#260); the two have to stay in step.
Both patches verified by applying them to a pristine tree and confirming
every touched file matches what we ship.
* note the tbb stub install failure fix in news
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On Linux, RcppParallel 6.0.x installs unversioned TBB libraries —
libtbb.sowith SONAMElibtbb.so, and nolibtbb.so.2. This is a regression from 5.1.11 and earlier, which shipped a reallibtbb.so.2plus alibtbb.soredirect.The cause is the oneTBB cmake build: it only versions its output under
if (WIN32)(OUTPUT_NAME "tbb${TBB_BINARY_VERSION}") and sets noSOVERSION, so Linux/macOS get the bare target name. 5.1.11's versioned layout was never something RcppParallel did explicitly — it came from Intel TBB's make-based build (build/linux.incset the SONAME suffix fromTBB_COMPATIBLE_INTERFACE_VERSION, i.e.2, andMakefile.tbbemittedlibtbb.soas anINPUT(libtbb.so.2)linker script).Binaries compiled against RcppParallel ≤ 5.1.11 recorded a load-time dependency on
libtbb.so.2. After an upgrade to 6.0.x that file no longer exists, so they fail to load:This was reported by the CRAN maintainer during revdep checking on Fedora.
Fix
For the bundled TBB on Linux only, after copying the libraries, rename each real unversioned
libtbb*.sotolibtbb*.so.2and re-createlibtbb*.soas a relative symlink pointing back at it. This restores the historical layout so legacyNEEDED libtbb.so.2dependencies resolve again.INPUT(...)) works at link time but the dynamic loader can't parse it, so it would fail at runtime — and 6.0.x'sRcppParallel.soitself hasNEEDED libtbb.so.Verification
Rebuilt into a temporary library on Linux:
libtbb.so.2(real) +libtbb.so -> libtbb.so.2fortbb,tbbmalloc, andtbbmalloc_proxy.setThreadOptions()works;tbbLibraryPath("tbb")returns the.so.2.RcppParallel.so's ownNEEDED libtbb.soresolves via the new symlink.NEEDED libtbb.so.2(mimicking a 5.1.11-built revdep) now resolves the file against the new install — the load error is gone.Note
Giving the new-ABI oneTBB the legacy
.so.2name means an old binary now loads it and could fault on a removed symbol rather than getting a clean file-not-found; that is what the existing compatibility headers/symbols are for, and a rebuild remains the real fix. This change restores pre-6.0 behavior and stops the file-level breakage.