Skip to content

cuda.core: eliminate build warnings; set option to build with warnings as errors - #2966

Merged
Andy-Jost merged 3 commits into
NVIDIA:mainfrom
Andy-Jost:ajost/cython-warning-cleanup
Sep 30, 2026
Merged

Andy-Jost merged 3 commits into
NVIDIA:mainfrom
Andy-Jost:ajost/cython-warning-cleanup

Conversation

@Andy-Jost

@Andy-Jost Andy-Jost commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The wheel builds print no Cython errors or warnings, since build_hooks.py already sets warning_errors, but each build prints about 150 Cython performance hints, a dozen gcc warnings, and several hundred MSVC narrowing warnings. All of them trace back to a handful of source lines. This PR fixes the ones that are ours, silences the ones we can't fix, and treats warnings as errors to avoid future drift.

Changes

  • build_hooks.py, build-wheel.yml, AGENTS.md: the CUDA_PYTHON_WERROR switch described below, set for the CI wheel builds.

  • _rt.pxd / _rt.pyx: sm_resource_split and memcpy_with_attributes_async return a CUresult and never raise, but were declared nogil without noexcept. Cython cannot pick an error sentinel for an enum return type, so every call from a nogil block acquired the GIL to run PyErr_Occurred(). Both are now noexcept nogil, which removes the two hints Cython raised at their call sites in _device_resources.pyx and _memory/_buffer.pyx.

  • _rt.pyx: # cython: show_performance_hints=False, with a comment explaining why. Every except+ nogil handle factory trips the same generic hint, but for except+ the GIL is taken only inside the C++ catch handler, so the success path pays nothing. That accounts for 54 of the hints.

  • _memoryview.pyx: initialize three locals that gcc could not prove set on every path.

  • graph/_graph_builder.pyx: a guarded #pragma GCC diagnostic ignored "-Wunused-function" in a cdef extern from * block. Each @overload stub of Graph.__getitem__ compiles to a static wrapper that nothing references, because only the final definition is bound, and the stubs stay because stubgen reads them. The pragma covers only that translation unit and sits next to the comment that explains the stubs.

  • system/_device_utils.pxi, _device_resources.pyx: loop index types now match their bounds (sign-compare).

  • _layout.pxd / _layout.pyx, _tensor_map.pyx, _launch_config.pyx, _stream.pyx, _device_resources.pyx: explicit casts where a Py_ssize_t, size_t, int64_t, or long feeds an int or unsigned int that only ever holds a rank, a count, or a gcd bounded by its int input, plus an int return type for _init_dense, which returns a status. These are the MSVC C4244/C4267 sites in hand-written code.

What stays

  • MSVC C4551 (643 per build) and the remaining C4244 come from Cython's own utility code and from the overflow-check helpers that @cython.overflowcheck(True) instantiates in _layout.pxd. They cannot be fixed in .pyx sources.
  • The remaining Cython hints are in generated cuda.bindings files.

Warnings as errors

Cython warnings are already errors, since build_hooks.py sets warning_errors. This PR adds the compiler side as an opt-in: CUDA_PYTHON_WERROR=1 appends -Werror on gcc/clang and /WX /wd4551 /wd4244 on MSVC. The two MSVC exemptions are the families above that Cython's utility code produces in every module; gcc and clang need none. The CI wheel builds set the variable in build-wheel.yml; source builds and local builds do not, because they run on compilers we do not control. A debug build with the variable set trips the glibc _FORTIFY_SOURCE warning at -O0, so it is meant for optimized builds.

If a future Cython or compiler version introduces a new warning class in generated code, the CI build fails and the exemption list or the source gets the fix. That is the point of the switch.

🤖 Generated with Claude Code

The wheel builds print no Cython errors or warnings (warning_errors is
already on), but they do print about 150 Cython performance hints per
build, a dozen gcc warnings, and several hundred MSVC narrowing warnings
that all trace back to a handful of source lines.

- _rt: sm_resource_split and memcpy_with_attributes_async return a
  CUresult and never raise, but were declared nogil without noexcept.
  Cython cannot pick an error sentinel for an enum, so every call from a
  nogil block acquired the GIL to run PyErr_Occurred(). Both are now
  noexcept nogil. The remaining hints in this module come from the
  except+ nogil handle factories, where the GIL is taken only inside the
  C++ catch handler; the module turns show_performance_hints off and
  says why.
- _memoryview: initialize three locals that gcc could not prove set on
  every path (maybe-uninitialized).
- system/_device_utils.pxi, _device_resources: match loop index types to
  their bounds (sign-compare).
- _layout, _tensor_map, _launch_config, _stream, _device_resources:
  explicit casts where a Py_ssize_t, size_t, int64_t or long feeds an
  int or unsigned int that only holds a rank, count or small gcd value,
  and an int return type for _init_dense, which returns a status. These
  are the MSVC C4244/C4267 sites in hand-written code; the ones left come
  from Cython's overflow-check helpers and its utility code.
- graph/_graph_builder: a guarded #pragma GCC diagnostic in a verbatim
  extern block turns off -Wunused-function for that translation unit.
  Each @overload stub of Graph.__getitem__ compiles to a wrapper that
  nothing references, because only the final definition is bound; the
  stubs stay because stubgen reads them.
- build_hooks: CUDA_PYTHON_WERROR=1 turns compiler warnings into errors
  (-Werror, or /WX with /wd4551 /wd4244 on MSVC for the warnings that
  Cython's utility code produces in every module). The wheel builds in
  CI set it; source builds do not.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Andy-Jost Andy-Jost added this to the cuda.core 1.3.0 milestone Sep 29, 2026
@Andy-Jost Andy-Jost added enhancement Any code-related improvements P0 High priority - Must do! CI/CD CI/CD infrastructure cuda.core Everything related to the cuda.core module labels Sep 29, 2026
@Andy-Jost Andy-Jost self-assigned this Sep 29, 2026
@Andy-Jost Andy-Jost changed the title cuda.core: clear Cython hints and compiler warnings, build CI wheels with warnings as errors cuda.core: eliminate build warnings; set option to build with warnings as errors Sep 29, 2026
…cleanup

# Conflicts:
#	.github/workflows/build-wheel.yml
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor
Doc Preview CI
Preview removed because the pull request was closed or merged.

@juenglin juenglin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks!

Comment thread cuda_core/cuda/core/_device_resources.pyx
Comment thread cuda_core/cuda/core/_layout.pxd
leofang added a commit to leofang/cuda-python that referenced this pull request Sep 30, 2026
resolve_toolchain gains a required keyword-only ``cxx_std`` argument and
an optional ``tweak`` post-hook. The shared ``_build_flags`` no longer
picks a default C++ standard — the two backends legitimately differ
(bindings stays on c++14 to avoid a c++17 variadic-template regression
on kernel-launch paths; core is on c++17), and there is no defensible
shared default. See NVIDIA#1882 for the audit.

- ``cuda_bindings/build_hooks.py`` adds ``_tweak_flags`` which layers
  ``-Wno-deprecated-declarations`` on Linux for the 38 warnings
  ``cudaMemcpy*Array*`` / ``cudaGetDriverEntryPoint`` produce in 13.4
  headers. Without it, once NVIDIA#2966 turns on ``-Werror``, bindings would
  stop building.
- Call site passes ``cxx_std=14, tweak=_tweak_flags``.
- ``cuda_core/build_hooks.py`` passes ``cxx_std=17`` (no tweak).

Also addresses Andy-Jost's line-anchored review comments:

- ``TestForceReachesBuildExt._finalized_build_ext`` now patches
  ``sys.modules["_build_shared"]`` instead of ``build_hooks``. Because
  ``build_hooks.force_build_ext`` is a module-level ``__getattr__``
  fallback, monkey-patching it on ``build_hooks`` creates a real
  attribute that shadows the fallback and persists across teardown —
  a poison state that made ``TestBuildConfigStamp`` order-dependent
  (reproduces under ``pytest -p randomly --randomly-seed=1``).
- Restores ``test_flag_set_forces_rebuild`` — the only direct check
  that ``force_build_ext=True`` reaches ``build_ext.force`` through
  the ``__getattr__`` re-export. Without it, a later
  ``from _build_shared import force_build_ext`` would silently snapshot
  ``False`` and forced rebuilds would stop with no failing test.
- Restores ``test_default_preserves_existing_cc`` in
  ``ResolveToolchainSharedMixin``. A green wheel build wouldn't catch
  a regression here — if the default toolchain overwrote ``CC``,
  sccache would silently stop and ``sccache-summary`` only warns.
- ``CONTRIBUTING.md`` recovery recipe now runs ``git config
  core.symlinks true`` (no ``--global``) inside the existing clone.
  ``git clone`` probes symlink support and writes
  ``core.symlinks=false`` into the *repo-local* config on failure;
  repo-local overrides ``--global``, so the previous recipe couldn't
  fix an existing clone.

Test bookkeeping:

- Per-package ``TestResolveToolchain.test_llvm_sets_env_and_flags``
  restored in both packages — bindings asserts ``-std=c++14`` +
  ``-Wno-deprecated-declarations``, core asserts ``-std=c++17`` and the
  absence of ``-Wno-deprecated-declarations``. Removed from the shared
  mixin because it now asserts package-specific flag content.
- Other shared-mixin resolve tests just pick ``cxx_std=17`` as a
  placeholder — they test env/name/error mechanics, not flag content.

Pass counts: bindings 29 (was 28), core 61 (was 59). Randomized-order
runs (``-p randomly --randomly-seed={1,2,4,42}``) all green.
@Andy-Jost
Andy-Jost requested a review from leofang September 30, 2026 17:40
@Andy-Jost
Andy-Jost merged commit a8ee450 into NVIDIA:main Sep 30, 2026
223 of 225 checks passed
@Andy-Jost
Andy-Jost deleted the ajost/cython-warning-cleanup branch September 30, 2026 17:41
leofang added a commit to leofang/cuda-python that referenced this pull request Sep 30, 2026
The auto-merge conflict was in cuda_core/build_hooks.py: HEAD had deleted
the whole in-file toolchain block (moved to _build_shared.py), while main
had added a -Werror / /WX branch inside that same block via NVIDIA#2966.

Resolved by taking HEAD's structural side and hoisting the toggle into
the shared code:

- _build_shared._build_flags and resolve_toolchain grow a
  warnings_as_errors=False keyword; the flag emission sits next to
  -O2/debug for a single legible flag path.
- cuda_core/build_hooks.py keeps its WARNINGS_AS_ERRORS constant (reads
  CUDA_PYTHON_WERROR) and threads it through the resolve_toolchain call
  site. No per-project _tweak_flags needed for this.
- Shared-mixin tests (test_warnings_as_errors_off_by_default,
  test_warnings_as_errors_on) guard both the default-off case and the
  exact tokens emitted per platform, so a future refactor can't silently
  disable CI's -Werror.

cuda_bindings does not currently pass warnings_as_errors=True; its
existing -Wno-deprecated-declarations tweak is order-independent from
-Werror and composes cleanly when it opts in later.
Andy-Jost added a commit to Andy-Jost/cuda-python that referenced this pull request Sep 30, 2026
…untime-floor

Conflicts in _rt.pxd and _rt.pyx: NVIDIA#2966 added noexcept to the sm_resource_split
and memcpy_with_attributes_async wrappers, which this branch removed. The
removal stands; NVIDIA#2966's performance-hints directive and its other changes are
kept.
github-actions Bot pushed a commit that referenced this pull request Oct 1, 2026
Removed preview folders for the following PRs:
- PR #2939
- PR #2953
- PR #2962
- PR #2966
- PR #2968
- PR #2969
- PR #2972
- PR #2974
- PR #2976
- PR #2977
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.core Everything related to the cuda.core module enhancement Any code-related improvements P0 High priority - Must do!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants