Skip to content

Improve readability of dgrad overlap variable#3165

Open
Prachi-kushwaha wants to merge 2 commits into
NVIDIA:mainfrom
Prachi-kushwaha:rename_dgrad_reduce_scatter_overlap
Open

Improve readability of dgrad overlap variable#3165
Prachi-kushwaha wants to merge 2 commits into
NVIDIA:mainfrom
Prachi-kushwaha:rename_dgrad_reduce_scatter_overlap

Conversation

@Prachi-kushwaha

Copy link
Copy Markdown

Description

Rename dgrad_reduce_scatter_overlap to better reflect its purpose.

Please include a brief summary of the changes, relevant motivation and context.

Fixes #3164

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@Prachi-kushwaha Prachi-kushwaha requested a review from ksivaman as a code owner July 2, 2026 02:33
@github-actions github-actions Bot added the community-contribution PRs from external contributor outside the core maintainers, representing community-driven work. label Jul 2, 2026
@greptile-apps

greptile-apps Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This is a pure rename refactoring that replaces the local variable dgrad_reduce_scatter_overlap with layers_dgrad_ag_and_rs_overlap in initialize_ub, touching one declaration and two loop usages in base.py.

  • The new name adds the layers_ prefix, aligning with the adjacent sibling variables layers_all_gather_overlap and layers_reduce_scatter_overlap, and more accurately conveys that the listed dgrad layers participate in both all-gather and reduce-scatter overlaps rather than reduce-scatter only.
  • No logic changes are made; the variable's contents and all call sites remain identical.

Confidence Score: 5/5

Safe to merge — the change is a mechanical rename with no logic or behavioral modifications.

All three sites (one declaration and two loop usages) are updated consistently. The variable's contents and every code path that consumes it are unchanged. The new name is strictly more descriptive, adding the layers_ prefix and reflecting that dgrad layers participate in both all-gather and reduce-scatter overlaps.

No files require special attention.

Important Files Changed

Filename Overview
transformer_engine/pytorch/module/base.py Renames dgrad_reduce_scatter_overlap to layers_dgrad_ag_and_rs_overlap in 3 places (declaration + 2 usages). The rename adds the layers_ prefix for consistency with siblings and reflects that the listed layers participate in both all-gather and reduce-scatter overlaps, not just reduce-scatter.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["initialize_ub()"] --> B["Declare layers_dgrad_ag_and_rs_overlap\n= ['qkv_dgrad', 'fc1_dgrad']"]
    B --> C["Loop 1: validate user_ub_cfg\nfor name in layers_dgrad_ag_and_rs_overlap"]
    C --> D{"name in user_ub_cfg\nand method set?"}
    D -- Yes --> E["Check method is ring_exchange or pipeline\n(raise error if so)"]
    D -- No --> F["continue"]
    B --> G["Loop 2: populate overlap lists\nfor name in layers_dgrad_ag_and_rs_overlap"]
    G --> H["Append name to\nlayers_reduce_scatter_overlap if missing"]
    G --> I["Append wgrad_name to\nlayers_all_gather_overlap if missing"]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A["initialize_ub()"] --> B["Declare layers_dgrad_ag_and_rs_overlap\n= ['qkv_dgrad', 'fc1_dgrad']"]
    B --> C["Loop 1: validate user_ub_cfg\nfor name in layers_dgrad_ag_and_rs_overlap"]
    C --> D{"name in user_ub_cfg\nand method set?"}
    D -- Yes --> E["Check method is ring_exchange or pipeline\n(raise error if so)"]
    D -- No --> F["continue"]
    B --> G["Loop 2: populate overlap lists\nfor name in layers_dgrad_ag_and_rs_overlap"]
    G --> H["Append name to\nlayers_reduce_scatter_overlap if missing"]
    G --> I["Append wgrad_name to\nlayers_all_gather_overlap if missing"]
Loading

Reviews (2): Last reviewed commit: "Improve readability of dgrad overlap var..." | Re-trigger Greptile

Comment thread transformer_engine/pytorch/module/base.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PRs from external contributor outside the core maintainers, representing community-driven work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve readability of dgrad_reduce_scatter_overlap variable name

1 participant