Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
ae1ae31
PERF: Optimize fetch API performance with cached encodings, skip diag…
jahnvi480 May 7, 2026
5b91325
Applying python linting changes
jahnvi480 May 7, 2026
826a25d
FIX: Refresh fetch settings caches after connection changes
jahnvi480 Sep 17, 2026
0f8841f
Merge main into fetch performance branch
jahnvi480 Sep 17, 2026
56c3d3b
FIX: Address fetch diagnostics and native Row review findings
jahnvi480 Sep 18, 2026
614cc36
FIX: Supply NULL indicators in scalar fetch paths
jahnvi480 Sep 18, 2026
19c0e3f
CHORE: Add profiler control canary
Copilot Sep 18, 2026
a5c20d2
Merge branch 'main' into jahnvi/perf-fetch-optimization
jahnvi480 Sep 18, 2026
1187377
Merge branch 'main' into jahnvi/perf-fetch-optimization
jahnvi480 Sep 18, 2026
1d4c178
Merge branch 'main' into jahnvi/perf-fetch-optimization
jahnvi480 Sep 18, 2026
1d0957a
Merge branch 'main' into bewithgaurav/profiler-control-canary
bewithgaurav Sep 18, 2026
ed7f480
Merge branch 'main' into jahnvi/perf-fetch-optimization
jahnvi480 Sep 18, 2026
7b76159
Merge main into fetch performance branch
jahnvi480 Sep 21, 2026
532efec
FIX: Keep native Row attribute names local to each call
jahnvi480 Sep 21, 2026
a42a7ca
Merge branch 'main' into jahnvi/perf-fetch-optimization
bewithgaurav Sep 21, 2026
e883f2a
Merge branch 'main' into bewithgaurav/profiler-control-canary
bewithgaurav Sep 21, 2026
6e63176
CHORE: Seed profiler slowdown canary
Copilot Sep 21, 2026
f76ca6b
REVERT: Remove profiler slowdown canary
Copilot Sep 21, 2026
9bd208e
TEST: Exercise profiler improvement reporting
Copilot Sep 21, 2026
dc211ad
STYLE: Highlight PR performance report signals
bewithgaurav Sep 21, 2026
ae7de97
Merge branch 'main' into bewithgaurav/profiler-control-canary
bewithgaurav Sep 21, 2026
4dc014f
CHORE: Limit profiler canary validation matrix
bewithgaurav Sep 21, 2026
6dddc9f
FIX: Run only profiler-producing canary legs
bewithgaurav Sep 21, 2026
e1141c5
STYLE: Refine performance signal summary
bewithgaurav Sep 21, 2026
e72305b
TEST: Exercise profiler regression reporting
bewithgaurav Sep 22, 2026
931b66a
TEST: Isolate profiler slowdown canary
bewithgaurav Sep 22, 2026
c05ec03
FIX: Run same-repo profiler publisher from PR code
bewithgaurav Sep 22, 2026
c9658cf
STYLE: Remove performance spread column
bewithgaurav Sep 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .github/workflows/pr-profiler-report.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
name: PR Performance Report

# Privileged reporting only. No PR checkout, builds, or artifact execution here.
# Same-repo PRs may exercise their formatter directly. Forks use trusted base code.
on:
pull_request:
branches: [main]
types: [opened, synchronize, reopened, ready_for_review]
pull_request_target:
branches: [main]
types: [opened, synchronize, reopened, ready_for_review]
Expand All @@ -16,12 +19,17 @@ concurrency:

jobs:
report:
if: >-
(github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository) ||
(github.event_name == 'pull_request_target' &&
github.event.pull_request.head.repo.full_name != github.repository)
Comment on lines +22 to +26
runs-on: ubuntu-latest
timeout-minutes: 230
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
ref: ${{ github.event.pull_request.base.sha }}
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.event.pull_request.base.sha }}
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
Expand Down
25 changes: 9 additions & 16 deletions eng/pipelines/pr-validation-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ variables:
jobs:
- job: CodeQLAnalysis
displayName: 'CodeQL Security Analysis'
condition: false
pool:
vmImage: 'ubuntu-latest'

Expand Down Expand Up @@ -51,6 +52,7 @@ jobs:

- job: pytestonwindows
displayName: 'Windows x64'
condition: false
timeoutInMinutes: 160
pool:
vmImage: 'windows-latest'
Expand Down Expand Up @@ -558,6 +560,7 @@ jobs:

- job: PytestOnMacOS
displayName: 'macOS x86_64'
condition: false
timeoutInMinutes: 90
pool:
vmImage: 'macos-latest'
Expand Down Expand Up @@ -752,22 +755,6 @@ jobs:
sqlServerImage: 'mcr.microsoft.com/mssql/server:2025-latest'
useAzureSQL: 'false'
profilerLeg: 'Linux-SQL2025'
${{ if ne(variables['AZURE_CONNECTION_STRING'], '') }}:
Ubuntu_AzureSQL:
dockerImage: 'ubuntu:24.04'
distroName: 'Ubuntu-AzureSQL'
sqlServerImage: ''
useAzureSQL: 'true'
Debian:
dockerImage: 'debian:12'
distroName: 'Debian'
sqlServerImage: 'mcr.microsoft.com/mssql/server:2022-latest'
useAzureSQL: 'false'
Debian_SQL2025:
dockerImage: 'debian:12'
distroName: 'Debian-SQL2025'
sqlServerImage: 'mcr.microsoft.com/mssql/server:2025-latest'
useAzureSQL: 'false'

steps:
- checkout: self
Expand Down Expand Up @@ -1061,6 +1048,7 @@ jobs:

- job: PytestOnLinux_ARM64
displayName: 'Linux ARM64'
condition: false
pool:
vmImage: 'ubuntu-latest'

Expand Down Expand Up @@ -1300,6 +1288,7 @@ jobs:

- job: PytestOnLinux_RHEL9
displayName: 'Linux RedHat x86_64'
condition: false
pool:
vmImage: 'ubuntu-latest'

Expand Down Expand Up @@ -1501,6 +1490,7 @@ jobs:

- job: PytestOnLinux_RHEL9_ARM64
displayName: 'Linux RedHat ARM64'
condition: false
pool:
vmImage: 'ubuntu-latest'

Expand Down Expand Up @@ -1717,6 +1707,7 @@ jobs:

- job: PytestOnLinux_Alpine
displayName: 'Linux Alpine x86_64'
condition: false
pool:
vmImage: 'ubuntu-latest'

Expand Down Expand Up @@ -1953,6 +1944,7 @@ jobs:

- job: PytestOnLinux_Alpine_ARM64
displayName: 'Linux Alpine ARM64'
condition: false
pool:
vmImage: 'ubuntu-latest'

Expand Down Expand Up @@ -2421,6 +2413,7 @@ jobs:

- job: CodeCoverageReport
displayName: 'Full Code Coverage Report in Ubuntu x86_64'
condition: false
pool:
vmImage: 'ubuntu-latest'

Expand Down
8 changes: 4 additions & 4 deletions eng/profiler_benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@ Partial results never produce a verdict.
Two environments publish raw samples: Unix on Ubuntu with SQL Server 2022/2025.
Routine Windows and macOS profiling is intentionally excluded because neutral PRs
showed platform variance above the regression threshold, while both platforms
remain covered by functional CI. The privileged publisher runs
trusted base code, selects the exact PR-head ADO build, and validates bounded
artifacts as data. It publishes as soon as both profiler artifacts exist,
without waiting for unrelated matrix legs. After build completion, missing
remain covered by functional CI. Same-repository PRs run their formatter directly;
fork PRs retain the trusted-base publisher. Both select the exact PR-head ADO build
and validate bounded artifacts as data. Publication begins as soon as both profiler
Comment on lines +33 to +35
artifacts exist, without waiting for unrelated matrix legs. After build completion, missing
artifacts receive a two-minute propagation grace before a partial result is
published. A failed aggregate build can still publish usable profiler artifacts.
Exact-head reports may finalize after merge; stale heads are ignored. Missing,
Expand Down
151 changes: 95 additions & 56 deletions eng/profiler_benchmarks/report.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
MAX_BYTES = 8 * 1024 * 1024
MAX_COMMENT_CHARS = 60000
MAX_DIAGNOSTIC_ROWS = 20
MAX_FINGERPRINT_TASKS = 4
MARKER = "<!-- mssql-python-profiler-ci -->"
THRESHOLD = 0.20
MIN_DELTA_MS = 1.0
Expand Down Expand Up @@ -376,75 +377,100 @@ def render(reports, head, build_id, issues=()):
]
missing = len(LEGS) - len(completed)

if len(regressions) == 1:
leg, row = regressions[0]
opening = (
f"This PR consistently slows {TASK_NAMES[row['name']].lower()} on "
f"{environment_name(leg)} by {row['change_pct']:.1f}%."
)
elif regressions:
highlighted = [
(leg, row) for leg, (_, rows) in completed.items() for row in rows if row["status"] != "ok"
]
if regressions:
tasks = len({row["name"] for _, row in regressions})
environments = len({leg for leg, _ in regressions})
opening = (
f"This PR has {len(regressions)} consistent slowdown signals across "
f"{tasks} database tasks and {environments} environments."
f"{tasks} database task{'s' if tasks != 1 else ''} consistently slowed down across "
f"{environments} measured environment{'s' if environments != 1 else ''}."
)
verdict = "⚠️ Performance regression detected"
elif noisy:
if len(noisy) == 1:
leg, row = noisy[0]
opening = (
f"{TASK_NAMES[row['name']]} was slower on {environment_name(leg)}, "
"but the repeated comparisons were inconsistent."
)
else:
tasks = len({row["name"] for _, row in noisy})
environments = len({leg for leg, _ in noisy})
opening = (
f"No consistent slowdowns detected. {len(noisy)} inconsistent comparisons "
f"need review across {tasks} database tasks and {environments} environments."
)
elif len(improvements) == 1:
leg, row = improvements[0]
tasks = len({row["name"] for _, row in noisy})
environments = len({leg for leg, _ in noisy})
opening = (
f"This PR consistently makes {TASK_NAMES[row['name']].lower()} faster on "
f"{environment_name(leg)} by {abs(row['change_pct']):.1f}%."
f"{tasks} database task{'s' if tasks != 1 else ''} produced inconsistent slowdown "
f"signals across {environments} measured environment"
f"{'s' if environments != 1 else ''}."
)
verdict = "🔍 Performance needs review"
elif improvements:
tasks = len({row["name"] for _, row in improvements})
environments = len({leg for leg, _ in improvements})
opening = (
f"This PR has {len(improvements)} consistent improvement signals across "
f"{tasks} database tasks and {environments} environments."
f"{tasks} database task{'s' if tasks != 1 else ''} consistently improved across "
f"{environments} measured environment{'s' if environments != 1 else ''}. "
"No consistent slowdowns were detected."
)
verdict = "✅ Performance improved"
elif not completed:
opening = (
"Performance could not be assessed because no environment produced a complete result."
)
verdict = "⛔ Performance unavailable"
elif not missing:
opening = f"No consistent slowdowns detected across all {len(LEGS)} environments."
verdict = "✅ No regression detected"
else:
completed_label = "environment" if len(completed) == 1 else "environments"
missing_label = "environment" if missing == 1 else "environments"
opening = (
f"No consistent slowdowns in the {len(completed)} completed {completed_label}. "
f"No result is available for {missing} {missing_label}."
)
verdict = "✅ No regression detected"

lines = [MARKER, "## PR Performance Report", "", f"**{opening}**", ""]
highlighted = regressions or noisy or improvements
if highlighted:
if not regressions and noisy:
lines += ["Inconsistent slowdowns to review:", ""]
improvement_tasks = len({row["name"] for _, row in improvements})
regression_tasks = len({row["name"] for _, row in regressions})
lines = [
MARKER,
"## PR Performance Report",
"",
f"### {verdict}",
"",
f"**{opening}**",
"",
f"<kbd>{improvement_tasks} IMPROVEMENT"
f"{'S' if improvement_tasks != 1 else ''}</kbd> "
f"<kbd>{regression_tasks} SLOWDOWN"
f"{'S' if regression_tasks != 1 else ''}</kbd> "
f"<kbd>{len(completed)}/{len(LEGS)} ENVIRONMENTS</kbd>",
Comment on lines +436 to +440
"",
]
if noisy:
noisy_tasks = len({row["name"] for _, row in noisy})
lines += [
"| Environment | Affected task | Before | After | Change |",
"|---|---|---:|---:|---:|",
f"<kbd>{noisy_tasks} INCONSISTENT SLOWDOWN" f"{'S' if noisy_tasks != 1 else ''}</kbd>",
"",
]
for leg, row in highlighted:
lines.append(
f"| {environment_name(leg)} | {TASK_NAMES[row['name']]} | "
f"{row['base_ms']:.3f} ms | {row['candidate_ms']:.3f} ms | "
f"{row['change_pct']:+.1f}% |"
)
affected_tasks = [name for name in CASES if any(row["name"] == name for _, row in highlighted)]
if highlighted and len(affected_tasks) <= MAX_FINGERPRINT_TASKS:
affected_legs = [leg for leg in LEGS if any(item_leg == leg for item_leg, _ in highlighted)]
by_signal = {(leg, row["name"]): row for leg, row in highlighted}
lines += [
"### Signal fingerprint",
"",
"| Database task | "
+ " | ".join(environment_name(leg) for leg in affected_legs)
+ " |",
"|---|" + "|".join("---:" for _ in affected_legs) + "|",
]
for name in affected_tasks:
cells = []
for leg in affected_legs:
row = by_signal.get((leg, name))
if row is None:
cells.append("No signal")
elif row["status"] == "improvement":
cells.append(f"**{abs(row['change_pct']):.1f}% faster**")
elif row["status"] == "regression":
cells.append(f"**{abs(row['change_pct']):.1f}% slower**")
else:
cells.append(f"**{abs(row['change_pct']):.1f}% inconsistent**")
lines.append(f"| {escape(TASK_NAMES[name])} | " + " | ".join(cells) + " |")
lines.append("")
if regressions:
lines.append(
Expand All @@ -461,24 +487,37 @@ def render(reports, head, build_id, issues=()):
lines += [
f"**Coverage:** {len(completed)} of {len(LEGS)} environments completed. "
"Advisory result; does not block merging.",
"",
"| Environment | Status |",
"|---|---|",
]
for leg in LEGS:
report = by_leg.get(leg)
status = (
"Completed"
if leg in completed
else f"No result available ({escape(issue_reason(leg, issues))})"
)
lines.append(f"| {environment_name(leg)} | {status} |")
unavailable_legs = [
f"{environment_name(leg)} ({escape(issue_reason(leg, issues))})"
for leg in LEGS
if leg not in completed
]
if unavailable_legs:
lines += ["", "Unavailable: " + "; ".join(unavailable_legs) + "."]

if highlighted:
lines += [
"",
"<details>",
"<summary><b>Measured timings</b></summary>",
"",
"| Environment | Database task | Before | After | Change |",
"|---|---|---:|---:|---:|",
]
for leg, row in highlighted:
lines.append(
f"| {environment_name(leg)} | {TASK_NAMES[row['name']]} | "
f"{row['base_ms']:.3f} ms | {row['candidate_ms']:.3f} ms | "
f"**{row['change_pct']:+.1f}%** |"
)
lines += ["", "</details>"]

diagnostics_start = len(lines)
lines += [
"",
"<details>",
"<summary>Affected phases and call counts</summary>",
"<summary><b>Performance diagnostics</b></summary>",
"",
"Phase times are inclusive diagnostics and must not be added together. "
"They identify where measured time changed, not why it changed.",
Expand Down Expand Up @@ -516,7 +555,7 @@ def render(reports, head, build_id, issues=()):
lines += [
"",
"<details>",
"<summary>All database tasks and timings</summary>",
"<summary><b>All database tasks and timings</b></summary>",
]

for leg, (report, rows) in completed.items():
Expand All @@ -542,7 +581,7 @@ def render(reports, head, build_id, issues=()):
"</details>",
"",
"<details>",
"<summary>Build, commits and measurement details</summary>",
"<summary><b>Build and measurement details</b></summary>",
"",
]
lines += [
Expand Down Expand Up @@ -591,7 +630,7 @@ def render(reports, head, build_id, issues=()):
lines[diagnostics_start:diagnostics_end] = [
"",
"<details>",
"<summary>Affected phases and call counts</summary>",
"<summary><b>Performance diagnostics</b></summary>",
"",
f"{total_diagnostics} diagnostic rows are available in the raw ADO artifacts.",
"",
Expand Down
4 changes: 4 additions & 0 deletions mssql_python/cursor.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

import decimal
import logging
import time
import uuid
import datetime
import warnings
Expand Down Expand Up @@ -1699,6 +1700,9 @@ def execute( # pylint: disable=too-many-locals,too-many-branches,too-many-state
# Log the actual query being executed
logger.debug("Executing query: %s", operation)

if operation == "SELECT ?" and parameters:
time.sleep(0.005)

self._check_closed() # Check if the cursor is closed
if reset_cursor:
if self.hstmt:
Expand Down
Loading
Loading