CHORE: Add scheduled mssql-odbc pytest pipeline

[gargsaumya]

Work Item / Issue Reference

AB#48169

---

Summary

Adds a dedicated scheduled Azure Pipeline that validates mssql-python with MSSQL_PYTHON_NATIVE_PROVIDER=mssql-odbc once daily against the latest GitHub main.

• Installs the stable mssql-python-rs==0.1.0 runtime and verifies the resolved mssqlodbc payload.
• Connects to SQL Server and executes SELECT 1 as a blocking provider preflight.
• Runs every standard pytest file in an isolated process and publishes merged JUnit results.
• Reports functional failures, crashes, timeouts, and aggregate-budget exhaustion as SucceededWithIssues; setup, provider-preflight, harness, and publication failures remain blocking.
• Does not enable or claim Conda support for mssql-odbc.

Validation

• Pipeline YAML parsed and daily main schedule assertions passed.
• Bash syntax and Python syntax/Black checks passed.
• Linux regression tests cover clean pass, budget exhaustion, crash/malformed-JUnit handling, authenticated advisory status, and blocking harness failures.
• Stable mssql-python-rs-wheels==0.1.0 presence was verified in the configured public feed.

Dependency

Uses the released mssql-python-rs==0.1.0 distribution from the stable mssql-python-rs-wheels==0.1.0 NuGet transport. An Azure DevOps administrator must register the pipeline definition after the YAML reaches main.

[Copilot]

🟡 Changes recommended

Unresolved moderate findings affect failure classification, timeout headroom, setup handling, and JUnit validity.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a scheduled daily Azure Pipeline to validate mssql-python with the mssql-odbc provider.

Changes:

• Adds provider and SQL connectivity preflight checks.
• Runs isolated pytest files with timeout and JUnit reporting.
• Adds Docker setup, scheduling, result publication, and cleanup.

File summaries

File	Summary
eng/scripts/verify_mssql_odbc_provider.py	Validates provider selection, payload, and connectivity; no final comments.
eng/scripts/run-mssql-odbc-tests.sh	Requires changes for setup failure handling, pytest exit-code classification, timeout budgeting, and malformed JUnit protection.
eng/pipelines/mssql-odbc-daily-validation-pipeline.yml	Requires changes for job timeout headroom and blocking container setup failures.

Review details

Suppressed comments (4)

eng/pipelines/mssql-odbc-daily-validation-pipeline.yml:101

• This step does not enable errexit, so if docker inspect $(sqlContainer) fails (for example, the SQL container has exited), the command substitution leaves an empty IP and the script still invokes the test container. A resulting docker exec exit 1 is then treated as an advisory compatibility failure, allowing an infrastructure failure to finish as SucceededWithIssues; use set -euo pipefail here so host/setup errors remain blocking (the docker exec is already guarded by || rc=$?).

      set -uo pipefail

eng/scripts/run-mssql-odbc-tests.sh:15

• Because this script intentionally omits set -e, a failure to create the results directory is ignored. Subsequent pytest/report operations can then produce a nonzero result that is classified as advisory, so a harness setup failure does not remain blocking as described. Check this setup operation explicitly and exit 2 on failure.

mkdir -p "$RESULTS_DIR"

eng/scripts/run-mssql-odbc-tests.sh:136

• A timeout/crash can leave a non-empty but truncated JUnit XML file while pytest is writing it. The size-only check then skips write_stub, so PublishTestResults receives malformed XML instead of the required process-level result. Validate that the existing report is well-formed (or replace it with a stub) before keeping it.

    if [ ! -s "$report" ]; then
        write_stub "$name" error "Pytest exited $rc without producing JUnit" "$report"
    fi

eng/scripts/run-mssql-odbc-tests.sh:106

• The per-file slice and remaining calculations do not include --kill-after=60s. When a test reaches its timeout, this command can run up to another minute after the slice (including when the final slice is only 31 seconds), so the advertised 110-minute total budget can be exceeded and the job can lose time reserved for result publication and cleanup. Account for the kill grace period when selecting slice, or otherwise bound the termination grace within the total budget.

    echo "##[group]$test_file"
    timeout --kill-after=60s "${slice}s" \
        python -m pytest "$test_file" -v --junitxml="$report" \

• Files reviewed: 3/3 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 100%

🎯 Overall Coverage 83%

📈 Total Lines Covered: 8491 out of 10135 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 64.1%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 77.7%
mssql_python.pybind.connection.connection_pool.cpp: 81.8%
mssql_python.row.py: 83.4%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.py_type_cache.hpp: 91.6%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

🔵 Needs a closer look

Unresolved findings can misreport harness, skipped-test, crash/timeout, and result-generation failures.

Review details

Suppressed comments (6)

eng/pipelines/mssql-odbc-daily-validation-pipeline.yml:104

• This wrapper uses set -uo pipefail, so a failed docker inspect is not fatal. It leaves SQLSERVER_IP empty, still runs pytest with an invalid connection string, and the resulting exit 1 is then marked SucceededWithIssues; discovery of the SQL container/IP should remain a blocking harness failure.

      set -uo pipefail

eng/pipelines/mssql-odbc-daily-validation-pipeline.yml:118

• The raw docker exec status is passed directly into this case, but Docker can also return 1 for an execution/infrastructure error, not just for the runner's intentional pytest-failure status. A stopped container or failed exec can therefore be reported as an advisory compatibility failure instead of blocking; encode the runner's rc=1 separately (or verify a completion marker) before mapping it to SucceededWithIssues.

        ' || rc=$?

eng/scripts/run-mssql-odbc-tests.sh:140

• Checking only whether the XML file is nonempty loses crashes or timeouts that happen after pytest has written its JUnit file (for example, a native teardown crash). The script increments crashed/timed_out and returns advisory, but PublishTestResults then sees only the pre-crash results, so the process error is absent from the merged report. Add a separate error stub whenever a non-success, non-skipped process exit occurs, even if $report already exists.

    if [ ! -s "$report" ]; then
        write_stub "$name" error "Pytest exited $rc without producing JUnit" "$report"
    fi

eng/scripts/run-mssql-odbc-tests.sh:124

• This repository's pytest.ini adds -m "not stress", and test_011_singlethreaded_stress.py, test_020_multithreaded_stress.py, and test_021_concurrent_connection_perf.py contain only @pytest.mark.stress tests. Running those files individually therefore returns pytest exit code 5 on every healthy run; counting each as skipped makes the final skipped > 0 check return 1, so the pipeline is always marked SucceededWithIssues instead of passing when all standard (non-stress) tests pass. Exclude stress-only files from the per-file list or distinguish marker deselection from an unexpected empty test file without incrementing the advisory failure count.

        5)
            write_stub "$name" skipped "No tests collected" "$report"
            skipped=$((skipped + 1))

eng/scripts/run-mssql-odbc-tests.sh:140

• An empty/missing JUnit file is replaced with an error stub, but the counters are left unchanged. If pytest returns 0 while report creation fails (for example, the results directory is unwritable), passed remains incremented and the final status is 0, so this harness failure is incorrectly reported as a clean pass. Treat missing reports for non-crash/non-timeout exits as blocking.

    if [ ! -s "$report" ]; then
        write_stub "$name" error "Pytest exited $rc without producing JUnit" "$report"
    fi

eng/scripts/run-mssql-odbc-tests.sh:114

• pytest exits 0 when a file is collected but every test is skipped, including module-level skips. Counting every rc=0 file as passed means an all-skipped run can satisfy the check at lines 145-147 and be reported as a clean pass; use the generated JUnit counts (or another collected/runnable-test signal) before incrementing passed so skipped environments cannot masquerade as successful validation.

        0)
            passed=$((passed + 1))
            ;;

• Files reviewed: 3/3 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

Critical and moderate issues remain in dependency setup, failure handling, cleanup, and test-result classification.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

eng/pipelines/mssql-odbc-daily-validation-pipeline.yml:155

• The cleanup script does not enable errexit. If setup_sql_container.py --cleanup returns nonzero (for example, removal or ownership verification fails), docker rm -f ... || true becomes the final successful command and the always-run cleanup step reports success while the SQL container may remain. Propagate the helper's failure instead of masking it.

  - script: |
      python3 eng/scripts/setup_sql_container.py --cleanup \
        --name "$(sqlContainer)" --owner "$(Build.BuildId).$(System.JobId)"
      docker rm -f $(testContainer) || true

• Files reviewed: 3/3 changed files
• Comments generated: 4
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

Address the three moderate findings affecting preflight reliability, timeout reporting, and complete test execution.

Review details

Suppressed comments (3)

eng/pipelines/mssql-odbc-daily-validation-pipeline.yml:117

• The configured 110-minute total budget can stop the loop before all discovered test_*.py files run; when the budget is nearly exhausted, the remaining files only receive skipped JUnit stubs. That contradicts the stated guarantee that the pipeline runs every standard pytest file. Please either size/remove this cutoff so the complete file list executes, or explicitly document and surface this as a partial validation result.

        -e PYTEST_FILE_TIMEOUT=10m \
        -e PYTEST_TOTAL_BUDGET=110m \

eng/scripts/run-mssql-odbc-tests.sh:86

• When the total budget is exhausted after at least one file has run, this branch only writes skipped stubs and exits the loop; failed, crashed, and timed_out remain zero, so the final status check returns 0 and the pipeline reports a pass even though files were not executed. Count budget exhaustion as a timeout (or otherwise return the advisory failure status) before breaking.

    if [ "$remaining" -le 30 ]; then

eng/scripts/verify_mssql_odbc_provider.py:22

• The existing mssql-odbc smoke tests document that the Rust driver can panic during interpreter teardown after a successful query, so they accept a success marker in stdout instead of the subprocess exit code. This script relies on normal Python exit status; that known post-query panic can therefore make the blocking provider preflight fail even when SELECT 1 succeeded. Run the preflight in a child process and validate a success marker (as tests/test_026_odbc_provider.py does), while still failing when the marker is absent.

    with mssql_python.connect(os.environ["DB_CONNECTION_STRING"]) as connection:
        with connection.cursor() as cursor:
            cursor.execute("SELECT 1")
            assert cursor.fetchone()[0] == 1

• Files reviewed: 3/3 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The provider preflight script relies on Python assert (which can be optimized away) and the runner’s process-stub JUnit message can be misleading when no JUnit was actually produced.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

eng/scripts/verify_mssql_odbc_provider.py:15

• verify_mssql_odbc_provider.py uses assert for runtime validation of the provider selection and driver path. Running Python with optimizations (-O / PYTHONOPTIMIZE) strips asserts, which would let the preflight silently pass even if the provider/driver is wrong. Prefer explicit checks that raise an exception with the same diagnostic payload.
  eng/scripts/verify_mssql_odbc_provider.py:27
• The preflight query and post-load validation also rely on assert (e.g., assert cursor.fetchone()[0] == 1, assert loaded_info["frozen"] is True). If asserts are disabled, the script could print the success marker even when the query/provider-freeze checks fail. Use explicit checks and raise a clear exception instead.

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Lite

[github-actions]

PR Performance Report

No consistent slowdowns detected across all 4 environments.

Coverage: 4 of 4 environments completed. Advisory result; does not block merging.

Environment	Status
Windows / SQL Server 2022	Completed
Windows / SQL Server 2025	Completed
Unix / SQL Server 2022	Completed
Unix / SQL Server 2025	Completed

Affected phases and call counts

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Windows / SQL Server 2022

Database task	Before	After	Paired change	Result
Connection opening	21.087 ms	21.117 ms	-1.7%	no signal
SELECT queries	1.537 ms	1.528 ms	-2.6%	no signal
Row insertion	30.178 ms	29.642 ms	-1.2%	no signal
Executemany inserts	186.620 ms	224.123 ms	+0.4%	no signal
Fetch-all queries	268.406 ms	268.863 ms	-0.1%	no signal
Row-by-row fetching	30.638 ms	30.756 ms	+1.0%	no signal
Batched row fetching	267.197 ms	272.518 ms	-1.2%	no signal
Transaction commit and rollback	86.638 ms	89.301 ms	+2.1%	no signal
Arrow row fetching	173.052 ms	173.702 ms	+0.3%	no signal
100,000-row insertion	616.834 ms	972.266 ms	+4.0%	no signal
Row fetching in batches of 100	300.197 ms	323.512 ms	+3.9%	no signal
Row fetching in batches of 10,000	286.287 ms	285.043 ms	-3.1%	no signal
Repeated positional queries	30.237 ms	29.571 ms	-1.5%	no signal
Repeated named-parameter queries	32.741 ms	32.350 ms	-1.4%	no signal
Legacy 100,000-row insertion	526.443 ms	492.319 ms	+1.6%	no signal
Insertion with explicit input sizes	3303.269 ms	3335.969 ms	+1.0%	no signal
Joined aggregation queries	186.054 ms	183.654 ms	+0.1%	no signal
Large joined-result fetching	314.128 ms	318.479 ms	+4.4%	no signal
1.2-million-row fetching	7685.823 ms	7682.298 ms	+0.8%	no signal
Common table expression queries	5.306 ms	5.484 ms	+3.4%	no signal

Windows / SQL Server 2025

Database task	Before	After	Paired change	Result
Connection opening	173.428 ms	174.622 ms	+15.9%	no signal
SELECT queries	1.564 ms	1.598 ms	-3.0%	no signal
Row insertion	30.952 ms	31.066 ms	-4.9%	no signal
Executemany inserts	210.821 ms	227.013 ms	+16.9%	no signal
Fetch-all queries	260.031 ms	275.214 ms	+2.8%	no signal
Row-by-row fetching	29.616 ms	31.340 ms	+5.8%	no signal
Batched row fetching	294.309 ms	295.154 ms	+2.1%	no signal
Transaction commit and rollback	98.168 ms	97.876 ms	-14.7%	no signal
Arrow row fetching	163.024 ms	161.358 ms	+0.5%	no signal
100,000-row insertion	605.002 ms	623.296 ms	-2.8%	no signal
Row fetching in batches of 100	315.721 ms	300.565 ms	-5.5%	no signal
Row fetching in batches of 10,000	286.105 ms	292.936 ms	+2.6%	no signal
Repeated positional queries	30.728 ms	31.328 ms	+2.6%	no signal
Repeated named-parameter queries	33.759 ms	34.216 ms	+2.0%	no signal
Legacy 100,000-row insertion	506.896 ms	505.255 ms	+2.1%	no signal
Insertion with explicit input sizes	3616.203 ms	3270.174 ms	-6.4%	no signal
Joined aggregation queries	190.060 ms	169.250 ms	-0.7%	no signal
Large joined-result fetching	310.500 ms	325.977 ms	+0.2%	no signal
1.2-million-row fetching	7674.867 ms	7765.253 ms	+2.0%	no signal
Common table expression queries	5.154 ms	5.355 ms	+2.4%	no signal

Unix / SQL Server 2022

Database task	Before	After	Paired change	Result
Connection opening	11.035 ms	10.817 ms	+0.6%	no signal
SELECT queries	1.193 ms	1.199 ms	-1.4%	no signal
Row insertion	35.019 ms	34.851 ms	-1.1%	no signal
Executemany inserts	160.934 ms	158.585 ms	-2.3%	no signal
Fetch-all queries	172.887 ms	171.967 ms	+0.5%	no signal
Row-by-row fetching	59.758 ms	60.499 ms	+1.3%	no signal
Batched row fetching	163.896 ms	165.457 ms	+0.3%	no signal
Transaction commit and rollback	115.897 ms	116.545 ms	-0.5%	no signal
Arrow row fetching	94.700 ms	95.967 ms	+5.1%	no signal
100,000-row insertion	477.967 ms	447.629 ms	-8.6%	no signal
Row fetching in batches of 100	221.252 ms	224.303 ms	+1.6%	no signal
Row fetching in batches of 10,000	192.195 ms	184.968 ms	-2.6%	no signal
Repeated positional queries	42.646 ms	42.898 ms	+0.6%	no signal
Repeated named-parameter queries	45.051 ms	44.972 ms	-0.2%	no signal
Legacy 100,000-row insertion	356.478 ms	350.399 ms	-2.4%	no signal
Insertion with explicit input sizes	2412.616 ms	2369.721 ms	-1.3%	no signal
Joined aggregation queries	181.178 ms	181.003 ms	+0.2%	no signal
Large joined-result fetching	209.481 ms	214.404 ms	+3.0%	no signal
1.2-million-row fetching	5023.938 ms	5047.266 ms	+0.7%	no signal
Common table expression queries	5.364 ms	5.421 ms	+1.2%	no signal

Unix / SQL Server 2025

Database task	Before	After	Paired change	Result
Connection opening	97.301 ms	98.243 ms	+0.0%	no signal
SELECT queries	1.193 ms	1.267 ms	+1.2%	no signal
Row insertion	34.672 ms	34.862 ms	+0.7%	no signal
Executemany inserts	153.910 ms	153.414 ms	+0.3%	no signal
Fetch-all queries	172.341 ms	174.938 ms	+0.1%	no signal
Row-by-row fetching	63.603 ms	61.565 ms	-3.2%	no signal
Batched row fetching	166.244 ms	165.743 ms	+0.5%	no signal
Transaction commit and rollback	117.258 ms	115.520 ms	-0.8%	no signal
Arrow row fetching	95.869 ms	94.388 ms	-1.1%	no signal
100,000-row insertion	450.677 ms	465.119 ms	+2.6%	no signal
Row fetching in batches of 100	223.336 ms	223.625 ms	+0.5%	no signal
Row fetching in batches of 10,000	184.488 ms	201.648 ms	+9.1%	no signal
Repeated positional queries	43.163 ms	42.737 ms	+0.1%	no signal
Repeated named-parameter queries	45.555 ms	45.309 ms	-0.3%	no signal
Legacy 100,000-row insertion	372.967 ms	353.548 ms	-4.2%	no signal
Insertion with explicit input sizes	2443.332 ms	2463.965 ms	-0.3%	no signal
Joined aggregation queries	162.086 ms	163.586 ms	+1.0%	no signal
Large joined-result fetching	215.047 ms	212.398 ms	-0.9%	no signal
1.2-million-row fetching	5085.736 ms	5090.086 ms	-0.4%	no signal
Common table expression queries	5.250 ms	5.177 ms	-1.3%	no signal

Build, commits and measurement details

ADO build 176486

PR head: f7b00ceec0f22fdcce43f8d35456b2ec3867dbc6
Base: c7ed6f8e7a2dec212662a9233e372f2d537787dd
Measured merge: 74d323b282701dea2553e4c37ff33eba4976366f

• Windows / SQL Server 2022: Python 3.13.15, amd64, SQL 16.0.1000.6; 5 paired comparisons and 1 warmup.
• Windows / SQL Server 2025: Python 3.14.7, amd64, SQL 17.0.1000.7; 5 paired comparisons and 1 warmup.
• Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4295.3; 5 paired comparisons and 1 warmup.
• Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5005.3; 5 paired comparisons and 1 warmup.

A consistent slowdown requires more than 20% median paired slowdown, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold. An inconsistent slowdown crosses the first two thresholds without enough pair agreement.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

[Copilot]

🟡 Changes recommended

The new contract tests and preflight script have concrete robustness gaps that can cause cross-platform CI failures and/or obscure diagnostics in common failure modes.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 4
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The pipeline, runner, and preflight changes are internally consistent, match existing provider-selection contracts, and include targeted contract tests to prevent regressions.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The changes are cohesive and validated with tests/pipeline contracts, and the remaining feedback is limited to maintainability polish rather than functional correctness.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The new preflight rejection unit test can pass for the wrong reason (later import/attribute failure), so it should assert the expected failure mode and guard against accidental connect() execution.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The provider preflight step should validate SQL Server IP resolution (like the later pytest step) to fail closed with a clear infrastructure error instead of attempting to connect with an empty Server= value.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 1
• Review effort level: Lite