fix: Make the script analysis severity threshold fail the build

CHANGELOG.md

@@ -20,6 +20,24 @@
 
 ### Fixed
 
+- [**#96**](https://github.com/psake/PowerShellBuild/issues/96)
+  `Test-PSBuildScriptAnalysis` now fails the build when PSScriptAnalyzer
+  reports findings at or above the configured severity threshold. The
+  severity counts were computed from `$_Severity` — an undefined variable —
+  instead of `$_.Severity`, so all three counts were always zero and the
+  `Error`, `Warning`, and `Information` thresholds could never fail a build.
+  Script analysis reported its findings and the build passed regardless.
+  Because the default `FailBuildOnSeverityLevel` is `Error`, any consumer
+  running the `Analyze` task had a gate that only ever looked like it was
+  working. See the
+  [v0.8 → v1.0 migration guide](docs/migration-v0.8-to-v1.0.md) — a build
+  that passed before may now correctly fail.
+- [**#96**](https://github.com/psake/PowerShellBuild/issues/96)
+  `Test-PSBuildScriptAnalysis` no longer fails with a path-resolution error
+  when `SettingsPath` is not supplied. An unsupplied path was forwarded to
+  PSScriptAnalyzer as `-Settings ''`, which resolved against the current
+  directory and threw before any analysis ran, so the function's own
+  documented example could not run as written.
 - [**#102**](https://github.com/psake/PowerShellBuild/issues/102)
   `Test-PSBuildPester` no longer raises a parameter-binding error from its
   cleanup logic when the optional `ModuleName` parameter is not supplied.

PowerShellBuild/Public/Test-PSBuildScriptAnalysis.ps1

@@ -28,12 +28,27 @@
 
     Write-Verbose ($LocalizedData.SeverityThresholdSetTo -f $SeverityThreshold)
 
-    $analysisResult = Invoke-ScriptAnalyzer -Path $Path -Settings $SettingsPath -Recurse -Verbose:$VerbosePreference
-    $errors = ($analysisResult.where({ $_Severity -eq 'Error' })).Count
-    $warnings = ($analysisResult.where({ $_Severity -eq 'Warning' })).Count
-    $infos = ($analysisResult.where({ $_Severity -eq 'Information' })).Count
+    $invokeScriptAnalyzerParameters = @{
+        Path    = $Path
+        Recurse = $true
+    }
+    # An unsupplied SettingsPath must not be forwarded. PSScriptAnalyzer resolves an empty
+    # -Settings value against the current directory and fails before any analysis runs.
+    if (-not [string]::IsNullOrWhiteSpace($SettingsPath)) {
+        $invokeScriptAnalyzerParameters.Settings = $SettingsPath
+    }
+
+    $analysisResult = Invoke-ScriptAnalyzer @invokeScriptAnalyzerParameters -Verbose:$VerbosePreference
+
+    # A single diagnostic record comes back as a scalar rather than a collection, and Windows
+    # PowerShell 5.1 does not expose .Where() or .Count on every scalar type. Wrapping in @()
+    # guarantees collection semantics on both engines.
+    $analysisRecords = @($analysisResult)
+    $errorCount = ($analysisRecords.Where({ $_.Severity -eq 'Error' })).Count
+    $warningCount = ($analysisRecords.Where({ $_.Severity -eq 'Warning' })).Count
+    $informationCount = ($analysisRecords.Where({ $_.Severity -eq 'Information' })).Count
 
-    if ($analysisResult) {
+    if ($analysisRecords.Count -gt 0) {
         Write-Host $LocalizedData.PSScriptAnalyzerResults -ForegroundColor Yellow
         $analysisResult | Format-Table -AutoSize
     }
@@ -43,22 +58,22 @@
             return
         }
         'Error' {
-            if ($errors -gt 0) {
+            if ($errorCount -gt 0) {
                 throw $LocalizedData.ScriptAnalyzerErrors
             }
         }
         'Warning' {
-            if ($errors -gt 0 -or $warnings -gt 0) {
+            if ($errorCount -gt 0 -or $warningCount -gt 0) {
                 throw $LocalizedData.ScriptAnalyzerWarnings
             }
         }
         'Information' {
-            if ($errors -gt 0 -or $warnings -gt 0 -or $infos -gt 0) {
+            if ($errorCount -gt 0 -or $warningCount -gt 0 -or $informationCount -gt 0) {
                 throw $LocalizedData.ScriptAnalyzerWarnings
             }
         }
         default {
-            if ($analysisResult.Count -ne 0) {
+            if ($analysisRecords.Count -ne 0) {
                 throw $LocalizedData.ScriptAnalyzerIssues
             }
         }

docs/migration-v0.8-to-v1.0.md

@@ -10,8 +10,10 @@
 This guide helps you upgrade a consumer `build.ps1` (or equivalent) from
 PowerShellBuild **0.8.x** to **1.0.0**.
 
-It only covers **breaking changes**. For new features and bug fixes that
-do not require user action, see [`CHANGELOG.md`](../CHANGELOG.md).
+It covers **breaking changes**, plus any **behavioral change that can
+require action when you upgrade** — including bug fixes that make a
+previously passing build start failing. For new features and fixes that
+need nothing from you, see [`CHANGELOG.md`](../CHANGELOG.md).
 
 ## Quick Start
 
@@ -20,6 +22,9 @@ One line per break; follow the link for details and migration steps.
 - [Minimum supported PowerShell version is now 5.1](#minimum-supported-powershell-version-is-now-51)
   — the manifest requires PowerShell 5.1+; the support floor is
   Windows PowerShell 5.1 or PowerShell 7.4+.
+- [Script analysis now actually fails the build](#script-analysis-now-actually-fails-the-build)
+  — the `Analyze` task's severity threshold never fired in 0.8.x; a build
+  that passed before may now correctly fail.
 
 > More entries will follow as the Phase 2 migrations to
 > Microsoft.PowerShell.PlatyPS 1.x and psake 5.x land.
@@ -104,6 +109,53 @@ Tracked in PR
 record and platform validation details in
 [#120 (comment)](https://github.com/psake/PowerShellBuild/issues/120#issuecomment-5028978464).
 
+### Script analysis now actually fails the build
+
+This is a bug fix, but it is listed here because it can turn a build that
+passed on 0.8.x red on 1.0.0 without anything else changing.
+
+In 0.8.x, `Test-PSBuildScriptAnalysis` counted findings by severity using
+`$_Severity` — an undefined variable — rather than `$_.Severity`. Every
+count was therefore zero, and the `Error`, `Warning`, and `Information`
+thresholds could never fail a build. The `Analyze` task printed its
+findings and then passed. Since `FailBuildOnSeverityLevel` defaults to
+`Error`, this affected every consumer who ran the task: the gate reported
+problems but never enforced them.
+
+The counts are now correct, so the threshold you have configured is
+enforced for the first time.
+
+**No configuration change is required.** If your build starts failing at
+the `Analyze` task after upgrading, the analyzer findings it reports were
+present before too — they were simply never enforced. You have three
+options:
+
+1. Fix the reported findings (recommended — they are real).
+2. Exclude specific rules through your PSScriptAnalyzer settings file,
+   pointed to by `$PSBPreference.Test.ScriptAnalysis.SettingsPath`.
+3. Raise the bar or turn enforcement off:
+
+**Report findings without failing the build:**
+
+    $PSBPreference.Test.ScriptAnalysis.FailBuildOnSeverityLevel = 'None'
+
+**Fail only on errors, ignoring warnings:**
+
+    $PSBPreference.Test.ScriptAnalysis.FailBuildOnSeverityLevel = 'Error'
+
+To see what will be enforced before you upgrade, run PSScriptAnalyzer
+against your built module directly:
+
+    Invoke-ScriptAnalyzer -Path ./Output/MyModule/1.0.0 -Recurse |
+        Group-Object -Property Severity
+
+A related fix ships alongside it: omitting `-SettingsPath` no longer
+fails with a path-resolution error, so
+`Test-PSBuildScriptAnalysis -Path ./Output/MyModule/0.1.0 -SeverityThreshold Error`
+now runs as documented instead of throwing.
+
+Tracked in issue #96.
+
 ## Adding an entry (for PR contributors)
 
 Every breaking-change PR that lands in v1.0.0 must add an entry here for