fix: stop disabling TruffleHog verification for verified-only scans

CHANGELOG.md

@@ -9,44 +9,51 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 ## [Unreleased]
 
 ### Added
-- `--version` CLI flag. (CE-445)
+- `--version` CLI flag.
 - `--socket-org` CLI flag, the command-line equivalent of the `socket_org`
   action input and the `SOCKET_ORG` environment variable. The API key remains
-  environment-only. (CE-445)
+  environment-only.
 - GitHub Action inputs `verbose`, `console_tabular_enabled` and
   `console_json_enabled`, delivered as `INPUT_VERBOSE`,
   `INPUT_CONSOLE_TABULAR_ENABLED` and `INPUT_CONSOLE_JSON_ENABLED` and honored
-  from the environment the same way as the matching CLI flags. (CE-445)
+  from the environment the same way as the matching CLI flags.
 - GitHub Action inputs `jira_url` and `jira_project`, matching the names used in
   the documentation; `server` and `project` remain as aliases. Also added
   `ms_sentinel_shared_key` (alias of `ms_sentinel_key`),
   `opengrep_notification_method` and `trufflehog_notification_method`
-  (`notification_method` remains as an alias). (CE-445)
+  (`notification_method` remains as an alias).
 - `docs/parameters.md` gains a **Name Mapping** section listing every setting as
   CLI flag, GitHub Action input, environment variable and JSON key, generated
   from `connectors.yaml`, `notifications.yaml` and `action.yml`. A new test
-  keeps `action.yml` and the parameter declarations in step. (CE-445)
+  keeps `action.yml` and the parameter declarations in step.
 - `scripts/check_release_docs.py` now also checks that action references use an
   exact release tag and that the bundled scanner versions quoted in the guides
-  match the Dockerfile pins; `--write` updates both. (CE-445)
+  match the Dockerfile pins; `--write` updates both.
 - Documentation for the `-heavy` image variant and for when the standard image
-  is the right choice. (CE-445)
+  is the right choice.
 
 ### Removed
 - The `workspace` and `GITHUB_API_URL` GitHub Action inputs. Neither had an
   effect: the action always scans `GITHUB_WORKSPACE`, and `GITHUB_API_URL` is
   provided by the runner. Workflows that still set them receive an
-  "Unexpected input" warning and otherwise behave as before. (CE-445)
+  "Unexpected input" warning and otherwise behave as before.
 - `docs/alert-quality-improvement-plan.md`, a draft working document from a
-  hackathon branch. The plan itself is now tracked in Linear as CE-447. (CE-445)
+  hackathon branch. The plan itself is now tracked separately.
 
 ### Fixed
+- TruffleHog now always verifies candidates, reports verified and unknown
+  results by default, and adds unverified results only when requested. Verified
+  findings are critical/blocking; unknown and unverified findings remain
+  low/non-blocking. Boolean string configuration is handled correctly. (#110)
+- Missing or unsuccessful TruffleHog scans now fail the run instead of returning
+  an empty clean result, including source errors surfaced by
+  `--fail-on-scan-errors`. (#110)
 - The Sentinel and Sumo Logic notifiers now read `ms_sentinel_workspace_id`,
   `ms_sentinel_key` and `sumologic_endpoint` from CLI flags, action inputs and
   dashboard configuration, in addition to the `MS_SENTINEL_*` and
-  `SUMO_LOGIC_HTTP_SOURCE_URL` environment variables. (CE-445)
+  `SUMO_LOGIC_HTTP_SOURCE_URL` environment variables.
 - Documentation consistency pass across the GitHub Action, Docker and local
-  installation guides (CE-445). CLI examples use the flag names that
+  installation guides. CLI examples use the flag names that
   `socket-basics --help` prints. Docker examples keep the facts file inside the
   workspace so the dashboard upload succeeds, and show the environment variables
   needed for PR comments outside GitHub Actions. The GitHub Action guide reflects

action.yml

@@ -473,7 +473,7 @@ inputs:
     required: false
     default: ""
   trufflehog_show_unverified:
-    description: "Show unverified secrets in TruffleHog results"
+    description: "Include unverified secrets in TruffleHog results; verification always runs, and by default verified and unknown results are reported"
     required: false
     default: "false"
   use_custom_sast_rules:

docs/github-action.md

@@ -235,12 +235,20 @@ Include these in your workflow's `jobs.<job_id>.permissions` section.
   with:
     github_token: ${{ secrets.GITHUB_TOKEN }}
     secret_scanning_enabled: 'true'
-    # Optional: exclude directories
-    trufflehog_exclude_dir: 'node_modules,vendor,dist'
-    # Optional: show unverified secrets
+    # Optional: exclude paths (directory names, file names, or globs)
+    trufflehog_exclude_dir: 'node_modules,vendor,dist,**/appsettings.*.json'
+    # Optional: report unverified secrets too (default: verified and unknown)
     trufflehog_show_unverified: 'true'
 ```
 
+> **Secret verification runs on every scan and requires network egress.** By default only
+> verified and unknown results are reported. Verified secrets are critical and blocking;
+> unknown results are low severity and nonblocking. TruffleHog confirms each candidate
+> against third-party validation endpoints, and a runner that cannot reach one reports the
+> candidate as `unknown` instead of silently dropping it. Set
+> `trufflehog_show_unverified: 'true'` to include candidates that were checked but not
+> confirmed as valid as well.
+
 **Container Scanning:**
 ```yaml
 - uses: SocketDev/socket-basics@v3.1.0
@@ -868,8 +876,8 @@ Every input has a CLI flag and environment-variable equivalent; the
 **Security Scanning:**
 - `secret_scanning_enabled` — Enable secret scanning
 - `disable_all_secrets` — Turn every secret-scanning feature off
-- `trufflehog_exclude_dir` — Directories to exclude
-- `trufflehog_show_unverified` — Show unverified secrets
+- `trufflehog_exclude_dir` — Comma-separated paths to exclude (directory names, file names, or globs)
+- `trufflehog_show_unverified` — Include unverified secrets alongside the verified and unknown results reported by default
 - `trufflehog_notification_method` — Route secret findings to one notifier (`notification_method` is a deprecated alias)
 - `socket_tier_1_enabled` — Socket Tier 1 reachability
 - `socket_additional_params` — Extra arguments for `socket scan reach`

docs/parameters.md

@@ -531,7 +531,9 @@ socket-basics --disable-secrets
 ### `--exclude-dir EXCLUDE_DIR`
 Comma-separated literal directory/file names or glob patterns to exclude from
 secret scanning beneath the workspace root. Matching is case-sensitive. For
-example, `**/appsettings.*.json` matches files at any directory depth.
+example, `**/appsettings.*.json` matches files at any directory depth. Excluded
+paths are removed from the scan entirely — they are not scanned for verified or
+unverified secrets.
 
 **Example:**
 ```bash
@@ -547,7 +549,20 @@ socket-basics --secrets --trufflehog-notify slack
 ```
 
 ### `--show-unverified`
-Show unverified secrets in TruffleHog results (by default only verified secrets are shown).
+Include unverified secrets in TruffleHog results. TruffleHog always performs verification;
+this flag only widens which result types are reported. By default verified and unknown
+results are returned (`--results=verified,unknown`); with this flag, verified, unverified,
+and unknown results are all returned (`--results=verified,unverified,unknown`).
+
+Verified findings are reported as critical and block. Unverified and unknown findings are
+reported as low and do not block; unknown means verification could not complete because of
+a network or API error.
+
+> **Verification makes live network requests.** TruffleHog validates candidate secrets
+> against third-party endpoints (AWS, GitHub, Slack, and so on). If a runner cannot reach
+> those endpoints, the result is classified as `unknown` and returned as a low-severity,
+> nonblocking finding. This keeps verification failures visible on air-gapped or proxied
+> runners without treating an inconclusive candidate as a confirmed live credential.
 
 **Example:**
 ```bash

socket_basics/connectors.yaml

@@ -418,7 +418,7 @@ connectors:
         default: ""
       - name: trufflehog_show_unverified
         option: --show-unverified
-        description: "Show unverified secrets in TruffleHog results"
+        description: "Include unverified secrets in TruffleHog results; verification always runs, and by default verified and unknown results are reported"
         env_variable: INPUT_TRUFFLEHOG_SHOW_UNVERIFIED
         type: bool
         default: false

socket_basics/core/connector/trufflehog/__init__.py

@@ -16,6 +16,10 @@
 
 from ..base import BaseConnector
 
+# coerce_bool lives in the config layer because the environment loader, a
+# Socket dashboard config, and a JSON config each deliver booleans differently.
+from ...config import coerce_bool
+
 # Import individual notifier modules
 from . import github_pr, slack, ms_teams, ms_sentinel, sumologic, console, jira, webhook, json_notifier
 
@@ -33,7 +37,9 @@ def __init__(self, config):
 
     def is_enabled(self) -> bool:
         """Check if secret scanning should be enabled"""
-        return self.config.get('secret_scanning_enabled', False)
+        return coerce_bool(
+            self.config.get('secret_scanning_enabled'), False
+        )
 
     @staticmethod
     def _path_regex(value: str) -> str:
@@ -258,19 +264,39 @@ def scan(self) -> Dict[str, Any]:
             if (
                 not changed_files
                 and not scope_requested
-                and not self.config.get('scan_all', False)
+                and not coerce_bool(self.config.get('scan_all'), False)
             ):
                 try:
                     from socket_basics.core.config import _detect_git_changed_files
                     changed_files = _detect_git_changed_files(str(self.config.workspace), mode='staged')
                 except Exception:
                     changed_files = []
 
+            # Verification always runs so that findings carry a trustworthy
+            # Verified flag; the setting only controls which result types are
+            # returned. Detector selection is deliberately independent of it.
+            #
+            # coerce_bool, not truthiness: only the environment loader coerces
+            # bool params, while a Socket dashboard config is passed through
+            # verbatim at higher priority. A dashboard-supplied string "false"
+            # is truthy, and reading it as "on" would report unverified secrets
+            # to someone who explicitly left unverified results off.
+            show_unverified = coerce_bool(
+                self.config.get('trufflehog_show_unverified'), False
+            )
+            results_filter = (
+                'verified,unverified,unknown'
+                if show_unverified
+                else 'verified,unknown'
+            )
+
             cmd = [
                 'trufflehog',
                 'filesystem',
                 '--json',
-                '--no-verification' if not self.config.get('trufflehog_show_unverified', False) else '--include-detectors=all'
+                '--include-detectors=all',
+                '--fail-on-scan-errors',
+                f'--results={results_filter}',
             ]
 
             # TruffleHog accepts --exclude-paths only once and expects a file
@@ -315,8 +341,23 @@ def scan(self) -> Dict[str, Any]:
             result = subprocess.run(cmd, capture_output=True, text=True)
 
             if result.returncode != 0:
-                logger.error(f"Trufflehog failed: {result.stderr}")
-                return {}
+                # Fail closed. Returning {} here reports "no secrets found" and
+                # exits green, so a malformed exclude pattern, source error, or
+                # broken install can silently hide all or part of the scan.
+                # An incomplete scanner run must not look like a clean scan.
+                # SystemExit is deliberate: the connector manager catches
+                # Exception, and this must not be downgraded to a skipped
+                # connector.
+                stderr = (result.stderr or '').strip()
+                detail = f": {stderr}" if stderr else ''
+                raise SystemExit(
+                    f"TruffleHog exited {result.returncode} before the scan "
+                    "completed successfully"
+                    f"{detail}\nSecret scanning results are incomplete, so the "
+                    "run is failing rather than reporting a clean scan. Check "
+                    "the exclude patterns in 'trufflehog_exclude_dir' and that "
+                    "the trufflehog binary is working."
+                )
 
             # Parse JSON output line by line
             findings = []
@@ -374,7 +415,12 @@ def scan(self) -> Dict[str, Any]:
             }
 
         except FileNotFoundError:
-            logger.error("Trufflehog not found. Please install Trufflehog")
+            # Also fail closed: secret scanning was asked for and did not run.
+            raise SystemExit(
+                "TruffleHog is enabled but the 'trufflehog' binary was not "
+                "found, so no secret scanning ran. Install TruffleHog or use "
+                "the Socket Basics container image, which bundles it."
+            )
         except Exception as e:
             logger.error(f"Error running Trufflehog: {e}")
         finally:
@@ -474,6 +520,24 @@ def _create_alert(self, finding: Dict[str, Any]) -> Dict[str, Any]:
         """Create a generic alert from a Trufflehog finding"""
         detector_name = finding.get('DetectorName', 'unknown')
         verified = finding.get('Verified', False)
+        verification_error = finding.get('VerificationError')
+        if verified:
+            verification_status = 'verified'
+            risk_assessment = (
+                "**CRITICAL**: This secret has been verified and is likely active!"
+            )
+        elif verification_error:
+            verification_status = 'unknown'
+            risk_assessment = (
+                "**LOW**: Verification could not complete, so this secret's "
+                "validity is unknown."
+            )
+        else:
+            verification_status = 'unverified'
+            risk_assessment = (
+                "**LOW**: This appears to be a potential secret but was not "
+                "confirmed as valid."
+            )
         file_path = finding.get('SourceMetadata', {}).get('Data', {}).get('Filesystem', {}).get('file', 'unknown')
         line = finding.get('SourceMetadata', {}).get('Data', {}).get('Filesystem', {}).get('line', 0)
 
@@ -492,11 +556,11 @@ def _create_alert(self, finding: Dict[str, Any]) -> Dict[str, Any]:
 - **File**: `{file_path}`
 - **Line**: {line}
 - **Detector**: {detector_name}
-- **Verified**: {"✅ Yes" if verified else "❌ No"}
+- **Verification status**: {verification_status}
 - **Redacted Value**: `{redacted_secret}`
 
 ### Risk Assessment
-{"**CRITICAL**: This secret has been verified and is likely active!" if verified else "**LOW**: This appears to be a potential secret but has not been verified."}
+{risk_assessment}
 
 ### Immediate Actions Required
 1. **Rotate the credential immediately**
@@ -534,6 +598,7 @@ def _create_alert(self, finding: Dict[str, Any]) -> Dict[str, Any]:
             "props": {
                 "ruleId": detector_name,
                 "verified": verified,
+                "verificationStatus": verification_status,
                 "filePath": file_path,
                 "lineNumber": line,
                 "secretType": detector_name.lower(),

tests/test_changed_files_scope.py

@@ -887,7 +887,9 @@ def record_run(cmd, *args, **kwargs):
                 "trufflehog",
                 "filesystem",
                 "--json",
-                "--no-verification",
+                "--include-detectors=all",
+                "--fail-on-scan-errors",
+                "--results=verified,unknown",
                 str(pr_repo),
             ]
         ]

tests/test_trufflehog_excludes.py

@@ -4,6 +4,8 @@
 import re
 from types import SimpleNamespace
 
+import pytest
+
 from socket_basics.core.connector.trufflehog import TruffleHogScanner
 
 
@@ -488,6 +490,9 @@ def fake_run(command, **kwargs):
         fake_run,
     )
 
-    scanner.scan()
+    # A failed run surfaces rather than reporting a clean scan (CE-347); the
+    # temporary filter file must still be cleaned up on that path.
+    with pytest.raises(SystemExit):
+        scanner.scan()
 
     assert not captured["exclude_path"].exists()