From 126754f85626388c9977974e10a3a62c8e8ab7c1 Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 18:40:41 -0400 Subject: [PATCH 01/11] chore(release): 3.4.0 Stamps [Unreleased] as [3.4.0], bumps the version files and uv.lock, and synchronizes the current-release references across README.md and docs/**. Minor rather than patch: #119 changes the snippet, detailed report, dataflow trace and description that every consumer of a finding reads, and adds the `redact` rule-metadata key. Also records the socketdev 3.5.0 -> 3.6.0 lockfile bump from #117, which merged without a changelog entry. --- CHANGELOG.md | 47 +++++++++++++++--- README.md | 6 +-- action.yml | 2 +- docs/github-action.md | 58 +++++++++++----------- docs/github-pr-comment-guide.md | 2 +- docs/local-install-docker.md | 88 ++++++++++++++++----------------- docs/parameters.md | 2 +- docs/pre-commit-hook.md | 10 ++-- pyproject.toml | 2 +- socket_basics/__init__.py | 2 +- socket_basics/version.py | 2 +- uv.lock | 4 +- 12 files changed, 128 insertions(+), 97 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 317ce7c..7995585 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,34 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +## [3.4.0] - 2026-09-18 + +Findings for the hardcoded-credential rules now report where a credential is +without reproducing what it is. Minor rather than patch: the snippet a finding +carries changes for every consumer that reads it, and the release adds a new +rule-metadata key. + +### Upgrade notes + +No configuration change is required, but findings differ on the first run after +upgrading. + +- **`codeSnippet` content changes for the credential rules.** The field keeps + the assignment target, the syntax, the file and the line, and masks the + literal. A baseline keyed on exact snippet text will not match; key on rule ID + plus location instead. Rules whose match is not a credential are unaffected. + (#119) +- **The same applies to `detailedReport.content` and `dataflowTrace`.** Both + quote source lines and both are masked on the same terms. (#119) +- **A finding's `description` can also change.** OpenGrep expands metavariables + into a rule's message before returning a result, so a message quoting the + matched value carried it too. Expanded metavariables are masked for the + credential rules. (#119) +- **Masking is deliberately conservative in two visible places.** + `define('SECRET', '...')` masks the constant name along with the value, and + `password: "admin"` hides which default was used. Rule ID, file and line still + identify the finding in both cases. (#119) + ### Fixed - **A finding's snippet no longer reproduces the value it reports.** A SAST finding's `codeSnippet` is the source line the rule matched. For nearly every @@ -20,29 +48,32 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `*-hardcoded-secret(s)`, `*-hardcoded-credentials`, `*-hardcoded-password-default`, `*-default-credentials`, `*-plain-text-password`, `*-weak-jwt-secret` and `*-empty-password`. Rules - whose match is not a credential keep their snippets verbatim. -- Every snippet, dataflow-trace step and detailed report, whatever rule produced - it, is now masked of values matching a well-known credential format: AWS key + whose match is not a credential keep their snippets verbatim. (#119) +- Every snippet, dataflow-trace step, rule message and detailed report, whatever + rule produced it, is now masked of values matching a well-known credential + format: AWS key IDs, GitHub tokens, Stripe keys, Slack tokens, Google API keys, npm and PyPI tokens, JWTs, PEM private key bodies, and credentials in a URL authority. A - rule unrelated to secrets can still match a line that carries one. + rule unrelated to secrets can still match a line that carries one. (#119) - TruffleHog's `redactedValue` kept the first and last four characters of any value longer than eight, which left most of a short password readable. Values - under sixteen characters are now masked in full. + under sixteen characters are now masked in full. (#119) - TruffleHog no longer scans the facts file the run writes. That file lands inside the scan target, so a previous run's output was on disk during the walk and its contents were reported as findings of their own, pointing at the - output file rather than the source line. + output file rather than the source line. (#119) ### Changed +- socketdev 3.5.0 -> 3.6.0 in the lockfile. The `>=3.5.0` floor in + `pyproject.toml` is unchanged. (#117) - `load_explicit_env_config` builds its "API key sources detected" debug line by iterating a tuple of variable names rather than a dict of presence booleans. The line is unchanged, including the exclusion of an exported-but-empty - variable. + variable. (#119) ### Added - A `redact` rule-metadata key. Set it on a custom SAST rule to mark the match - as a credential, or to opt a rule out; without it, the rule name decides. + as a credential, or to opt a rule out; without it, the rule name decides. (#119) ## [3.3.0] - 2026-09-15 diff --git a/README.md b/README.md index ee1ca05..8e45e3c 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ jobs: - name: Run Socket Basics # Pin to a commit SHA for supply-chain safety. # Dependabot will keep this up to date automatically — see docs/github-action.md. - uses: SocketDev/socket-basics@ # v3.3.0 + uses: SocketDev/socket-basics@ # v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -179,10 +179,10 @@ For GitHub Actions, see the [Quick Start](#-quick-start---github-actions) above ```bash # Pull the pre-built image (recommended — no build step required) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 # Run scan -docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0 \ +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ diff --git a/action.yml b/action.yml index 626ac6f..0c69b8b 100644 --- a/action.yml +++ b/action.yml @@ -4,7 +4,7 @@ author: "Socket" runs: using: "docker" - image: "docker://ghcr.io/socketdev/socket-basics:3.3.0" + image: "docker://ghcr.io/socketdev/socket-basics:3.4.0" env: # Core GitHub variables (these are automatically available, but we explicitly pass GITHUB_TOKEN) GITHUB_TOKEN: ${{ inputs.github_token }} diff --git a/docs/github-action.md b/docs/github-action.md index ae051aa..3eb44a2 100644 --- a/docs/github-action.md +++ b/docs/github-action.md @@ -43,7 +43,7 @@ jobs: steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -57,7 +57,7 @@ With just your `SOCKET_SECURITY_API_KEY`, all scanning configurations are manage ### How the action is currently built -When you reference `uses: SocketDev/socket-basics@v3.3.0`, GitHub Actions pulls the +When you reference `uses: SocketDev/socket-basics@v3.4.0`, GitHub Actions pulls the pre-built image referenced by [`action.yml`](../action.yml). The historical multi-stage Docker build still matters for maintainers because it determines what lands in the published image: @@ -75,7 +75,7 @@ Socket Basics from source in every workflow run. ### Pre-built image Starting with v2, the action pulls a pre-built image from GHCR rather than -building from source on every run. Pinning to a specific version tag (e.g. `@v3.3.0`) +building from source on every run. Pinning to a specific version tag (e.g. `@v3.4.0`) means the action starts in seconds — the image is built, integration-tested, and published before the release tag is ever created. @@ -85,7 +85,7 @@ If you run socket-basics in other CI systems (Jenkins, GitLab, CircleCI, etc.) o as a standalone `docker run`, pull the pre-built image directly: ```bash -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 ``` See [Local Docker Installation](local-install-docker.md) for usage examples. @@ -101,7 +101,7 @@ is immediately affected. We've seen this happen across the ecosystem: publish `:latest`/`:latest-heavy` Docker aliases as an onboarding convenience, but treat them as exactly that — production pipelines should pin an exact version or digest.) -- **Version tags** (`@v3.3.0`) are better, but tags are mutable by default. +- **Version tags** (`@v3.4.0`) are better, but tags are mutable by default. A tag can be deleted and recreated pointing at a different commit. There are documented cases of this happening — maliciously and accidentally. - **Commit SHAs** are the only truly immutable reference. A SHA cannot be @@ -126,14 +126,14 @@ The only truly immutable reference. Dependabot keeps it current automatically. ```yaml - name: Run Socket Basics # Dependabot keeps this SHA up to date — see .github/dependabot.yml setup below. - uses: SocketDev/socket-basics@ # v3.3.0 + uses: SocketDev/socket-basics@ # v3.4.0 with: socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} ``` Get the SHA for any release: ```bash -git ls-remote https://github.com/SocketDev/socket-basics refs/tags/v3.3.0 +git ls-remote https://github.com/SocketDev/socket-basics refs/tags/v3.4.0 ``` --- @@ -145,7 +145,7 @@ enforces tag protection rules). SHA pinning is still preferable for defence in depth. ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} ``` @@ -166,7 +166,7 @@ updates: ``` Dependabot opens a PR for each new release, updating the SHA or version tag -and keeping the `# v3.3.0` comment in sync. You review, approve, and merge +and keeping the `# v3.4.0` comment in sync. You review, approve, and merge on your own schedule — automated upgrades with a human gate. --- @@ -176,7 +176,7 @@ on your own schedule — automated upgrades with a human gate. | Strategy | Immutable? | Auto-updates | Review gate | |---|---|---|---| | `@v2` floating tag | ❌ (not published) | — | — | -| `@v3.3.0` + Dependabot | ✅ (tag protection enforced) | Yes (weekly PR) | Yes | +| `@v3.4.0` + Dependabot | ✅ (tag protection enforced) | Yes (weekly PR) | Yes | | `@` + Dependabot | ✅ always | Yes (weekly PR) | Yes | ## Basic Configuration @@ -217,7 +217,7 @@ Include these in your workflow's `jobs..permissions` section. **SAST (Static Analysis):** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} # Enable SAST for specific languages @@ -231,7 +231,7 @@ Include these in your workflow's `jobs..permissions` section. **Secret Scanning:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} secret_scanning_enabled: 'true' @@ -251,7 +251,7 @@ Include these in your workflow's `jobs..permissions` section. **Container Scanning:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} # Listing images or Dockerfiles auto-enables the matching Trivy scan. @@ -271,7 +271,7 @@ Include these in your workflow's `jobs..permissions` section. **Socket Tier 1 Reachability:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_tier_1_enabled: 'true' @@ -280,7 +280,7 @@ Include these in your workflow's `jobs..permissions` section. ### Output Configuration ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} python_sast_enabled: 'true' @@ -318,7 +318,7 @@ jobs: fetch-depth: 0 - name: Run Socket Basics (changed files only) - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number }} with: @@ -447,7 +447,7 @@ Configure Socket Basics centrally from the [Socket Dashboard](https://socket.dev **Enable in workflow:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -460,7 +460,7 @@ Configure Socket Basics centrally from the [Socket Dashboard](https://socket.dev > [!NOTE] > You can also pass credentials using environment variables instead of the `with:` section: > ```yaml -> - uses: SocketDev/socket-basics@v3.3.0 +> - uses: SocketDev/socket-basics@v3.4.0 > env: > SOCKET_SECURITY_API_KEY: ${{ secrets.SOCKET_SECURITY_API_KEY }} > with: @@ -478,7 +478,7 @@ All notification integrations require Socket Enterprise. **Slack Notifications:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -490,7 +490,7 @@ All notification integrations require Socket Enterprise. **Jira Issue Creation:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -505,7 +505,7 @@ All notification integrations require Socket Enterprise. **Microsoft Teams:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -517,7 +517,7 @@ All notification integrations require Socket Enterprise. **Generic Webhook:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -529,7 +529,7 @@ All notification integrations require Socket Enterprise. **SIEM Integration:** ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} socket_org: ${{ secrets.SOCKET_ORG }} @@ -565,7 +565,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -607,7 +607,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Full Security Scan - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -666,7 +666,7 @@ jobs: run: docker build -t myapp:${{ github.sha }} . - name: Run Socket Basics (image + Dockerfile scan) - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number }} with: @@ -727,7 +727,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -785,7 +785,7 @@ jobs: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Run Socket Basics - uses: SocketDev/socket-basics@v3.3.0 + uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} with: @@ -946,7 +946,7 @@ in the [name mapping](parameters.md#name-mapping). ```yaml steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - Must be first - - uses: SocketDev/socket-basics@v3.3.0 + - uses: SocketDev/socket-basics@v3.4.0 ``` ### PR Comments Not Appearing diff --git a/docs/github-pr-comment-guide.md b/docs/github-pr-comment-guide.md index 66a241e..914bb7b 100644 --- a/docs/github-pr-comment-guide.md +++ b/docs/github-pr-comment-guide.md @@ -315,7 +315,7 @@ PR. This is for teams who want to review finding quality in the Socket dashboard first, without every PR growing a comment that developers have to scroll past. ```yaml -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 with: socket_security_api_key: ${{ secrets.SOCKET_SECURITY_API_KEY }} github_token: ${{ secrets.GITHUB_TOKEN }} diff --git a/docs/local-install-docker.md b/docs/local-install-docker.md index 30c301c..570aa1f 100644 --- a/docs/local-install-docker.md +++ b/docs/local-install-docker.md @@ -16,7 +16,7 @@ Run Socket Basics locally using Docker without installing security tools on your ```bash # 1. Pull a pinned release from GHCR (no build step required) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 # 2. Create .env file with your credentials (the API key is environment-only; # the organization can also be passed per run with --socket-org) @@ -29,14 +29,14 @@ EOF docker run --rm \ -v "$PWD:/workspace" \ --env-file .env \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ --console-tabular-enabled ``` -The Docker image should always be pinned to an exact version such as `3.3.0`. Avoid +The Docker image should always be pinned to an exact version such as `3.4.0`. Avoid floating tags like `:latest` in CI/CD. ## Using Pre-built Images @@ -46,13 +46,13 @@ The baked-in security tool versions are recorded in the image labels so you can inspect exactly what's inside: ```bash -docker inspect ghcr.io/socketdev/socket-basics:3.3.0 \ +docker inspect ghcr.io/socketdev/socket-basics:3.4.0 \ | jq '.[0].Config.Labels' # { # "com.socket.trivy-version": "0.73.0", # "com.socket.trufflehog-version": "3.96.0", # "com.socket.opengrep-version": "v1.26.0", -# "org.opencontainers.image.version": "3.3.0", +# "org.opencontainers.image.version": "3.4.0", # ... # } ``` @@ -90,8 +90,8 @@ tool: `socketcli` runs the Python CLI; `socket-basics`, or any other argument, runs Socket Basics: ```bash -docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0-heavy socketcli --help -docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0-heavy --workspace /workspace --python +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0-heavy socketcli --help +docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0-heavy --workspace /workspace --python ``` `latest` and `latest-heavy` are floating aliases published for onboarding @@ -109,7 +109,7 @@ comments and labels. If you must run the image directly, pin the exact version: -v "$GITHUB_WORKSPACE:/workspace" \ -e SOCKET_SECURITY_API_KEY=${{ secrets.SOCKET_SECURITY_API_KEY }} \ -e SOCKET_ORG=${{ secrets.SOCKET_ORG }} \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --javascript \ @@ -122,7 +122,7 @@ comments and labels. If you must run the image directly, pin the exact version: ```yaml security-scan: image: - name: ghcr.io/socketdev/socket-basics:3.3.0 + name: ghcr.io/socketdev/socket-basics:3.4.0 entrypoint: [""] # GitLab needs a shell; the image's entrypoint is socket-basics stage: test script: @@ -141,7 +141,7 @@ security-scan: ```dockerfile # Pin socket-basics and let Dependabot send upgrade PRs automatically -FROM ghcr.io/socketdev/socket-basics:3.3.0 +FROM ghcr.io/socketdev/socket-basics:3.4.0 ``` ### Staying Up to Date with Dependabot @@ -159,7 +159,7 @@ updates: interval: "weekly" ``` -Dependabot will detect the `FROM ghcr.io/socketdev/socket-basics:3.3.0` reference +Dependabot will detect the `FROM ghcr.io/socketdev/socket-basics:3.4.0` reference and open a PR with the version bump when a new release is available. ## Building the Docker Image @@ -170,10 +170,10 @@ Pull a specific release without building locally: ```bash # GHCR (preferred) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 # Docker Hub -docker pull socketdev/socket-basics:3.3.0 +docker pull socketdev/socket-basics:3.4.0 ``` ### Build from Source @@ -186,7 +186,7 @@ git clone https://github.com/SocketDev/socket-basics.git cd socket-basics # Build with version tag (multi-stage; first build is slower, subsequent ones are fast) -docker build -t socket-basics:3.3.0 . +docker build -t socket-basics:3.4.0 . # Verify the build docker images | grep socket-basics @@ -195,7 +195,7 @@ docker images | grep socket-basics ### Build for a Specific Platform (M1/M2 Macs) ```bash -docker build --platform linux/amd64 -t socket-basics:3.3.0 . +docker build --platform linux/amd64 -t socket-basics:3.4.0 . ``` ### Build with Custom Tool Versions @@ -206,7 +206,7 @@ The image pins the bundled tools to specific versions. You can override them at docker build \ --build-arg TRUFFLEHOG_VERSION=3.96.0 \ --build-arg OPENGREP_VERSION=v1.26.0 \ - -t socket-basics:3.3.0 . + -t socket-basics:3.4.0 . ``` Trivy comes from a Socket-built image pinned by digest via the `TRIVY_IMAGE` @@ -219,13 +219,13 @@ tests image, build from the `app_tests` directory and use the same build args. ```bash # The image's entrypoint is `socket-basics`, so its own flags need no prefix -docker run --rm socket-basics:3.3.0 --version +docker run --rm socket-basics:3.4.0 --version # Other bundled tools need --entrypoint -docker run --rm --entrypoint socket socket-basics:3.3.0 --version -docker run --rm --entrypoint opengrep socket-basics:3.3.0 --version -docker run --rm --entrypoint trufflehog socket-basics:3.3.0 --version -docker run --rm --entrypoint trivy socket-basics:3.3.0 --version +docker run --rm --entrypoint socket socket-basics:3.4.0 --version +docker run --rm --entrypoint opengrep socket-basics:3.4.0 --version +docker run --rm --entrypoint trufflehog socket-basics:3.4.0 --version +docker run --rm --entrypoint trivy socket-basics:3.4.0 --version ``` ### Smoke Test @@ -260,7 +260,7 @@ Mount your project directory into the container: # Scan current directory docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -277,7 +277,7 @@ docker run --rm \ # Scan a specific project directory docker run --rm \ -v "/path/to/your/project:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --javascript \ --secrets @@ -288,7 +288,7 @@ docker run --rm \ ```bash docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --all-languages \ --secrets \ @@ -341,7 +341,7 @@ INPUT_VERBOSE=false docker run --rm \ -v "$PWD:/workspace" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets @@ -356,7 +356,7 @@ docker run --rm \ -v "$PWD:/workspace" \ -e "SOCKET_SECURITY_API_KEY=scrt_your_api_key" \ -e "SOCKET_ORG=your-org-slug" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -378,7 +378,7 @@ docker run --rm \ --env-file .env.socket \ --env-file .env.notifiers \ --env-file .env.scanning \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --all-languages ``` @@ -397,7 +397,7 @@ docker run --rm \ -v "$PWD:/workspace" \ -e "SOCKET_SECURITY_API_KEY=$SOCKET_SECURITY_API_KEY" \ -e "SOCKET_ORG=$SOCKET_ORG" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python ``` @@ -424,7 +424,7 @@ docker run --rm \ -e GITHUB_TOKEN \ -e GITHUB_REPOSITORY=owner/repo \ -e GITHUB_PR_NUMBER=123 \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python --javascript --secrets ``` @@ -463,7 +463,7 @@ mkdir -p ./scan-results docker run --rm \ -v "$PWD:/workspace" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -488,7 +488,7 @@ docker run --rm -it \ -v "$PWD:/workspace" \ --env-file .env \ --entrypoint /bin/bash \ - socket-basics:3.3.0 + socket-basics:3.4.0 # Inside container, run commands manually: # cd /workspace @@ -517,7 +517,7 @@ docker run --rm \ -v "$PWD:/workspace" \ -v "$PWD/socket-config.json:/config.json" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --config /config.json ``` @@ -541,7 +541,7 @@ for PROJECT in "${PROJECTS[@]}"; do docker run --rm \ -v "$PROJECT:/workspace" \ --env-file .env \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --all-languages \ --secrets \ @@ -584,7 +584,7 @@ pipeline { script { // --entrypoint='' is required: Jenkins runs `cat` to keep the // container alive, and the image's entrypoint is socket-basics. - docker.image('ghcr.io/socketdev/socket-basics:3.3.0').inside( + docker.image('ghcr.io/socketdev/socket-basics:3.4.0').inside( "--entrypoint='' -v ${WORKSPACE}:/workspace --env-file .env" ) { sh ''' @@ -607,7 +607,7 @@ pipeline { ```yaml security-scan: image: - name: ghcr.io/socketdev/socket-basics:3.3.0 + name: ghcr.io/socketdev/socket-basics:3.4.0 entrypoint: [""] # GitLab needs a shell; the image's entrypoint is socket-basics stage: test script: @@ -650,7 +650,7 @@ security-scan: ```bash docker run --rm \ -v "$(pwd):/workspace" \ # Use $(pwd) instead of $PWD - socket-basics:3.3.0 + socket-basics:3.4.0 ``` 2. Verify mount (the entrypoint is `socket-basics`, so override it to run `ls`): @@ -658,7 +658,7 @@ security-scan: docker run --rm \ -v "$PWD:/workspace" \ --entrypoint ls \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ -la /workspace ``` @@ -688,7 +688,7 @@ security-scan: docker run --rm \ -v "$PWD:/workspace" \ --env-file "$(pwd)/.env" \ - socket-basics:3.3.0 + socket-basics:3.4.0 ``` ### Container Image Too Large @@ -719,7 +719,7 @@ security-scan: ```bash docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --python \ --secrets \ @@ -740,7 +740,7 @@ security-scan: ```bash docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --output /workspace/results.json # Save to mounted directory ``` @@ -751,7 +751,7 @@ security-scan: mkdir -p ./scan-results docker run --rm \ -v "$PWD:/workspace" \ - socket-basics:3.3.0 \ + socket-basics:3.4.0 \ --workspace /workspace \ --output /workspace/scan-results/scan.json ``` @@ -784,7 +784,7 @@ Add these to your `~/.bashrc` or `~/.zshrc` for quick access: ```bash # Socket Basics Docker aliases -alias sb-docker='docker run --rm -v "$PWD:/workspace" --env-file .env ghcr.io/socketdev/socket-basics:3.3.0 --workspace /workspace' +alias sb-docker='docker run --rm -v "$PWD:/workspace" --env-file .env ghcr.io/socketdev/socket-basics:3.4.0 --workspace /workspace' alias sb-quick='sb-docker --secrets --console-tabular-enabled' alias sb-python='sb-docker --python --secrets --console-tabular-enabled' alias sb-js='sb-docker --javascript --secrets --console-tabular-enabled' @@ -810,7 +810,7 @@ sb-all 1. **Use pre-built images** — Pull `ghcr.io/socketdev/socket-basics:` instead of building locally 2. **Use the standard image** — `-heavy` exists for one deployment constraint (see [Image Variants](#image-variants)); it adds nothing to Socket Basics -3. **Pin to a specific version** — Avoid `:latest` in production CI; pin to `3.3.0` and upgrade deliberately +3. **Pin to a specific version** — Avoid `:latest` in production CI; pin to `3.4.0` and upgrade deliberately 4. **Use Dependabot** — Reference the image in your Dockerfile/Compose to get automatic upgrade PRs 5. **Inspect baked-in labels** — Run `docker inspect | jq '.[0].Config.Labels'` to verify tool versions 6. **Use .env files** — Keep credentials out of command history @@ -830,7 +830,7 @@ set -e # Configuration PROJECT_DIR="$(pwd)" RESULTS_DIR="scan-results" # relative to the project: it must stay inside the workspace -IMAGE_NAME="ghcr.io/socketdev/socket-basics:3.3.0" +IMAGE_NAME="ghcr.io/socketdev/socket-basics:3.4.0" ENV_FILE=".env" # Create results directory (add it to .gitignore) diff --git a/docs/parameters.md b/docs/parameters.md index 6e011cf..b06bb13 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -310,7 +310,7 @@ workflow and pass it in yourself: run: echo "ref=$(gh pr view ${{ github.event.issue.number }} --json baseRefName -q .baseRefName)" >> "$GITHUB_OUTPUT" env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} -- uses: SocketDev/socket-basics@v3.3.0 +- uses: SocketDev/socket-basics@v3.4.0 env: GITHUB_BASE_REF: ${{ steps.prbase.outputs.ref }} with: diff --git a/docs/pre-commit-hook.md b/docs/pre-commit-hook.md index c6d5fdd..496ffbe 100644 --- a/docs/pre-commit-hook.md +++ b/docs/pre-commit-hook.md @@ -41,7 +41,7 @@ Best for: Teams wanting consistent environments without installing security tool ```bash # Pull the pre-built image (no build step required) -docker pull ghcr.io/socketdev/socket-basics:3.3.0 +docker pull ghcr.io/socketdev/socket-basics:3.4.0 ``` **2. Create pre-commit hook:** @@ -64,7 +64,7 @@ fi # Run Socket Basics in Docker docker run --rm \ -v "$PWD:/workspace" \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --javascript \ @@ -118,7 +118,7 @@ fi # Scope the scan to the staged changes docker run --rm \ -v "$PWD:/workspace" \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --changed-files auto \ --python \ @@ -153,7 +153,7 @@ docker run --rm \ -e SOCKET_ORG="$SOCKET_ORG" \ -e SOCKET_SECURITY_API_KEY="$SOCKET_SECURITY_API_KEY" \ -e SLACK_WEBHOOK_URL="$SLACK_WEBHOOK_URL" \ - ghcr.io/socketdev/socket-basics:3.3.0 \ + ghcr.io/socketdev/socket-basics:3.4.0 \ --workspace /workspace \ --python \ --javascript \ @@ -504,7 +504,7 @@ repos: hooks: - id: socket-basics name: Socket Basics Security Scan - entry: docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.3.0 --workspace /workspace --changed-files auto --python --secrets + entry: docker run --rm -v "$PWD:/workspace" ghcr.io/socketdev/socket-basics:3.4.0 --workspace /workspace --changed-files auto --python --secrets language: system pass_filenames: false ``` diff --git a/pyproject.toml b/pyproject.toml index c75006a..9a6f9b7 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "socket_basics" -version = "3.3.0" +version = "3.4.0" description = "Socket Basics with integrated SAST, secret scanning, and container analysis" readme = "README.md" requires-python = ">=3.10" diff --git a/socket_basics/__init__.py b/socket_basics/__init__.py index b768586..aa00ec2 100644 --- a/socket_basics/__init__.py +++ b/socket_basics/__init__.py @@ -12,7 +12,7 @@ from .socket_basics import SecurityScanner, main from .core.config import load_config_from_env, Config -__version__ = "3.3.0" +__version__ = "3.4.0" __author__ = "Socket.dev" __email__ = "support@socket.dev" diff --git a/socket_basics/version.py b/socket_basics/version.py index 88c513e..903a158 100644 --- a/socket_basics/version.py +++ b/socket_basics/version.py @@ -1 +1 @@ -__version__ = "3.3.0" +__version__ = "3.4.0" diff --git a/uv.lock b/uv.lock index 21d9722..7ddf171 100644 --- a/uv.lock +++ b/uv.lock @@ -244,7 +244,7 @@ name = "exceptiongroup" version = "1.3.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "typing-extensions" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/0b/9f/a65090624ecf468cdca03533906e7c69ed7588582240cfe7cc9e770b50eb/exceptiongroup-1.3.0.tar.gz", hash = "sha256:b241f5885f560bc56a59ee63ca4c6a8bfa46ae4ad651af316d4e81817bb9fd88", size = 29749, upload-time = "2025-05-10T17:42:51.123Z" } wheels = [ @@ -692,7 +692,7 @@ wheels = [ [[package]] name = "socket-basics" -version = "3.3.0" +version = "3.4.0" source = { editable = "." } dependencies = [ { name = "jsonschema" }, From 5990962c9e51302a5af2af78029cf55a6a4ca99c Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 18:52:03 -0400 Subject: [PATCH 02/11] fix(redaction): treat plain-text-password as logic, not a credential The rule that fragment names matches password *handling* -- assigning request input to a password field, or comparing against one -- so its match is an expression rather than a literal. Running the literal pass on it reduced `user.password = request.form.get('password')` to a row of asterisks, which is the rule's main pattern and leaves nothing to act on. It belongs with `hardcoded-ip` and the password-policy rules, which the same comment already excludes for the same reason. A comparison against a hardcoded value is the one shape it covers that carries a credential, and that is what the `hardcoded-*` rules are for. Also folds a duplicated TestRedactMessage class into one. The second definition shadowed the first, so two message tests never ran. --- CHANGELOG.md | 10 ++++++---- socket_basics/core/utils/redaction.py | 9 ++++++++- tests/test_secret_redaction.py | 18 ++++++++++++++++-- 3 files changed, 30 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7995585..3edf3ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,12 +43,14 @@ upgrading. rules it contains the credential, so the finding carried the value into `.socket.facts.json`, the uploaded facts and the configured notifiers. Snippets for those rules now keep the assignment target, the syntax, the file - and the line, and mask the literal's contents. This covers 20 rules across all + and the line, and mask the literal's contents. This covers 19 rules across all fifteen bundled language rule sets, not only the Python and JavaScript ones: `*-hardcoded-secret(s)`, `*-hardcoded-credentials`, - `*-hardcoded-password-default`, `*-default-credentials`, - `*-plain-text-password`, `*-weak-jwt-secret` and `*-empty-password`. Rules - whose match is not a credential keep their snippets verbatim. (#119) + `*-hardcoded-password-default`, `*-default-credentials`, `*-weak-jwt-secret` + and `*-empty-password`. Rules whose match is logic rather than a literal keep + their snippets verbatim — `*-hardcoded-ip`, the password-policy rules and + `python-plain-text-password`, which matches password handling such as + assigning request input to a password field. (#119) - Every snippet, dataflow-trace step, rule message and detailed report, whatever rule produced it, is now masked of values matching a well-known credential format: AWS key diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 6f69474..437c9ce 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -128,6 +128,14 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # Rule-name fragments whose finding *is* the credential. ``hardcoded-ip`` and # the password-policy rules deliberately do not appear: their snippets are # logic, and masking them would remove the reason the finding was raised. +# +# ``plain-text-password`` belongs with them despite the name. The rule it names +# matches password *handling* -- assigning request input to a password field, or +# comparing against one -- so its match is an expression rather than a literal, +# and the literal pass reduces ``user.password = request.form.get('password')`` +# to a row of asterisks. A comparison against a hardcoded value is the one shape +# it covers that carries a credential, and that shape is what the +# ``hardcoded-*`` rules are for. _CREDENTIAL_RULE_FRAGMENTS = ( 'hardcoded-secret', 'hardcoded-credential', @@ -135,7 +143,6 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, 'hardcoded-key', 'hardcoded-token', 'default-credentials', - 'plain-text-password', 'empty-password', 'weak-jwt-secret', 'private-key', diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index 2862897..fe8941b 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -232,6 +232,10 @@ def test_hardcoded_credential_rules_are_selected(self, rule_id): # These match a length comparison, not a credential. "python-weak-password-validation", "js-weak-password-validation", + # Matches password handling -- request input assigned to a password + # field, or compared against one -- so the match is an expression, + # not a literal, and the literal pass leaves nothing readable. + "python-plain-text-password", "python-sql-injection-format", "js-eval-usage", ], @@ -300,8 +304,6 @@ def test_interpolated_metavariables_are_masked_for_credential_findings(self): def test_known_tokens_are_scrubbed_from_every_message(self): assert AWS_KEY_ID not in redact_message(f"Logged value: {AWS_KEY_ID}") - -class TestRedactMessage: def test_a_short_bound_value_does_not_mangle_the_rest_of_the_message(self): """The replace is by value, so a short one is also an ordinary substring. @@ -343,6 +345,18 @@ def test_a_non_credential_finding_keeps_its_message(self): assert redact_message(message, {"$X": {"abstract_content": "eval"}}) == message +class TestPasswordLogicRules: + def test_a_password_handling_snippet_stays_readable(self): + """``python-plain-text-password`` reports logic, so the logic must show. + + Its main pattern assigns request input to a password field. No literal + credential appears on the line, and masking it leaves nothing to act on. + """ + snippet = "user.password = request.form.get('password')" + credential = is_credential_finding("python-plain-text-password", {}) + assert redact_snippet(snippet, credential_finding=credential) == snippet + + class TestRedactDataflowTrace: def test_a_credential_finding_masks_literals_in_its_trace(self): """A trace step is a source line, so it gets the snippet's treatment. From 6f70d062c63411ccceb0116d6815b2a62c242a74 Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 19:01:13 -0400 Subject: [PATCH 03/11] fix(redaction): serve both shapes of the password-logic rule Dropping plain-text-password from the credential fragments fixed the over-masking but opened a hole: one of that rule's patterns is a comparison against a hardcoded string, and no hardcoded-* rule matches that shape, so `if user.password == "hunter2"` went into the facts file verbatim. Confirmed by scanning a file with exactly that line. The fragment goes back, and the over-masking is fixed where it belongs. An assigned value that calls something is an expression, not a bare credential, so it skips the unquoted fallback and the literal pass masks just the quoted parts. `user.password = request.form.get('password')` keeps its expression, the comparison value is masked, and a bare value with a trailing comment is still masked whole so a short credential cannot be partly revealed. --- CHANGELOG.md | 13 ++++---- socket_basics/core/utils/redaction.py | 30 +++++++++++------ tests/test_secret_redaction.py | 47 ++++++++++++++++++++------- 3 files changed, 62 insertions(+), 28 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3edf3ac..273daeb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,14 +43,15 @@ upgrading. rules it contains the credential, so the finding carried the value into `.socket.facts.json`, the uploaded facts and the configured notifiers. Snippets for those rules now keep the assignment target, the syntax, the file - and the line, and mask the literal's contents. This covers 19 rules across all + and the line, and mask the literal's contents. This covers 20 rules across all fifteen bundled language rule sets, not only the Python and JavaScript ones: `*-hardcoded-secret(s)`, `*-hardcoded-credentials`, - `*-hardcoded-password-default`, `*-default-credentials`, `*-weak-jwt-secret` - and `*-empty-password`. Rules whose match is logic rather than a literal keep - their snippets verbatim — `*-hardcoded-ip`, the password-policy rules and - `python-plain-text-password`, which matches password handling such as - assigning request input to a password field. (#119) + `*-hardcoded-password-default`, `*-default-credentials`, + `*-plain-text-password`, `*-weak-jwt-secret` and `*-empty-password`. Rules + whose match is logic keep their snippets verbatim, and an assigned value that + calls something is treated as code, so + `user.password = request.form.get('password')` keeps its expression while the + quoted argument is masked. (#119) - Every snippet, dataflow-trace step, rule message and detailed report, whatever rule produced it, is now masked of values matching a well-known credential format: AWS key diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 437c9ce..28e81a8 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -125,17 +125,20 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, r'^(?P[^=:]*(?::=|=(?!=)|:(?!:))\s*)(?P\S.*?)(?P\s*)$' ) +# An assigned value that calls something is an expression rather than a bare +# credential, so the literal pass handles it instead of the unquoted fallback. +_CALL_EXPRESSION = re.compile(r'[\w\]\)]\s*\(') + # Rule-name fragments whose finding *is* the credential. ``hardcoded-ip`` and # the password-policy rules deliberately do not appear: their snippets are # logic, and masking them would remove the reason the finding was raised. # -# ``plain-text-password`` belongs with them despite the name. The rule it names -# matches password *handling* -- assigning request input to a password field, or -# comparing against one -- so its match is an expression rather than a literal, -# and the literal pass reduces ``user.password = request.form.get('password')`` -# to a row of asterisks. A comparison against a hardcoded value is the one shape -# it covers that carries a credential, and that shape is what the -# ``hardcoded-*`` rules are for. +# ``plain-text-password`` does stay, even though the rule it names mostly +# matches password *handling* rather than a literal. One of its patterns is a +# comparison against a hardcoded string, and no ``hardcoded-*`` rule covers that +# shape, so dropping it here is the difference between masking a password and +# publishing one. Keeping the handling snippets readable is the job of the +# expression carve-out in ``redact_literals``, not of this list. _CREDENTIAL_RULE_FRAGMENTS = ( 'hardcoded-secret', 'hardcoded-credential', @@ -143,6 +146,7 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, 'hardcoded-key', 'hardcoded-token', 'default-credentials', + 'plain-text-password', 'empty-password', 'weak-jwt-secret', 'private-key', @@ -201,9 +205,15 @@ def _mask_literal(match: 're.Match[str]') -> str: match = _UNQUOTED_ASSIGNMENT.match(line) body = match.group('body') if match else '' if match and not body.lstrip().startswith(('"', "'", '`')): - # If quoted text appears later in an unquoted value, mask the whole - # body. Measuring that combined text could otherwise make a short - # credential eligible for a partial reveal. + if _CALL_EXPRESSION.search(body): + # A call is code, not a value: ``request.form.get('password')`` + # is the finding. Starring the whole body leaves nothing to act + # on, so the literal pass masks just the quoted parts. + masked_lines.append(line) + continue + # A bare value with quoted text after it, such as a trailing + # comment, is masked whole. Measuring the combined text could + # otherwise make a short credential eligible for a partial reveal. masked_body = ( '*' * len(body) if _STRING_LITERAL.search(body) else mask_value(body) ) diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index fe8941b..307803c 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -217,6 +217,7 @@ class TestCredentialRuleSelection: "python-hardcoded-password-default", "js-default-credentials", "js-weak-jwt-secret", + "python-plain-text-password", ], ) def test_hardcoded_credential_rules_are_selected(self, rule_id): @@ -232,10 +233,6 @@ def test_hardcoded_credential_rules_are_selected(self, rule_id): # These match a length comparison, not a credential. "python-weak-password-validation", "js-weak-password-validation", - # Matches password handling -- request input assigned to a password - # field, or compared against one -- so the match is an expression, - # not a literal, and the literal pass leaves nothing readable. - "python-plain-text-password", "python-sql-injection-format", "js-eval-usage", ], @@ -346,15 +343,41 @@ def test_a_non_credential_finding_keeps_its_message(self): class TestPasswordLogicRules: - def test_a_password_handling_snippet_stays_readable(self): - """``python-plain-text-password`` reports logic, so the logic must show. + """``python-plain-text-password`` matches two shapes and needs both served. - Its main pattern assigns request input to a password field. No literal - credential appears on the line, and masking it leaves nothing to act on. - """ - snippet = "user.password = request.form.get('password')" - credential = is_credential_finding("python-plain-text-password", {}) - assert redact_snippet(snippet, credential_finding=credential) == snippet + Its handling patterns assign request input to a password field, where the + expression is the finding. Its comparison pattern can bind a hardcoded + string, and no ``hardcoded-*`` rule covers that shape, so the value has to + be masked here or it is not masked at all. + """ + + def test_a_password_handling_snippet_keeps_its_expression(self): + redacted = redact_snippet( + "user.password = request.form.get('password')", credential_finding=True + ) + assert redacted.startswith("user.password = request.form.get(") + assert redacted.endswith(")") + + def test_a_hardcoded_comparison_value_is_masked(self): + redacted = redact_snippet( + 'if user.password == "hunter2":', credential_finding=True + ) + assert "hunter2" not in redacted + assert redacted.startswith('if user.password == "') + + def test_a_bare_value_with_a_trailing_comment_is_masked_whole(self): + # No call, so this stays on the unquoted path: measuring the value plus + # the comment could otherwise partially reveal a short credential. + redacted = redact_snippet('password: hunter2 # see "notes"', credential_finding=True) + assert "hunter2" not in redacted + assert "notes" not in redacted + + def test_a_literal_argument_to_a_call_is_still_masked(self): + redacted = redact_snippet( + "password = get_secret('default_pw')", credential_finding=True + ) + assert "default_pw" not in redacted + assert redacted.startswith("password = get_secret(") class TestRedactDataflowTrace: From bbfc1d707889e9908770f284e52b4275820a2a4e Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 19:14:17 -0400 Subject: [PATCH 04/11] fix(redaction): bind the right operator and mask trailing comments Three defects in the unquoted-assignment fallback, found by working through the shapes the credential rules actually produce. A call matched anywhere in the value skipped masking entirely, so `password: hunter2 # see get_secret()` kept the credential. The check is now anchored: only a value that opens with a call is treated as an expression. The first operator on the line bound, so a type annotation won over the assignment after it and `password: str = "..."` was starred out whole rather than reaching the literal pass -- ordinary Python and TypeScript. The last operator now binds, and operators covered by a string literal are skipped so the `:` in `url = "https://..."` cannot bind either. That needs the literal spans, which one regex cannot express, so _split_assignment walks the matches. Masking the value left a comment beside it holding the plaintext, as in `password = get_secret() # real value is hunter2`. Text after an unquoted comment marker is now masked too. Also stops measuring an unquoted value together with whatever follows it. `password: hunter2 # plain comment` is long enough for a partial reveal even though `hunter2` is not, and it was rendering as `password: hunt...ment`. --- socket_basics/core/utils/redaction.py | 111 ++++++++++++++++++++------ tests/test_secret_redaction.py | 45 +++++++++++ 2 files changed, 130 insertions(+), 26 deletions(-) diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 28e81a8..66e9220 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -114,20 +114,33 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # Fallback for unquoted forms such as ``password: hunter2`` in config-style # sources, used only when a credential finding has no string literal to mask. # -# The operator alternation is what keeps a quoted value out of this branch. A -# bare ``[=:]`` stops on the first character of ``:=`` or ``==`` and leaves the -# rest of the operator at the head of the value, which no longer looks quoted, -# so a Go short declaration or a comparison would be starred out whole instead -# of going to the literal pass. ``=`` and ``:`` are matched only where they are -# not part of a longer operator, and a comparison assigns nothing, so it does -# not match here at all. -_UNQUOTED_ASSIGNMENT = re.compile( - r'^(?P[^=:]*(?::=|=(?!=)|:(?!:))\s*)(?P\S.*?)(?P\s*)$' -) +# Which operator binds decides whether a value reaches the literal pass, and +# three things have to hold at once: +# +# - Only a whole operator counts. Stopping on the first character of ``:=`` or +# ``==`` leaves the rest of it heading the value, which then does not look +# quoted, so a Go short declaration would be starred out whole. A comparison +# assigns nothing and does not match here at all. +# - The last operator on the line binds. Taking the colon of +# ``password: str = "..."`` leaves ``str = "..."`` as the value, so an +# annotated declaration -- ordinary Python and TypeScript -- would never +# reach the literal pass. +# - An operator inside a string literal is not an operator. The ``:`` in +# ``url = "https://..."`` would otherwise bind and star out the URL. +# +# A single regex cannot express the third, so ``_split_assignment`` walks the +# matches and skips the ones a literal covers. +_ASSIGNMENT_OPERATOR = re.compile(r':=|(?:])=(?!=)|(? str: return scrubbed +def _split_assignment(line: str) -> 'tuple[str, str, str] | None': + """Split a line at the assignment operator that binds, if it has one. + + Returns ``(head, value, trailing_whitespace)``, where ``head`` runs through + the operator and any space after it. Operators covered by a string literal + are skipped, and the last of the rest wins. + """ + spans = [m.span() for m in _STRING_LITERAL.finditer(line)] + chosen = None + for match in _ASSIGNMENT_OPERATOR.finditer(line): + if any(start <= match.start() < end for start, end in spans): + continue + chosen = match + if chosen is None: + return None + + rest = line[chosen.end():] + value = rest.lstrip() + if not value: + return None + head = line[:chosen.end()] + rest[:len(rest) - len(value)] + stripped = value.rstrip() + return head, stripped, value[len(stripped):] + + def redact_literals(text: Any) -> str: """Mask the body of every string literal, keeping the surrounding syntax. ``API_KEY = "sk_live_abc123"`` becomes ``API_KEY = "****************"``: the name, the operator and the line all survive, which is what makes the finding actionable, while the value does not. + + Where the shape of the value is not recognized the whole value is masked + rather than guessed at, so a subscript, a ternary or a prefixed literal + (``f"..."``, ``r'...'``) loses more of the line than a plain assignment + does. That direction is deliberate: the rule ID, file and line still + identify the finding, and the alternative is leaving a credential in place. """ if not isinstance(text, str) or not text: return text if isinstance(text, str) else '' @@ -202,27 +246,42 @@ def _mask_literal(match: 're.Match[str]') -> str: # masked even if another line or a trailing comment contains quoted text. masked_lines = [] for line in text.split('\n'): - match = _UNQUOTED_ASSIGNMENT.match(line) - body = match.group('body') if match else '' - if match and not body.lstrip().startswith(('"', "'", '`')): - if _CALL_EXPRESSION.search(body): + split = _split_assignment(line) + body = split[1] if split else '' + if split and not body.startswith(('"', "'", '`')): + if _CALL_EXPRESSION.match(body): # A call is code, not a value: ``request.form.get('password')`` # is the finding. Starring the whole body leaves nothing to act # on, so the literal pass masks just the quoted parts. masked_lines.append(line) continue - # A bare value with quoted text after it, such as a trailing - # comment, is masked whole. Measuring the combined text could - # otherwise make a short credential eligible for a partial reveal. - masked_body = ( - '*' * len(body) if _STRING_LITERAL.search(body) else mask_value(body) - ) - masked_lines.append( - f"{match.group('head')}{masked_body}{match.group('tail')}" - ) + # Everything else on this path is masked whole. Where the value + # ends is unknowable here -- a trailing comment reads the same as + # more value -- and measuring the two together would reveal the + # head of a short credential: ``hunter2 # plain comment`` is long + # enough for a partial reveal even though ``hunter2`` is not. + masked_lines.append(f"{split[0]}{'*' * len(body)}{split[2]}") else: masked_lines.append(line) - return _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) + masked = _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) + return '\n'.join(_mask_trailing_comment(line) for line in masked.split('\n')) + + +def _mask_trailing_comment(line: str) -> str: + """Mask whatever follows a comment marker on a credential finding's line. + + The passes above mask the value, which leaves a comment beside it holding + the plaintext -- ``password = get_secret() # real value is hunter2`` keeps + the credential the finding is about. A marker inside a string literal is not + a comment, so literal spans are skipped. + """ + spans = [m.span() for m in _STRING_LITERAL.finditer(line)] + for marker in _COMMENT_MARKER.finditer(line): + if any(start <= marker.start() < end for start, end in spans): + continue + rest = line[marker.end():] + return line[:marker.end()] + '*' * len(rest) + return line def is_credential_finding(rule_id: Any, metadata: Mapping[str, Any] | None = None) -> bool: diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index 307803c..f96b4ea 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -205,6 +205,51 @@ def test_empty_literals_are_left_as_they_are(self): assert redact_literals('password = ""') == 'password = ""' +class TestOperatorBinding: + """Which operator binds decides whether a value reaches the literal pass.""" + + def test_an_annotated_declaration_keeps_its_syntax(self): + # The colon of the annotation must not win over the assignment: binding + # it leaves `str = "..."` as the value, which is not quoted. + for line, prefix in ( + ('password: str = "SuperSecret123!"', 'password: str = "'), + ('const password: string = "SuperSecret123!";', 'const password: string = "'), + ): + redacted = redact_literals(line) + assert "SuperSecret123!" not in redacted + assert redacted.startswith(prefix), redacted + + def test_an_operator_inside_a_literal_does_not_bind(self): + # The `:` in the URL scheme would otherwise split the line and star out + # the value instead of masking it as a literal. + redacted = redact_literals('url = "https://example.com/a"') + assert redacted.startswith('url = "') + assert redacted.endswith('"') + + def test_a_value_containing_an_operator_is_masked(self): + assert "hunter2" not in redact_literals('password = "a=b:c hunter2"') + + def test_a_line_with_no_value_after_the_operator_is_left_alone(self): + assert redact_literals("password =") == "password =" + + +class TestTrailingComments: + def test_a_comment_beside_a_masked_value_is_masked_too(self): + """Masking the value alone leaves the plaintext sitting next to it.""" + for line in ( + 'DB_PASSWORD = "x" # real one is hunter2', + "password = get_secret() # real value is hunter2", + "user.password = request.form.get('pw') // was hunter2", + 'password = "SuperSecret123!" -- legacy hunter2', + ): + assert "hunter2" not in redact_literals(line), line + + def test_a_marker_inside_a_literal_is_not_a_comment(self): + # The `#` is part of the URL, so the line is not truncated there. + redacted = redact_literals('password = "a # b"') + assert redacted == 'password = "*****"' + + class TestCredentialRuleSelection: @pytest.mark.parametrize( "rule_id", From 63df31eb77c55f1b4a04eb6131d031fd3c44363d Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 19:32:57 -0400 Subject: [PATCH 05/11] fix(redaction): take the comment off first and mask per statement Two ways a credential stayed in the part of the line the value search never looked at. A comment can hold an operator later in the line than the real one. Because the comment was masked after the operator was chosen, that one bound and the value in front of it was left in the head: `password = hunter2 # see x = y` kept hunter2. The comment now comes off before anything else reads the line. A line can also carry more than one statement, and only one operator binds per statement, so `a = hunter2; password = x` masked the second value and left the first. Masking now runs per statement, split on separators outside string literals. --- socket_basics/core/utils/redaction.py | 93 +++++++++++++++++++-------- tests/test_secret_redaction.py | 33 ++++++++++ 2 files changed, 98 insertions(+), 28 deletions(-) diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 66e9220..48d1ffe 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -136,6 +136,10 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # marker inside a string literal is not a comment, so callers check the spans. _COMMENT_MARKER = re.compile(r'(?:#|//|--)') +# Statement separator. A line can carry more than one assignment, and only one +# operator binds per statement. +_STATEMENT_SEPARATOR = re.compile(r';') + # An assigned value that *opens* with a call is an expression rather than a bare # credential, so the literal pass handles it instead of the unquoted fallback. # Anchored deliberately: matching a call anywhere would let a trailing comment @@ -218,6 +222,53 @@ def _split_assignment(line: str) -> 'tuple[str, str, str] | None': return head, stripped, value[len(stripped):] +def _split_statements(code: str) -> 'list[str]': + """Split code on statement separators, keeping each separator as a piece. + + One operator binds per statement, so a line carrying more than one has to be + handled a statement at a time: in ``a = hunter2; password = x`` the last + operator is the second, which would leave the first value in the head. + A separator inside a string literal is part of the value. + """ + spans = [m.span() for m in _STRING_LITERAL.finditer(code)] + pieces: 'list[str]' = [] + start = 0 + for separator in _STATEMENT_SEPARATOR.finditer(code): + if any(span_start <= separator.start() < span_end + for span_start, span_end in spans): + continue + pieces.append(code[start:separator.start()]) + pieces.append(separator.group(0)) + start = separator.end() + pieces.append(code[start:]) + return pieces + + +def _mask_statement(code: str) -> str: + """Mask the assigned value in a single statement.""" + split = _split_assignment(code) + if not split: + return code + head, value, trailing = split + # A quoted value is the literal pass's job, and a value that opens with a + # call is code rather than a credential: ``request.form.get('password')`` is + # the finding, and starring it leaves nothing to act on. + if value.startswith(('"', "'", '`')) or _CALL_EXPRESSION.match(value): + return code + # Anything else is masked whole. Where the value ends is unknowable here, + # and measuring it together with what follows would reveal the head of a + # short credential. + return f"{head}{'*' * len(value)}{trailing}" + + +def _mask_code(code: str) -> str: + """Mask the assigned value in every statement on one line of code.""" + return ''.join( + piece if piece == ';' else _mask_statement(piece) + for piece in _split_statements(code) + ) + + def redact_literals(text: Any) -> str: """Mask the body of every string literal, keeping the surrounding syntax. @@ -246,42 +297,28 @@ def _mask_literal(match: 're.Match[str]') -> str: # masked even if another line or a trailing comment contains quoted text. masked_lines = [] for line in text.split('\n'): - split = _split_assignment(line) - body = split[1] if split else '' - if split and not body.startswith(('"', "'", '`')): - if _CALL_EXPRESSION.match(body): - # A call is code, not a value: ``request.form.get('password')`` - # is the finding. Starring the whole body leaves nothing to act - # on, so the literal pass masks just the quoted parts. - masked_lines.append(line) - continue - # Everything else on this path is masked whole. Where the value - # ends is unknowable here -- a trailing comment reads the same as - # more value -- and measuring the two together would reveal the - # head of a short credential: ``hunter2 # plain comment`` is long - # enough for a partial reveal even though ``hunter2`` is not. - masked_lines.append(f"{split[0]}{'*' * len(body)}{split[2]}") - else: - masked_lines.append(line) - masked = _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) - return '\n'.join(_mask_trailing_comment(line) for line in masked.split('\n')) + # The comment comes off first. Everything below reasons about where the + # value ends, and a comment can hold anything the value can -- including + # an operator later in the line than the real one, which would bind and + # leave the credential sitting in the head. + code, marker, comment = _split_comment(line) + masked_lines.append(f"{_mask_code(code)}{marker}{'*' * len(comment)}") + return _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) -def _mask_trailing_comment(line: str) -> str: - """Mask whatever follows a comment marker on a credential finding's line. +def _split_comment(line: str) -> 'tuple[str, str, str]': + """Split a line into code, comment marker and comment text. - The passes above mask the value, which leaves a comment beside it holding - the plaintext -- ``password = get_secret() # real value is hunter2`` keeps - the credential the finding is about. A marker inside a string literal is not - a comment, so literal spans are skipped. + The marker is kept so the masked line still reads as commented. A marker + inside a string literal is part of the value, not a comment, so literal + spans are skipped. Returns empty marker and text when there is no comment. """ spans = [m.span() for m in _STRING_LITERAL.finditer(line)] for marker in _COMMENT_MARKER.finditer(line): if any(start <= marker.start() < end for start, end in spans): continue - rest = line[marker.end():] - return line[:marker.end()] + '*' * len(rest) - return line + return line[:marker.start()], marker.group(0), line[marker.end():] + return line, '', '' def is_credential_finding(rule_id: Any, metadata: Mapping[str, Any] | None = None) -> bool: diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index f96b4ea..7567034 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -250,6 +250,39 @@ def test_a_marker_inside_a_literal_is_not_a_comment(self): assert redacted == 'password = "*****"' +class TestStatementsAndComments: + """A line can carry more than the one value the operator search finds.""" + + def test_each_statement_on_a_line_is_masked(self): + # One operator binds per statement. Searching the whole line finds the + # last one and leaves every earlier value sitting in the head. + for line in ( + "a = hunter2; password = x", + "password = hunter2; b = 1", + "user = admin; pwd = hunter2", + ): + assert "hunter2" not in redact_literals(line), line + + def test_a_separator_inside_a_literal_is_part_of_the_value(self): + assert redact_literals('password = "a;b"') == 'password = "***"' + + def test_an_operator_in_a_comment_does_not_bind(self): + # The comment's `=` is later in the line than the real one, so binding + # it would leave the credential in the head. + for line in ( + "password = hunter2 # see x = y", + "password: hunter2 # ratio a:b", + "token = SuperSecret123! # cf. k=v", + ): + redacted = redact_literals(line) + assert "hunter2" not in redacted and "SuperSecret123!" not in redacted, line + + def test_the_comment_marker_survives_so_the_line_still_reads(self): + assert redact_literals("password = hunter2 # note").startswith( + "password = ******* #" + ) + + class TestCredentialRuleSelection: @pytest.mark.parametrize( "rule_id", From 55bc48cc8f883c3d6c439a56b50c68b2d34dcd21 Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 19:46:19 -0400 Subject: [PATCH 06/11] fix(redaction): measure literal spans over the snippet, not per line A literal can open on one line and close on another. Spans were computed per line, so a marker on a literal's second line read as a comment and the rest of that line was starred -- dropping the closing quote, after which the literal pass no longer matched and the opening line's value survived. Spans are now measured once over the whole snippet and consulted by absolute position. Two things fall out of that. A snippet is a slice of a file, so a literal can also never close. The quoted value branch deferred to the literal pass, which never matches an unterminated literal, so `password = "hunter2` was left untouched. It now defers only when the quote opens a span the pass can find, and masks the value whole otherwise. A multi-line literal body was measured as one value, so the head-and-tail reveal exposed the start of its first line. Each line of such a body is now masked whole, with the line breaks kept so the snippet still shows where the literal begins and ends. --- socket_basics/core/utils/redaction.py | 111 +++++++++++++++----------- tests/test_secret_redaction.py | 27 +++++++ 2 files changed, 92 insertions(+), 46 deletions(-) diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 48d1ffe..80b3a27 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -197,78 +197,93 @@ def scrub_tokens(text: Any) -> str: return scrubbed -def _split_assignment(line: str) -> 'tuple[str, str, str] | None': - """Split a line at the assignment operator that binds, if it has one. +def _split_assignment(code: str, offset: int, in_literal) -> 'tuple[str, str, str] | None': + """Split a statement at the assignment operator that binds, if it has one. Returns ``(head, value, trailing_whitespace)``, where ``head`` runs through the operator and any space after it. Operators covered by a string literal are skipped, and the last of the rest wins. """ - spans = [m.span() for m in _STRING_LITERAL.finditer(line)] chosen = None - for match in _ASSIGNMENT_OPERATOR.finditer(line): - if any(start <= match.start() < end for start, end in spans): + for match in _ASSIGNMENT_OPERATOR.finditer(code): + if in_literal(offset + match.start()): continue chosen = match if chosen is None: return None - rest = line[chosen.end():] + rest = code[chosen.end():] value = rest.lstrip() if not value: return None - head = line[:chosen.end()] + rest[:len(rest) - len(value)] + head = code[:chosen.end()] + rest[:len(rest) - len(value)] stripped = value.rstrip() return head, stripped, value[len(stripped):] -def _split_statements(code: str) -> 'list[str]': +def _split_statements(code: str, offset: int, in_literal) -> 'list[tuple[str, int]]': """Split code on statement separators, keeping each separator as a piece. One operator binds per statement, so a line carrying more than one has to be handled a statement at a time: in ``a = hunter2; password = x`` the last operator is the second, which would leave the first value in the head. - A separator inside a string literal is part of the value. """ - spans = [m.span() for m in _STRING_LITERAL.finditer(code)] - pieces: 'list[str]' = [] + pieces: 'list[tuple[str, int]]' = [] start = 0 for separator in _STATEMENT_SEPARATOR.finditer(code): - if any(span_start <= separator.start() < span_end - for span_start, span_end in spans): + if in_literal(offset + separator.start()): continue - pieces.append(code[start:separator.start()]) - pieces.append(separator.group(0)) + pieces.append((code[start:separator.start()], offset + start)) + pieces.append((separator.group(0), offset + separator.start())) start = separator.end() - pieces.append(code[start:]) + pieces.append((code[start:], offset + start)) return pieces -def _mask_statement(code: str) -> str: +def _mask_statement(code: str, offset: int, in_literal) -> str: """Mask the assigned value in a single statement.""" - split = _split_assignment(code) + split = _split_assignment(code, offset, in_literal) if not split: return code head, value, trailing = split - # A quoted value is the literal pass's job, and a value that opens with a - # call is code rather than a credential: ``request.form.get('password')`` is - # the finding, and starring it leaves nothing to act on. - if value.startswith(('"', "'", '`')) or _CALL_EXPRESSION.match(value): + + # A value that opens with a call is code rather than a credential: + # ``request.form.get('password')`` is the finding, and starring it leaves + # nothing to act on. A quoted value is the literal pass's job -- but only + # where the quote opens a literal that pass can find. A snippet cut mid + # string has an opening quote and no closing one, so nothing matches and + # the value would survive untouched. + opens_literal = value.startswith(('"', "'", '`')) and in_literal(offset + len(head)) + if opens_literal or _CALL_EXPRESSION.match(value): return code + # Anything else is masked whole. Where the value ends is unknowable here, # and measuring it together with what follows would reveal the head of a # short credential. return f"{head}{'*' * len(value)}{trailing}" -def _mask_code(code: str) -> str: +def _mask_code(code: str, offset: int, in_literal) -> str: """Mask the assigned value in every statement on one line of code.""" return ''.join( - piece if piece == ';' else _mask_statement(piece) - for piece in _split_statements(code) + piece if piece == ';' else _mask_statement(piece, piece_offset, in_literal) + for piece, piece_offset in _split_statements(code, offset, in_literal) ) +def _split_comment(line: str, offset: int, in_literal) -> 'tuple[str, str, str]': + """Split a line into code, comment marker and comment text. + + The marker is kept so the masked line still reads as commented. A marker + inside a string literal is part of the value, not a comment. + """ + for marker in _COMMENT_MARKER.finditer(line): + if in_literal(offset + marker.start()): + continue + return line[:marker.start()], marker.group(0), line[marker.end():] + return line, '', '' + + def redact_literals(text: Any) -> str: """Mask the body of every string literal, keeping the surrounding syntax. @@ -290,37 +305,41 @@ def _mask_literal(match: 're.Match[str]') -> str: if not body: return match.group(0) quote = match.group('quote') - return f'{quote}{mask_value(body)}{quote}' + if '\n' in body: + # A multi-line body is a block of content rather than one opaque + # value, so the head-and-tail reveal would expose real text -- the + # first line of it. Mask every line and keep the line breaks, so + # the snippet still shows where the literal starts and ends. + masked = '\n'.join('*' * len(segment) for segment in body.split('\n')) + else: + masked = mask_value(body) + return f'{quote}{masked}{quote}' + + # Literal spans are measured over the whole snippet, not line by line. A + # literal can span lines, and a ``#`` or ``--`` on its second line is part + # of the value; reading it as a comment and starring the rest of that line + # can drop the closing quote, after which the literal pass no longer matches + # and the opening line's credential survives. + spans = [match.span() for match in _STRING_LITERAL.finditer(text)] + + def in_literal(position: int) -> bool: + return any(start <= position < end for start, end in spans) - # Mask unquoted assignments line by line before processing literals. A - # quoted value is left for the literal pass, while an unquoted value is - # masked even if another line or a trailing comment contains quoted text. masked_lines = [] + offset = 0 for line in text.split('\n'): # The comment comes off first. Everything below reasons about where the # value ends, and a comment can hold anything the value can -- including # an operator later in the line than the real one, which would bind and # leave the credential sitting in the head. - code, marker, comment = _split_comment(line) - masked_lines.append(f"{_mask_code(code)}{marker}{'*' * len(comment)}") + code, marker, comment = _split_comment(line, offset, in_literal) + masked_lines.append( + f"{_mask_code(code, offset, in_literal)}{marker}{'*' * len(comment)}" + ) + offset += len(line) + 1 return _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) -def _split_comment(line: str) -> 'tuple[str, str, str]': - """Split a line into code, comment marker and comment text. - - The marker is kept so the masked line still reads as commented. A marker - inside a string literal is part of the value, not a comment, so literal - spans are skipped. Returns empty marker and text when there is no comment. - """ - spans = [m.span() for m in _STRING_LITERAL.finditer(line)] - for marker in _COMMENT_MARKER.finditer(line): - if any(start <= marker.start() < end for start, end in spans): - continue - return line[:marker.start()], marker.group(0), line[marker.end():] - return line, '', '' - - def is_credential_finding(rule_id: Any, metadata: Mapping[str, Any] | None = None) -> bool: """Report whether a rule's match is itself a credential. diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index 7567034..412704f 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -283,6 +283,33 @@ def test_the_comment_marker_survives_so_the_line_still_reads(self): ) +class TestMultilineAndUnterminatedLiterals: + """Literal spans are a property of the snippet, not of one line. + + A snippet is a slice of a file, so a literal can open on one line and close + on another, or never close at all. + """ + + TRIPLE_DOUBLE = 'PASSWORD = ' + '"' * 3 + 'hunter2\n# not a comment\n' + '"' * 3 + TRIPLE_SINGLE = 'SQL = ' + "'" * 3 + '\nSELECT hunter2 -- inline\n' + "'" * 3 + BACKTICK = 'password = `hunter2\n// js template\n`' + + def test_a_marker_on_a_later_line_of_a_literal_is_not_a_comment(self): + for snippet in (self.TRIPLE_DOUBLE, self.TRIPLE_SINGLE, self.BACKTICK): + assert "hunter2" not in redact_literals(snippet), snippet + + def test_an_unterminated_literal_is_masked_rather_than_deferred(self): + # A snippet cut mid-string has an opening quote and no closing one, so + # the literal pass never matches it. Deferring would leave it untouched. + for snippet in ('password = "hunter2\n# broken', "password = 'hunter2\n-- sql"): + assert "hunter2" not in redact_literals(snippet), snippet + + def test_a_literal_spanning_lines_keeps_the_line_structure(self): + redacted = redact_literals(self.TRIPLE_DOUBLE) + assert redacted.count("\n") == 2 + assert "hunter2" not in redacted + + class TestCredentialRuleSelection: @pytest.mark.parametrize( "rule_id", From 371bf689f48ce88b816f21d5470381a37a643d9c Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 20:01:40 -0400 Subject: [PATCH 07/11] fix(redaction): recognize string prefixes and interpolated bodies A prefixed opener such as r""" or f""" was not read as opening a literal, so the opening line was starred and its quotes were removed. Later lines were still measured against the original spans, which said they were inside a literal, so nothing masked them and the final literal pass no longer matched. The prefix is now part of the opener check. That alone left a partial reveal: literal text around an interpolation inflates the body past the reveal threshold, so f"{b}_SuperSecret123!" showed 123!. An interpolated body is masked whole, on the same reasoning as a multi-line one -- it is a block of content, not a single opaque value. --- socket_basics/core/utils/redaction.py | 27 +++++++++++++++++----- tests/test_secret_redaction.py | 32 +++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 6 deletions(-) diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 80b3a27..e2b2946 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -146,6 +146,15 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # such as ``# see get_secret()`` disable masking for the value in front of it. _CALL_EXPRESSION = re.compile(r'^[\w.\[\]]+\s*\(') +# A value that opens a string literal, allowing the usual raw/bytes/format/ +# unicode prefixes. The prefix has to be recognized here: treating ``r"""...`` +# as unquoted stars the opening line, which removes the quotes the rest of the +# snippet is measured against. +_LITERAL_OPENER = re.compile(r'^(?:rb|br|rf|fr|r|b|u|f)?(?P["\'`])', re.IGNORECASE) + +# Interpolation placeholders: f-strings, template literals, shell-style. +_INTERPOLATION = re.compile(r'\$?\{[^}]*\}') + # Rule-name fragments whose finding *is* the credential. ``hardcoded-ip`` and # the password-policy rules deliberately do not appear: their snippets are # logic, and masking them would remove the reason the finding was raised. @@ -253,7 +262,10 @@ def _mask_statement(code: str, offset: int, in_literal) -> str: # where the quote opens a literal that pass can find. A snippet cut mid # string has an opening quote and no closing one, so nothing matches and # the value would survive untouched. - opens_literal = value.startswith(('"', "'", '`')) and in_literal(offset + len(head)) + opener = _LITERAL_OPENER.match(value) + opens_literal = bool(opener) and in_literal( + offset + len(head) + opener.start('quote') + ) if opens_literal or _CALL_EXPRESSION.match(value): return code @@ -305,11 +317,14 @@ def _mask_literal(match: 're.Match[str]') -> str: if not body: return match.group(0) quote = match.group('quote') - if '\n' in body: - # A multi-line body is a block of content rather than one opaque - # value, so the head-and-tail reveal would expose real text -- the - # first line of it. Mask every line and keep the line breaks, so - # the snippet still shows where the literal starts and ends. + if '\n' in body or _INTERPOLATION.search(body): + # A body that spans lines, or that interpolates, is a block of + # content rather than one opaque value. The head-and-tail reveal + # measures the whole thing, so the literal text around a placeholder + # inflates the length and buys a reveal the bare value would not get + # -- ``f"{b}_SuperSecret123!"`` would show ``123!``. Mask every line + # and keep the line breaks, so the snippet still shows where the + # literal starts and ends. masked = '\n'.join('*' * len(segment) for segment in body.split('\n')) else: masked = mask_value(body) diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index 412704f..d2c20d0 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -310,6 +310,38 @@ def test_a_literal_spanning_lines_keeps_the_line_structure(self): assert "hunter2" not in redacted +class TestPrefixedAndInterpolatedLiterals: + """A prefix still opens a literal, and an interpolated body is not one value.""" + + Q3 = '"' * 3 + + def test_a_prefixed_multiline_literal_is_masked(self): + # Treating the prefix as unquoted stars the opening line, which removes + # the quotes the rest of the snippet is measured against: later lines + # still read as inside a literal, so nothing masks them. + for prefix in ("r", "f", "rb", "R"): + snippet = f"password = {prefix}{self.Q3}hunter2\nmore SuperSecret123!\n{self.Q3}" + redacted = redact_literals(snippet) + assert "hunter2" not in redacted, snippet + assert "SuperSecret123!" not in redacted, snippet + + def test_a_prefixed_single_line_literal_keeps_its_syntax(self): + redacted = redact_literals("password = r'SuperSecret123!'") + assert "SuperSecret123!" not in redacted + assert redacted.startswith("password = r'") + + def test_an_interpolated_body_is_masked_whole(self): + # The literal text around a placeholder inflates the body past the + # partial-reveal threshold, which would expose the tail of the value. + for snippet in ( + 'password = f"{b}_SuperSecret123!"', + "password = `${b}_SuperSecret123!`", + ): + redacted = redact_literals(snippet) + assert "SuperSecret123!" not in redacted + assert "123!" not in redacted, redacted + + class TestCredentialRuleSelection: @pytest.mark.parametrize( "rule_id", From fcf7ba901b39acbb89bd342f9391d97e8c364d77 Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 20:59:11 -0400 Subject: [PATCH 08/11] fix(redaction): do not defer on a spurious empty-literal match An unterminated triple-quoted value still produces a literal match: the engine backtracks past the triple alternative and reads the first two quotes as an empty string. That match was enough to send the value to the masking pass, which then covered only those two quotes, so the credential stayed in the snippet. Affects bare and prefixed openers alike. Deferring now requires a span that starts at the quote and holds both delimiters, which an empty match cannot satisfy. _STRING_LITERAL also gained triple-quoted alternatives so a terminated block matches once with its real body rather than as an empty string followed by a second literal. --- socket_basics/core/utils/redaction.py | 44 +++++++++++++++++++++------ tests/test_secret_redaction.py | 12 ++++++++ 2 files changed, 46 insertions(+), 10 deletions(-) diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index e2b2946..cb17e4e 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -104,10 +104,16 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, re.DOTALL, ) -# Quoted string literals, including escaped quotes. Covers the single, double -# and backtick forms the bundled rules match across languages. +# Quoted string literals, including escaped quotes. Covers the single, double, +# backtick and triple-quoted forms the bundled rules match across languages. +# +# The triple-quoted alternatives come first so a terminated block matches as one +# literal with its real body. They do not rescue the unterminated case: the +# engine backtracks to the single-quote alternative, which matches the first two +# quotes of ``\"\"\"`` as an empty string. ``_mask_statement`` guards against that +# spurious match rather than the pattern. _STRING_LITERAL = re.compile( - r"""(?P["'`])(?P(?:\\.|(?!(?P=quote))[^\\])*)(?P=quote)""", + r'(?P"""|\'\'\'|["\'`])(?P(?:\\.|(?!(?P=quote))[^\\])*)(?P=quote)', re.DOTALL, ) @@ -150,7 +156,9 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # unicode prefixes. The prefix has to be recognized here: treating ``r"""...`` # as unquoted stars the opening line, which removes the quotes the rest of the # snippet is measured against. -_LITERAL_OPENER = re.compile(r'^(?:rb|br|rf|fr|r|b|u|f)?(?P["\'`])', re.IGNORECASE) +_LITERAL_OPENER = re.compile( + r'^(?:rb|br|rf|fr|r|b|u|f)?(?P"""|\'\'\'|["\'`])', re.IGNORECASE +) # Interpolation placeholders: f-strings, template literals, shell-style. _INTERPOLATION = re.compile(r'\$?\{[^}]*\}') @@ -249,7 +257,7 @@ def _split_statements(code: str, offset: int, in_literal) -> 'list[tuple[str, in return pieces -def _mask_statement(code: str, offset: int, in_literal) -> str: +def _mask_statement(code: str, offset: int, in_literal, literal_opens_at) -> str: """Mask the assigned value in a single statement.""" split = _split_assignment(code, offset, in_literal) if not split: @@ -263,8 +271,8 @@ def _mask_statement(code: str, offset: int, in_literal) -> str: # string has an opening quote and no closing one, so nothing matches and # the value would survive untouched. opener = _LITERAL_OPENER.match(value) - opens_literal = bool(opener) and in_literal( - offset + len(head) + opener.start('quote') + opens_literal = bool(opener) and literal_opens_at( + offset + len(head) + opener.start('quote'), len(opener.group('quote')) ) if opens_literal or _CALL_EXPRESSION.match(value): return code @@ -275,10 +283,11 @@ def _mask_statement(code: str, offset: int, in_literal) -> str: return f"{head}{'*' * len(value)}{trailing}" -def _mask_code(code: str, offset: int, in_literal) -> str: +def _mask_code(code: str, offset: int, in_literal, literal_opens_at) -> str: """Mask the assigned value in every statement on one line of code.""" return ''.join( - piece if piece == ';' else _mask_statement(piece, piece_offset, in_literal) + piece if piece == ';' + else _mask_statement(piece, piece_offset, in_literal, literal_opens_at) for piece, piece_offset in _split_statements(code, offset, in_literal) ) @@ -340,6 +349,20 @@ def _mask_literal(match: 're.Match[str]') -> str: def in_literal(position: int) -> bool: return any(start <= position < end for start, end in spans) + def literal_opens_at(position: int, delimiter: int) -> bool: + """Report whether a real literal starts at ``position``. + + A value is only deferred to the masking pass when that pass will cover + it. ``\"\"\"secret`` with no closing delimiter still produces a match -- + the first two quotes read as an empty string -- so requiring the span to + hold an opening and a closing delimiter is what separates a literal the + pass can mask from an artifact of the unterminated one. + """ + return any( + start == position and end - start >= 2 * delimiter + for start, end in spans + ) + masked_lines = [] offset = 0 for line in text.split('\n'): @@ -349,7 +372,8 @@ def in_literal(position: int) -> bool: # leave the credential sitting in the head. code, marker, comment = _split_comment(line, offset, in_literal) masked_lines.append( - f"{_mask_code(code, offset, in_literal)}{marker}{'*' * len(comment)}" + f"{_mask_code(code, offset, in_literal, literal_opens_at)}" + f"{marker}{'*' * len(comment)}" ) offset += len(line) + 1 return _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index d2c20d0..28816dc 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -298,6 +298,18 @@ def test_a_marker_on_a_later_line_of_a_literal_is_not_a_comment(self): for snippet in (self.TRIPLE_DOUBLE, self.TRIPLE_SINGLE, self.BACKTICK): assert "hunter2" not in redact_literals(snippet), snippet + def test_an_unterminated_triple_quote_is_masked(self): + """The spurious empty match must not read as a literal worth deferring. + + ``\"\"\"secret`` with no closing delimiter still produces a match: the + first two quotes parse as an empty string. Treating that as "a literal + the masking pass will cover" leaves the value untouched, because the + pass covers only the two quotes. + """ + for opener in ('"' * 3, "'" * 3, 'r' + '"' * 3, "f" + "'" * 3): + snippet = f"password = {opener}hunter2\nunterminated" + assert "hunter2" not in redact_literals(snippet), snippet + def test_an_unterminated_literal_is_masked_rather_than_deferred(self): # A snippet cut mid-string has an opening quote and no closing one, so # the literal pass never matches it. Deferring would leave it untouched. From 6626bec38c6ddbaf64617ede4085f331ab65f4b8 Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 21:33:55 -0400 Subject: [PATCH 09/11] fix(redaction): fail safe when string state is lost, and fuzz the invariant A generated-snippet sweep found three gaps the hand-written cases did not, all of them the same thing: masking depends on knowing where string literals start and end, and a truncated snippet can make that unknowable. An unterminated literal is not reached by the assignment fallback when the value sits in a comparison or a call argument, so it went to the masking pass, which cannot match it. A quote outside every matched span now marks the rest of the line as literal content. An unclosed triple-quoted block does not simply fail to match -- its first two quotes match as an empty string and the third pairs with any stray quote later, producing one long span that hides a real assignment on a later line. An odd count of triple delimiters now masks from the opener to the end. A " or ' literal cannot hold a raw newline in any language these rules cover, so a match that does is the same pairing artifact rather than a literal. Those spans are discarded; backticks and triple quotes keep theirs. tests/test_secret_redaction_fuzz.py generates the combinations rather than listing them, and asserts no credential survives, none is partly revealed, and non-credential snippets come through unchanged. 2,000,000 generated cases pass; 20,000 run in CI in about a second. --- socket_basics/core/utils/redaction.py | 62 +++++++++++++- tests/test_secret_redaction_fuzz.py | 114 ++++++++++++++++++++++++++ 2 files changed, 174 insertions(+), 2 deletions(-) create mode 100644 tests/test_secret_redaction_fuzz.py diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index cb17e4e..6a0a454 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -163,6 +163,13 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # Interpolation placeholders: f-strings, template literals, shell-style. _INTERPOLATION = re.compile(r'\$?\{[^}]*\}') +# Any quote character. Used to find the opening delimiter of a literal that +# never closes, which by definition no literal match can cover. +_QUOTE = re.compile(r'["\'`]') + +# Triple-quote delimiters, counted to detect a block that never closes. +_TRIPLE_QUOTE = re.compile(r'\"\"\"|\'\'\'') + # Rule-name fragments whose finding *is* the credential. ``hardcoded-ip`` and # the password-policy rules deliberately do not appear: their snippets are # logic, and masking them would remove the reason the finding was raised. @@ -344,7 +351,17 @@ def _mask_literal(match: 're.Match[str]') -> str: # of the value; reading it as a comment and starring the rest of that line # can drop the closing quote, after which the literal pass no longer matches # and the opening line's credential survives. - spans = [match.span() for match in _STRING_LITERAL.finditer(text)] + # A ``"`` or ``'`` literal cannot hold a raw newline in any language these + # rules cover, so a match that does is not a literal -- it is an unclosed + # quote that paired with a stray one further down, and the span between them + # would hide whatever it covers, including a real assignment on a later + # line. Backticks and triple quotes span lines legitimately and are kept. + spans = [ + match.span() for match in _STRING_LITERAL.finditer(text) + if len(match.group('quote')) > 1 + or match.group('quote') == '`' + or '\n' not in match.group(0) + ] def in_literal(position: int) -> bool: return any(start <= position < end for start, end in spans) @@ -363,6 +380,23 @@ def literal_opens_at(position: int, delimiter: int) -> bool: for start, end in spans ) + def _mask_unterminated(line: str, line_offset: int) -> str: + """Mask the content of a literal that never closes. + + A quote outside every matched span opens a literal with no end, so the + masking pass cannot reach what follows it. The assignment fallback + covers this where the value is an assignment, but a comparison or a call + argument reaches the literal pass directly and would keep the value. + Rather than enumerate those paths, treat an unmatched quote as the + boundary it is and mask the rest of the line. + """ + for quote in _QUOTE.finditer(line): + if in_literal(line_offset + quote.start()): + continue + kept = quote.end() + return line[:kept] + '*' * len(line[kept:]) + return line + masked_lines = [] offset = 0 for line in text.split('\n'): @@ -376,7 +410,31 @@ def literal_opens_at(position: int, delimiter: int) -> bool: f"{marker}{'*' * len(comment)}" ) offset += len(line) + 1 - return _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) + masked = _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) + + # An unclosed triple-quoted block poisons the spans rather than producing + # none: its first two quotes match as an empty string, and the third pairs + # with any stray quote further on, so one long bogus span swallows whatever + # lies between -- including a real assignment on a later line. Once string + # state is lost there is nothing trustworthy after the opener, so mask from + # it to the end. + unclosed = None + for delimiter in ('\"\"\"', "'''"): + found = [m.start() for m in re.finditer(re.escape(delimiter), text)] + if len(found) % 2: + unclosed = found[-1] if unclosed is None else min(unclosed, found[-1]) + if unclosed is not None: + kept = unclosed + 3 + masked = masked[:kept] + re.sub(r'[^\n]', '*', masked[kept:]) + + # Every pass above preserves length, so a position in the masked text still + # indexes the same character of the original and the spans stay valid. + final_lines = [] + offset = 0 + for line in masked.split('\n'): + final_lines.append(_mask_unterminated(line, offset)) + offset += len(line) + 1 + return '\n'.join(final_lines) def is_credential_finding(rule_id: Any, metadata: Mapping[str, Any] | None = None) -> bool: diff --git a/tests/test_secret_redaction_fuzz.py b/tests/test_secret_redaction_fuzz.py new file mode 100644 index 0000000..01905cd --- /dev/null +++ b/tests/test_secret_redaction_fuzz.py @@ -0,0 +1,114 @@ +"""Generated-snippet coverage for credential masking. + +The hand-written cases in ``test_secret_redaction`` assert the shapes someone +thought of. Every gap found in review was a shape nobody thought of, so this +generates them instead: assignment operators, quote styles and prefixes, +terminated and unterminated literals, comments, statement separators, and +nesting, combined at random and checked against one invariant. + + A credential in the input never survives to the output. + +The seed is fixed so a failure is reproducible. Raise ``ITERATIONS`` locally, or +run ``.context/fuzz_redaction.py`` for a longer sweep, when changing masking. +""" + +import random + +from socket_basics.core.utils.redaction import redact_snippet + +ITERATIONS = 20_000 +SEED = 1337 + +SHORT_SECRET = "hunter2" +LONG_SECRET = "Sup3rS3cr3tValue99xyz" + +TARGETS = [ + "password", "api_key", "DB_PASSWORD", "user.password", "apiKey", + "self.password", "PASSWORD", "password: str", "const password: string", + "creds['db']", "x", +] +OPERATORS = ["=", ":", ":=", "==", "!=", " = ", ": "] +QUOTES = ['"', "'", "`", '"""', "'''"] +PREFIXES = ["", "r", "f", "b", "u", "rb", "R", "F"] +COMMENT_MARKERS = ["#", "//", "--"] +COMMENT_BODIES = [ + "note", "see x = y", "ratio a:b", 'a "quoted" note', 'unbalanced " quote', + "cf. k=v", "see get_secret()", "", +] +WRAPPERS = [ + "{value}", "get_secret({value})", "a if b else {value}", + "{value} + other", "other + {value}", "[{value}]", "f({value}, x)", +] + + +def _value(rng, secret, allow_bare=True): + # A bare token inside a call argument or a comparison is a variable + # reference in any real language, not a credential, so it is not generated + # there -- it would report a leak that cannot occur in a real snippet. + choices = ["quoted", "quoted", "quoted", "wrapped"] + if allow_bare: + choices.append("bare") + style = rng.choice(choices) + if style == "bare": + return secret + if style == "wrapped": + return rng.choice(WRAPPERS).format(value=_value(rng, secret, allow_bare=False)) + body = secret + if rng.random() < 0.25: + body = "{x}_" + secret # interpolation inflates the length + if rng.random() < 0.2: + body = secret + "\nsecond line" # the literal spans lines + quote = rng.choice(QUOTES) + terminated = rng.random() < 0.8 + return rng.choice(PREFIXES) + quote + body + (quote if terminated else "") + + +def _statement(rng, secret): + operator = rng.choice(OPERATORS) + comparison = "==" in operator or "!=" in operator + return rng.choice(TARGETS) + operator + _value(rng, secret, allow_bare=not comparison) + + +def _snippet(rng, secret): + parts = [_statement(rng, secret)] + if rng.random() < 0.25: + decoy = rng.choice(["b = 1", "other = x", f"a = {secret}"]) + parts = [parts[0], decoy] if rng.random() < 0.5 else [decoy, parts[0]] + line = "; ".join(parts) + if rng.random() < 0.4: + line += f" {rng.choice(COMMENT_MARKERS)} {rng.choice(COMMENT_BODIES)}" + if rng.random() < 0.15: + line = f"def f():\n {line}\n return None" + if rng.random() < 0.1: + line = " " + line + " " + return line + + +def test_a_generated_credential_never_survives_masking(): + rng = random.Random(SEED) + for _ in range(ITERATIONS): + secret = rng.choice([SHORT_SECRET, LONG_SECRET]) + snippet = _snippet(rng, secret) + masked = redact_snippet(snippet, credential_finding=True) + assert secret not in masked, f"leaked\n in : {snippet!r}\n out: {masked!r}" + + +def test_a_generated_credential_is_never_partly_revealed(): + # The head-and-tail reveal is deliberate for a plain value, but it must not + # expose a recognizable run of a credential that was masked another way. + rng = random.Random(SEED + 1) + for _ in range(ITERATIONS): + snippet = _snippet(rng, LONG_SECRET) + masked = redact_snippet(snippet, credential_finding=True) + for fragment in (LONG_SECRET[:8], LONG_SECRET[-8:]): + assert fragment not in masked, ( + f"partial reveal\n in : {snippet!r}\n out: {masked!r}" + ) + + +def test_generated_non_credential_snippets_are_untouched(): + # Masking must not reach a finding whose match is ordinary code. + rng = random.Random(SEED + 2) + for _ in range(ITERATIONS // 4): + snippet = _snippet(rng, "ordinary_identifier") + assert redact_snippet(snippet, credential_finding=False) == snippet From a363e2a15efbca757e0f13d44558968bcba9aa1a Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 21:45:03 -0400 Subject: [PATCH 10/11] fix(redaction): mask to the end of a snippet once string state is lost Bugbot found that a credential on a continuation line survived, and extending the fuzzer to put the secret after the line break -- it had only ever put it before -- found a second case immediately. Both are the same thing: masking that stops at the opening line. Where a literal opens and its end is unknowable, everything after is inside it as far as any reader can tell, so masking now runs to the end of the snippet rather than the end of the line. The two ways state is lost -- a quote no surviving span covers, and an odd number of triple delimiters -- are handled together instead of separately. The second case was the opposite failure. Masking an unquoted value whole destroyed the opening quote of a literal that continued past the line, so the snippet-wide pass afterwards no longer matched and the rest of the literal was left alone. Such a value is now masked only up to the opener, and the pass takes the literal itself. Both generators put the secret on either side of a line break, so the shape is covered from here on. 1,000,000 generated cases pass. --- socket_basics/core/utils/redaction.py | 94 ++++++++++++++------------- tests/test_secret_redaction_fuzz.py | 5 +- 2 files changed, 54 insertions(+), 45 deletions(-) diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 6a0a454..10148da 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -168,6 +168,8 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, _QUOTE = re.compile(r'["\'`]') # Triple-quote delimiters, counted to detect a block that never closes. +TRIPLE_DOUBLE = chr(34) * 3 +TRIPLE_SINGLE = chr(39) * 3 _TRIPLE_QUOTE = re.compile(r'\"\"\"|\'\'\'') # Rule-name fragments whose finding *is* the credential. ``hardcoded-ip`` and @@ -264,7 +266,8 @@ def _split_statements(code: str, offset: int, in_literal) -> 'list[tuple[str, in return pieces -def _mask_statement(code: str, offset: int, in_literal, literal_opens_at) -> str: +def _mask_statement(code: str, offset: int, in_literal, literal_opens_at, + literal_start_within) -> str: """Mask the assigned value in a single statement.""" split = _split_assignment(code, offset, in_literal) if not split: @@ -287,14 +290,27 @@ def _mask_statement(code: str, offset: int, in_literal, literal_opens_at) -> str # Anything else is masked whole. Where the value ends is unknowable here, # and measuring it together with what follows would reveal the head of a # short credential. + # + # Unless the value contains a literal that runs past this line. Starring it + # would destroy the opening quote, and the masking pass -- which runs over + # the whole snippet afterwards -- would then no longer match, leaving the + # rest of that literal untouched on its continuation lines. Mask up to the + # opener and let the pass have the literal itself. + value_start = offset + len(head) + literal_at = literal_start_within(value_start, value_start + len(value)) + if literal_at is not None: + keep_from = literal_at - value_start + return f"{head}{'*' * keep_from}{value[keep_from:]}{trailing}" return f"{head}{'*' * len(value)}{trailing}" -def _mask_code(code: str, offset: int, in_literal, literal_opens_at) -> str: +def _mask_code(code: str, offset: int, in_literal, literal_opens_at, + literal_start_within) -> str: """Mask the assigned value in every statement on one line of code.""" return ''.join( piece if piece == ';' - else _mask_statement(piece, piece_offset, in_literal, literal_opens_at) + else _mask_statement(piece, piece_offset, in_literal, literal_opens_at, + literal_start_within) for piece, piece_offset in _split_statements(code, offset, in_literal) ) @@ -366,6 +382,11 @@ def _mask_literal(match: 're.Match[str]') -> str: def in_literal(position: int) -> bool: return any(start <= position < end for start, end in spans) + def literal_start_within(start: int, stop: int): + """Return the first literal opening inside ``[start, stop)``, if any.""" + found = [s for s, _ in spans if start <= s < stop] + return min(found) if found else None + def literal_opens_at(position: int, delimiter: int) -> bool: """Report whether a real literal starts at ``position``. @@ -380,23 +401,6 @@ def literal_opens_at(position: int, delimiter: int) -> bool: for start, end in spans ) - def _mask_unterminated(line: str, line_offset: int) -> str: - """Mask the content of a literal that never closes. - - A quote outside every matched span opens a literal with no end, so the - masking pass cannot reach what follows it. The assignment fallback - covers this where the value is an assignment, but a comparison or a call - argument reaches the literal pass directly and would keep the value. - Rather than enumerate those paths, treat an unmatched quote as the - boundary it is and mask the rest of the line. - """ - for quote in _QUOTE.finditer(line): - if in_literal(line_offset + quote.start()): - continue - kept = quote.end() - return line[:kept] + '*' * len(line[kept:]) - return line - masked_lines = [] offset = 0 for line in text.split('\n'): @@ -406,35 +410,37 @@ def _mask_unterminated(line: str, line_offset: int) -> str: # leave the credential sitting in the head. code, marker, comment = _split_comment(line, offset, in_literal) masked_lines.append( - f"{_mask_code(code, offset, in_literal, literal_opens_at)}" + f"{_mask_code(code, offset, in_literal, literal_opens_at, literal_start_within)}" f"{marker}{'*' * len(comment)}" ) offset += len(line) + 1 - masked = _STRING_LITERAL.sub(_mask_literal, '\n'.join(masked_lines)) - - # An unclosed triple-quoted block poisons the spans rather than producing - # none: its first two quotes match as an empty string, and the third pairs - # with any stray quote further on, so one long bogus span swallows whatever - # lies between -- including a real assignment on a later line. Once string - # state is lost there is nothing trustworthy after the opener, so mask from - # it to the end. - unclosed = None - for delimiter in ('\"\"\"', "'''"): + masked = _STRING_LITERAL.sub(_mask_literal, "\n".join(masked_lines)) + + # Everything above depends on knowing where literals begin and end. A + # snippet is a slice of a file, so that is sometimes unknowable, and the two + # ways it happens both read as "a literal opened and never closed": + # + # - a quote no surviving span covers, and + # - an odd number of triple delimiters, whose first two quotes match as an + # empty string while the third pairs with any stray quote further on. + # + # Past such a point every character is inside that literal as far as any + # reader can tell, so it is masked to the end of the snippet rather than to + # the end of its line: the credential is often on a continuation line. + # Every pass above preserves length, so positions still line up. + unreliable = [] + for quote in _QUOTE.finditer(text): + if not in_literal(quote.start()): + unreliable.append(quote.end()) + break + for delimiter in (TRIPLE_DOUBLE, TRIPLE_SINGLE): found = [m.start() for m in re.finditer(re.escape(delimiter), text)] if len(found) % 2: - unclosed = found[-1] if unclosed is None else min(unclosed, found[-1]) - if unclosed is not None: - kept = unclosed + 3 - masked = masked[:kept] + re.sub(r'[^\n]', '*', masked[kept:]) - - # Every pass above preserves length, so a position in the masked text still - # indexes the same character of the original and the spans stay valid. - final_lines = [] - offset = 0 - for line in masked.split('\n'): - final_lines.append(_mask_unterminated(line, offset)) - offset += len(line) + 1 - return '\n'.join(final_lines) + unreliable.append(found[-1] + len(delimiter)) + if unreliable: + kept = min(unreliable) + masked = masked[:kept] + re.sub(r"[^\n]", "*", masked[kept:]) + return masked def is_credential_finding(rule_id: Any, metadata: Mapping[str, Any] | None = None) -> bool: diff --git a/tests/test_secret_redaction_fuzz.py b/tests/test_secret_redaction_fuzz.py index 01905cd..2205e2a 100644 --- a/tests/test_secret_redaction_fuzz.py +++ b/tests/test_secret_redaction_fuzz.py @@ -57,7 +57,10 @@ def _value(rng, secret, allow_bare=True): if rng.random() < 0.25: body = "{x}_" + secret # interpolation inflates the length if rng.random() < 0.2: - body = secret + "\nsecond line" # the literal spans lines + # The secret goes on either side of the break: a continuation line is + # exactly where masking that stops at the opener's line loses it. + body = (secret + "\nsecond line" if rng.random() < 0.5 + else "first line\n" + secret) quote = rng.choice(QUOTES) terminated = rng.random() < 0.8 return rng.choice(PREFIXES) + quote + body + (quote if terminated else "") From d7903398a38828f936f78426761da022a6f423c0 Mon Sep 17 00:00:00 2001 From: lelia <2418071+lelia@users.noreply.github.com> Date: Fri, 18 Sep 2026 22:43:24 -0400 Subject: [PATCH 11/11] fix(redaction): fail closed on ambiguous credential syntax --- CHANGELOG.md | 27 +++-- socket_basics/core/utils/redaction.py | 151 +++++++++++++++++--------- tests/test_secret_redaction.py | 24 ++++ tests/test_secret_redaction_fuzz.py | 7 +- 4 files changed, 144 insertions(+), 65 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 273daeb..c22a108 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,20 +21,22 @@ No configuration change is required, but findings differ on the first run after upgrading. - **`codeSnippet` content changes for the credential rules.** The field keeps - the assignment target, the syntax, the file and the line, and masks the - literal. A baseline keyed on exact snippet text will not match; key on rule ID - plus location instead. Rules whose match is not a credential are unaffected. - (#119) + the assignment target and surrounding syntax where that is unambiguous, as + well as the file and line, and masks the literal. A baseline keyed on exact + snippet text will not match; key on rule ID plus location instead. Rules whose + match is not a credential are unaffected. (#119) - **The same applies to `detailedReport.content` and `dataflowTrace`.** Both quote source lines and both are masked on the same terms. (#119) - **A finding's `description` can also change.** OpenGrep expands metavariables into a rule's message before returning a result, so a message quoting the matched value carried it too. Expanded metavariables are masked for the credential rules. (#119) -- **Masking is deliberately conservative in two visible places.** +- **Masking is deliberately conservative in several visible places.** `define('SECRET', '...')` masks the constant name along with the value, and - `password: "admin"` hides which default was used. Rule ID, file and line still - identify the finding in both cases. (#119) + `password: "admin"` hides which default was used. Ambiguous unquoted values + can also mask the rest of a statement or line rather than risk treating part + of the credential as source syntax. Rule ID, file and line still identify the + finding in each case. (#119) ### Fixed - **A finding's snippet no longer reproduces the value it reports.** A SAST @@ -42,14 +44,15 @@ upgrading. rule that line is the code the finding is about; for the hardcoded-credential rules it contains the credential, so the finding carried the value into `.socket.facts.json`, the uploaded facts and the configured notifiers. - Snippets for those rules now keep the assignment target, the syntax, the file - and the line, and mask the literal's contents. This covers 20 rules across all - fifteen bundled language rule sets, not only the Python and JavaScript ones: + Snippets for those rules now keep the assignment target and syntax when safe, + keep the file and line, and mask the literal's contents. This covers 20 rules + across all fifteen bundled language rule sets, not only the Python and + JavaScript ones: `*-hardcoded-secret(s)`, `*-hardcoded-credentials`, `*-hardcoded-password-default`, `*-default-credentials`, `*-plain-text-password`, `*-weak-jwt-secret` and `*-empty-password`. Rules - whose match is logic keep their snippets verbatim, and an assigned value that - calls something is treated as code, so + whose match is logic keep their snippets verbatim, and a complete assigned + call with a literal argument is treated as code, so `user.password = request.form.get('password')` keeps its expression while the quoted argument is masked. (#119) - Every snippet, dataflow-trace step, rule message and detailed report, whatever diff --git a/socket_basics/core/utils/redaction.py b/socket_basics/core/utils/redaction.py index 10148da..47b2f2e 100644 --- a/socket_basics/core/utils/redaction.py +++ b/socket_basics/core/utils/redaction.py @@ -127,10 +127,11 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # ``==`` leaves the rest of it heading the value, which then does not look # quoted, so a Go short declaration would be starred out whole. A comparison # assigns nothing and does not match here at all. -# - The last operator on the line binds. Taking the colon of +# - The last operator before a real literal binds. Taking the colon of # ``password: str = "..."`` leaves ``str = "..."`` as the value, so an # annotated declaration -- ordinary Python and TypeScript -- would never -# reach the literal pass. +# reach the literal pass. Without a literal, the first operator wins so an +# ``=`` or ``:`` inside an unquoted credential stays in the masked value. # - An operator inside a string literal is not an operator. The ``:`` in # ``url = "https://..."`` would otherwise bind and star out the URL. # @@ -146,11 +147,10 @@ def mask_value(value: Any, reveal: int = _DEFAULT_REVEAL, # operator binds per statement. _STATEMENT_SEPARATOR = re.compile(r';') -# An assigned value that *opens* with a call is an expression rather than a bare -# credential, so the literal pass handles it instead of the unquoted fallback. -# Anchored deliberately: matching a call anywhere would let a trailing comment -# such as ``# see get_secret()`` disable masking for the value in front of it. -_CALL_EXPRESSION = re.compile(r'^[\w.\[\]]+\s*\(') +# A complete call with a literal argument is code rather than a bare credential, +# so the literal pass can handle it without erasing the expression. Calls with +# no literal stay on the fail-closed path: a config value can look call-shaped. +_CALL_EXPRESSION = re.compile(r'^[\w.\[\]]+\s*\(.*\)$') # A value that opens a string literal, allowing the usual raw/bytes/format/ # unicode prefixes. The prefix has to be recognized here: treating ``r"""...`` @@ -223,28 +223,38 @@ def scrub_tokens(text: Any) -> str: return scrubbed -def _split_assignment(code: str, offset: int, in_literal) -> 'tuple[str, str, str] | None': +def _split_assignment(code: str, offset: int, in_literal, + literal_start_within) -> 'tuple[str, str, str, str] | None': """Split a statement at the assignment operator that binds, if it has one. - Returns ``(head, value, trailing_whitespace)``, where ``head`` runs through - the operator and any space after it. Operators covered by a string literal - are skipped, and the last of the rest wins. + Returns ``(head, value, trailing_whitespace, operator)``, where ``head`` + runs through the operator and any space after it. Operators covered by a + string literal are skipped. The last operator before the first real literal + wins; with no literal, the first operator wins so punctuation inside an + unquoted credential cannot be mistaken for syntax. """ - chosen = None + operators = [] for match in _ASSIGNMENT_OPERATOR.finditer(code): if in_literal(offset + match.start()): continue - chosen = match - if chosen is None: + operators.append(match) + if not operators: return None + literal_at = literal_start_within(offset, offset + len(code)) + before_literal = [ + match for match in operators + if literal_at is not None and offset + match.end() <= literal_at + ] + chosen = before_literal[-1] if before_literal else operators[0] + rest = code[chosen.end():] value = rest.lstrip() if not value: return None head = code[:chosen.end()] + rest[:len(rest) - len(value)] stripped = value.rstrip() - return head, stripped, value[len(stripped):] + return head, stripped, value[len(stripped):], chosen.group(0) def _split_statements(code: str, offset: int, in_literal) -> 'list[tuple[str, int]]': @@ -266,53 +276,90 @@ def _split_statements(code: str, offset: int, in_literal) -> 'list[tuple[str, in return pieces +def _mask_outside_literals(text: str, offset: int, in_literal, + prefix: int = 0) -> str: + """Mask text except a trusted prefix and recognized literal spans.""" + return ''.join( + char if index < prefix or in_literal(offset + index) else '*' + for index, char in enumerate(text) + ) + + def _mask_statement(code: str, offset: int, in_literal, literal_opens_at, - literal_start_within) -> str: - """Mask the assigned value in a single statement.""" - split = _split_assignment(code, offset, in_literal) + literal_start_within) -> 'tuple[str, bool]': + """Mask one statement and report whether it contained an assignment.""" + split = _split_assignment(code, offset, in_literal, literal_start_within) if not split: - return code - head, value, trailing = split - - # A value that opens with a call is code rather than a credential: - # ``request.form.get('password')`` is the finding, and starring it leaves - # nothing to act on. A quoted value is the literal pass's job -- but only - # where the quote opens a literal that pass can find. A snippet cut mid - # string has an opening quote and no closing one, so nothing matches and - # the value would survive untouched. - opener = _LITERAL_OPENER.match(value) - opens_literal = bool(opener) and literal_opens_at( - offset + len(head) + opener.start('quote'), len(opener.group('quote')) - ) - if opens_literal or _CALL_EXPRESSION.match(value): - return code + return code, False + head, value, trailing, operator = split - # Anything else is masked whole. Where the value ends is unknowable here, - # and measuring it together with what follows would reveal the head of a - # short credential. - # - # Unless the value contains a literal that runs past this line. Starring it - # would destroy the opening quote, and the masking pass -- which runs over - # the whole snippet afterwards -- would then no longer match, leaving the - # rest of that literal untouched on its continuation lines. Mask up to the - # opener and let the pass have the literal itself. value_start = offset + len(head) literal_at = literal_start_within(value_start, value_start + len(value)) + + # A complete code assignment that calls something with a literal argument + # remains readable; the literal pass masks the argument. The literal is + # required because an unquoted config value can otherwise look like a call. + # Colon-delimited values stay conservative for the same reason. + call_with_literal = all(( + operator in ('=', ':='), + literal_at is not None, + _CALL_EXPRESSION.match(value), + )) + if call_with_literal: + return code, True + + # Preserve real literal spans for the literal pass, while masking every + # adjacent character. Keeping the suffix wholesale would expose values such + # as ``abc"decoy"hunter2``; masking the opening quote would instead break a + # multiline literal and leave its continuation unprotected. + opener = _LITERAL_OPENER.match(value) + opens_literal = bool(opener) and literal_opens_at( + value_start + opener.start('quote'), len(opener.group('quote')) + ) if literal_at is not None: - keep_from = literal_at - value_start - return f"{head}{'*' * keep_from}{value[keep_from:]}{trailing}" - return f"{head}{'*' * len(value)}{trailing}" + prefix = opener.start('quote') if opens_literal else 0 + masked_value = _mask_outside_literals( + value, value_start, in_literal, prefix + ) + return f"{head}{masked_value}{trailing}", True + + # With no literal there is no syntax worth guessing at. Mask the whole value + # so embedded operators and call-shaped config values cannot escape. + return f"{head}{'*' * len(value)}{trailing}", True def _mask_code(code: str, offset: int, in_literal, literal_opens_at, literal_start_within) -> str: - """Mask the assigned value in every statement on one line of code.""" - return ''.join( - piece if piece == ';' - else _mask_statement(piece, piece_offset, in_literal, literal_opens_at, - literal_start_within) - for piece, piece_offset in _split_statements(code, offset, in_literal) - ) + """Mask a line of code without trusting ambiguous statement boundaries.""" + handled = [] + has_assignment = False + for piece, piece_offset in _split_statements(code, offset, in_literal): + if piece == ';': + handled.append((piece, piece_offset, piece, False, True)) + continue + masked, assigned = _mask_statement( + piece, piece_offset, in_literal, literal_opens_at, + literal_start_within + ) + handled.append((piece, piece_offset, masked, assigned, False)) + has_assignment = has_assignment or assigned + + # Once a line contains an assignment, every other semicolon fragment is + # ambiguous: it may be another statement, or it may be part of an unquoted + # config value. Keep the first assignment readable, preserve literal spans + # for the later masking pass, and fail closed on every other fragment. + if has_assignment: + first_assignment = next( + index for index, item in enumerate(handled) if item[3] + ) + return ''.join( + original if separator + else masked if index == first_assignment or not original.strip() + else _mask_outside_literals(original, piece_offset, in_literal) + for index, (original, piece_offset, masked, _, separator) + in enumerate(handled) + ) + return ''.join(masked for _, _, masked, _, _ in handled) def _split_comment(line: str, offset: int, in_literal) -> 'tuple[str, str, str]': diff --git a/tests/test_secret_redaction.py b/tests/test_secret_redaction.py index 28816dc..07df5e1 100644 --- a/tests/test_secret_redaction.py +++ b/tests/test_secret_redaction.py @@ -229,6 +229,15 @@ def test_an_operator_inside_a_literal_does_not_bind(self): def test_a_value_containing_an_operator_is_masked(self): assert "hunter2" not in redact_literals('password = "a=b:c hunter2"') + def test_operators_inside_an_unquoted_value_do_not_rebind(self): + for line in ( + "password: hunter2=foo", + "password: hunter2:foo", + "password = hunter2=foo", + ): + redacted = redact_literals(line) + assert "hunter2" not in redacted, redacted + def test_a_line_with_no_value_after_the_operator_is_left_alone(self): assert redact_literals("password =") == "password =" @@ -266,6 +275,12 @@ def test_each_statement_on_a_line_is_masked(self): def test_a_separator_inside_a_literal_is_part_of_the_value(self): assert redact_literals('password = "a;b"') == 'password = "***"' + def test_a_separator_inside_an_unquoted_value_masks_every_fragment(self): + for line in ("password: abc;hunter2", "password: abc;hunter2=foo"): + redacted = redact_literals(line) + assert "hunter2" not in redacted + assert redacted.startswith("password: ***;") + def test_an_operator_in_a_comment_does_not_bind(self): # The comment's `=` is later in the line than the real one, so binding # it would leave the credential in the head. @@ -282,6 +297,10 @@ def test_the_comment_marker_survives_so_the_line_still_reads(self): "password = ******* #" ) + def test_a_call_shaped_unquoted_value_is_masked(self): + for line in ("password: hunter2(foo)", "password = hunter2(foo)"): + assert "hunter2" not in redact_literals(line), line + class TestMultilineAndUnterminatedLiterals: """Literal spans are a property of the snippet, not of one line. @@ -342,6 +361,11 @@ def test_a_prefixed_single_line_literal_keeps_its_syntax(self): assert "SuperSecret123!" not in redacted assert redacted.startswith("password = r'") + def test_text_adjacent_to_a_literal_is_masked_too(self): + for line in ('password: r""hunter2', 'password: abc"decoy"hunter2'): + redacted = redact_literals(line) + assert "hunter2" not in redacted, redacted + def test_an_interpolated_body_is_masked_whole(self): # The literal text around a placeholder inflates the body past the # partial-reveal threshold, which would expose the tail of the value. diff --git a/tests/test_secret_redaction_fuzz.py b/tests/test_secret_redaction_fuzz.py index 2205e2a..adbdcda 100644 --- a/tests/test_secret_redaction_fuzz.py +++ b/tests/test_secret_redaction_fuzz.py @@ -39,6 +39,11 @@ "{value}", "get_secret({value})", "a if b else {value}", "{value} + other", "other + {value}", "[{value}]", "f({value}, x)", ] +BARE_VALUES = [ + "{secret}", "{secret}=suffix", "prefix:{secret}", "prefix;{secret}", + "prefix;{secret}=suffix", "{secret}(arg)", + 'prefix"decoy"{secret}', 'r""{secret}', +] def _value(rng, secret, allow_bare=True): @@ -50,7 +55,7 @@ def _value(rng, secret, allow_bare=True): choices.append("bare") style = rng.choice(choices) if style == "bare": - return secret + return rng.choice(BARE_VALUES).format(secret=secret) if style == "wrapped": return rng.choice(WRAPPERS).format(value=_value(rng, secret, allow_bare=False)) body = secret