Build/Test Tools: Run the full PHPUnit matrix by label

[lancewillett]

Trac ticket: https://core.trac.wordpress.org/ticket/66071

Adding the Full PHPUnit Matrix label starts the weekly PHP and database matrix on an eligible PR without another code push. New commits keep using the full matrix while the label remains. Runs appear as PHPUnit Tests (full matrix), with checks and logs on the PR.

A separate label-trigger workflow calls the existing PHPUnit workflow. Unrelated labels skip only its uniquely named full-matrix job, preserving existing PHPUnit results. The called workflow manages concurrency; an existing reduced-matrix run continues alongside a label-triggered run.

Both workflows use the same branch and path filters. Contributors without label access can ask a reviewer in a PR comment. Fork approval requirements and individual test-job permissions remain unchanged.

The caller forwards the three reporting secrets explicitly. Its declared actions: write permission accommodates the existing recovery job in the called workflow; that job remains disabled for PR events, and test jobs use contents: read.

Depends on #13447 (https://core.trac.wordpress.org/ticket/66069), which must land first. This branch includes its revised database selection.

Validation

• actionlint, zizmor, and git diff --check passed.
• Matrix expansion confirms 68 ordinary jobs and 182 full jobs, including Gutenberg preparation. Scheduled, label-triggered, and subsequent labeled-push rows match exactly.
• Verified matching path/branch filters, PR event guards, skipped unrelated-label behavior, and secret/permission declarations.
• Live validation on d75e75a: the 68-job reduced run continued alongside the 182-job full run. Adding an unrelated label produced exactly one skipped wrapper job, without replacing or cancelling either test run. The test suites are still in progress; workflow lint passed.

Use of AI tools

AI assistance: Yes
Tool(s): Codex
Model(s): GPT-6, GPT-5.6
Used for: Workflow implementation, review, validation, and PR description/replies. I take responsibility for the submitted change.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props lancewillett, lucatume.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[Copilot]

🟡 Changes recommended

The workflow file has a YAML indentation error in the MariaDB matrix include block that would break parsing/execution.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR extends the existing PHPUnit GitHub Actions workflow so maintainers can trigger the weekly-sized PHP+DB test matrix on an in-repo pull request by applying a Full PHPUnit Matrix label, and documents the behavior for contributors.

Changes:

• Adds a PR label-trigger (pull_request labeled) and updates naming/concurrency so label-only runs can start without new commits and unrelated label events don’t cancel active test runs.
• Expands the PHP and database matrices to match the scheduled (weekly) set when the Full PHPUnit Matrix label is present.
• Documents how to request and interpret full-matrix runs in tests/phpunit/README.md.

File summaries

File	Description
.github/workflows/phpunit-tests.yml	Adds label-triggered full-matrix behavior, updates run naming, concurrency grouping, and matrix selection logic.
tests/phpunit/README.md	Documents how to use the Full PHPUnit Matrix label to request full runs on PRs.

Review details

Suppressed comments (4)

.github/workflows/phpunit-tests.yml:217

• The test-with-mariadb matrix include: block is misindented: the comment and - os: entries are aligned with include: instead of being nested under it. This breaks the YAML structure (and differs from the correctly-indented test-with-mysql include block at phpunit-tests.yml:125-128).

        db-version: ${{ ( github.event_name == 'schedule' || contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) ) && fromJSON('["5.5","10.3","10.5","10.6","10.11","11.4","11.8"]') || fromJSON('["5.5","10.11","11.4","11.8"]') }}
        multisite: [ false, true ]
        memcached: [ false ]

        include:

.github/workflows/phpunit-tests.yml:120

• The MySQL matrix expands based on contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) without a github.event_name == 'pull_request' guard. This workflow also runs on push and workflow_dispatch, and other workflows guard label contains(...) checks with github.event_name == 'pull_request' (see workflow-lint.yml:39-44).

        # Scheduled and labeled runs test every supported PHP version. Other runs test the highest and lowest of each major.
        php: ${{ ( github.event_name == 'schedule' || contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) ) && fromJSON('["7.4","8.0","8.1","8.2","8.3","8.4","8.5"]') || fromJSON('["7.4","8.0","8.5"]') }}
        db-type: [ 'mysql' ]
        # Scheduled and labeled runs test the full database matrix. Other runs test the oldest listed version and supported LTS releases.
        db-version: ${{ ( github.event_name == 'schedule' || contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) ) && fromJSON('["5.7","8.0","8.4","9.7"]') || fromJSON('["5.7","8.4","9.7"]') }}

.github/workflows/phpunit-tests.yml:278

• The innovation-release job uses the same unguarded contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) pattern in its PHP matrix. To match the repo’s established pattern for label contains(...) checks (e.g. workflow-lint.yml:39-44), guard it with github.event_name == 'pull_request'.

        # Scheduled and labeled runs test every supported PHP version. Other runs test the highest and lowest of each major.
        php: ${{ ( github.event_name == 'schedule' || contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) ) && fromJSON('["7.4","8.0","8.1","8.2","8.3","8.4","8.5"]') || fromJSON('["7.4","8.0","8.5"]') }}

.github/workflows/phpunit-tests.yml:213

• The MariaDB matrix expands based on contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) without a github.event_name == 'pull_request' guard. In this repo, label contains(...) checks are normally scoped to PR events (e.g. workflow-lint.yml:39-44) so other triggers don’t depend on pull_request payload fields.

        # Scheduled and labeled runs test every supported PHP version. Other runs test the highest and lowest of each major.
        php: ${{ ( github.event_name == 'schedule' || contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) ) && fromJSON('["7.4","8.0","8.1","8.2","8.3","8.4","8.5"]') || fromJSON('["7.4","8.0","8.5"]') }}
        db-type: [ 'mariadb' ]
        # Scheduled and labeled runs test the full database matrix. Other runs test the oldest listed version and supported LTS releases.
        db-version: ${{ ( github.event_name == 'schedule' || contains( github.event.pull_request.labels.*.name, 'Full PHPUnit Matrix' ) ) && fromJSON('["5.5","10.3","10.5","10.6","10.11","11.4","11.8"]') || fromJSON('["5.5","10.11","11.4","11.8"]') }}

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[lancewillett]

Added github.event_name == 'pull_request' guards to all six full-matrix label checks (run name and five matrix expressions) in d75e75a. I also checked the suppressed indentation concern: the MariaDB include block is a valid YAML indentless sequence, unchanged from the base. actionlint and zizmor pass, and GitHub has parsed and started the revised workflow.

---

_{Adversarial review · gpt-6}

[lucatume]

labeled can't be filtered by label name in on:, so this creates a run for every label added to every PR. The guard on prepare-gutenberg skips the jobs, but a skipped job still posts a check run, under the same name as the real one and against the same head SHA. Label a PR whose PHPUnit run is red and a fresh set of skipped checks lands on top of it; GitHub reads skipped as passing. No expression in this file can prevent that, since the run is created before any if is evaluated.

I'd move the trigger into its own file that calls this one:

# .github/workflows/phpunit-tests-full-matrix.yml
name: PHPUnit Tests (label)
run-name: PHPUnit Tests${{ github.event.label.name == 'Full PHPUnit Matrix' && ' (full matrix)' || ' (ignored label)' }}

on:
  pull_request:
    types: [ labeled ]
    branches: [ trunk, '3.[7-9]', '[4-9].[0-9]' ]

concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
  cancel-in-progress: true

permissions: {}

jobs:
  full-matrix:
    if: github.event.label.name == 'Full PHPUnit Matrix'
    permissions:
      contents: read
    uses: ./.github/workflows/phpunit-tests.yml
    secrets: inherit

The .github/workflows/phpunit-tests.yml file on: then gains a bare workflow_call: and nothing else. No input is needed: the github context inside a called workflow is the caller's, so github.event.pull_request.labels already carries Full PHPUnit Matrix and every contains( ... ) expression this PR adds keeps working as written. That also keeps the label sticky across pushes, which a full-matrix input would quietly break; a synchronize would come through the direct trigger with the input false and run the reduced matrix, against what the README promises.

secrets: inherit because this file forwards CODEVITALS_PROJECT_TOKEN, CODECOV_TOKEN and WPT_REPORT_API_KEY to its own callees. They're all required: false and their uses are gated, so nothing breaks without it; they'd just resolve empty and differ from a direct run for no good reason.

An unrelated label still creates a run, but it's one skipped job in a file of its own, under a name that never collides with the real checks.

The inline comments cover what comes off this file: the labeled type, the concurrency group and the if on prepare-gutenberg all revert. The matrix trimming and the run-name line stay as they are, and the README needs a word about the reduced run no longer being cancelled.

[lucatume]

Given a trigger in a separate file, this can be removed. At least the labeled trigger type should be removed.

[lancewillett]

Moved the labeled trigger into phpunit-tests-full-matrix.yml and restored the main workflow’s default PR events in d75e75a. The caller passes the label payload through without a matrix input, preserving full runs on later pushes. It forwards the three reporting secrets explicitly because zizmor flags blanket inheritance. The caller also declares the permission ceiling needed by the existing non-PR recovery job; individual test jobs retain contents: read.

---

_{Adversarial review · gpt-6}

[lucatume]

With the trigger in its own file this reverts to the original condition.

[lancewillett]

Restored the original prepare-gutenberg condition in d75e75a. The label-name guard now exists only on the separate caller’s full-matrix job.

---

_{Adversarial review · gpt-6}

[lucatume]

With a trigger in a separate file, this whole line can be reverted: a different file means a different github.workflow, a different group, and nothing gets cancelled. The price is the reduced matrix running alongside its own superset for a while, which is what the README describes anyway.

[lancewillett]

Restored the main concurrency expression in d75e75a and omitted wrapper concurrency entirely. The called workflow manages repeat full runs under the caller’s workflow name, while unrelated labels never enter it. Verified live: the reduced run continued alongside the label-triggered full run. The README now explains that existing reduced results are preserved.

---

_{Adversarial review · gpt-6}

[lucatume]

If the trigger moves to its own file, the simplest thing is to not copy the thirty-odd line paths list across, which makes the label mean "run it regardless of paths". That's probably what someone reaching for the label wants, but then the sentence says the opposite of what happens.

[lancewillett]

Kept the same ordered branch/path filters in both files in d75e75a, preserving the agreed behavior for documentation-only PRs excluded by the normal workflow. Both lists include the new wrapper file and have comments to keep them synchronized. The README describes the shared filters and explains that the existing reduced run continues alongside the full run.

---

_{Adversarial review · gpt-6}