Pin apm-sdks-benchmarks and add tooling to automatically update pin

[sarahchen6]

What Does This Do

Pin our apm-sdks-benchmarks reference to the current latest commit SHA on main. Add tooling to update this pinned SHA.

Motivation

Currently, any change made to apm-sdks-benchmarks will take immediate effect in this repo. We should pin this reference to protect CI and releases.

Additional Notes

The tooling automation will run once a week at the same frequency in which we update our gradle dependencies. It can also be manually triggered. The PR also goes in tandem with https://github.com/DataDog/apm-sdks-benchmarks/pull/228 (merged) which allows reading apm-sdks-benchmarks repo and https://github.com/DataDog/apm-sdks-benchmarks/pull/230 (waiting on review) which applies the logic to clone the repo at the specified SHA to the post-pr-comment job.

Testing

The tooling/update_apm_sdks_benchmarks_reference.sh script works locally, and the .github/workflows/update-apm-sdks-benchmarks-reference.yaml is based on existing working GitHub workflows. However, the workflow cannot be tested end-to-end until it lands on master because the dd-octo-sts permissions are scoped to and passed from master.

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: [PROJ-IDENT]

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.92 s	13.99 s	[-1.2%; +0.2%] (no difference)
startup:insecure-bank:tracing:Agent	12.86 s	12.94 s	[-1.2%; +0.0%] (no difference)
startup:petclinic:appsec:Agent	17.38 s	17.05 s	[+1.0%; +2.9%] (maybe worse)
startup:petclinic:iast:Agent	16.81 s	17.39 s	[-7.6%; +0.9%] (no difference)
startup:petclinic:profiling:Agent	17.24 s	17.30 s	[-1.3%; +0.7%] (no difference)
startup:petclinic:sca:Agent	17.45 s	17.20 s	[+0.3%; +2.5%] (maybe worse)
startup:petclinic:tracing:Agent	16.61 s	16.14 s	[-1.4%; +7.2%] (no difference)

Commit: 7470e12b · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[datadog-official]

More details

The new updater correctly replaces all five pinned GitLab includes and the runtime clone SHA, rejects malformed commit inputs, and is idempotent when rerun. Workflow syntax parsed successfully and its token, checkout, commit-headless, and PR steps match existing repository automation; no concrete behavioral regression was identified.

Was this helpful? React 👍 or 👎

_{📊 Validated against 7 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 761a752 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 761a752366

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[AlexeyKuznetsov-DD]

LGTM, but I proposed optional improvement to use references like *apm_sdks_benchmarks_sha that will reduce future PRs to one line.

[PerfectSlayer]

👏 praise: ‏Nice! Thanks for updating the workflow readme and crafting a nice PR from the update automation ✨

[sarahchen6]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-08-07 18:14:12 UTC ℹ️ Start processing command /merge

---

2026-08-07 18:14:17 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

---

2026-08-07 19:08:52 UTC ℹ️ MergeQueue: This merge request was merged