From 524067ba06881af28d4ccc84597774418483997d Mon Sep 17 00:00:00 2001 From: Remco Beckers Date: Thu, 27 Aug 2026 16:35:33 +0200 Subject: [PATCH 1/2] STAC-25553: drop the GitLab PyPI extra-index-url from the DEB build The private index existed for exactly one package. tasks/omnibus.py names it: vsphere-automation-sdk==1.82.0, which VMware withdrew from public PyPI (the name is squatted there by an unrelated 0.0.1 placeholder), so it was mirrored into the GitLab Package Registry and resolved with a read-only credential. That package is gone. stackstate-agent-integrations moved vsphere onto the renamed public VMware packages -- pyvmomi, vmware-vcenter, vmware-vapi-runtime and vmware-vapi-common-client, all 9.1.0.0 -- under STAC-25544, and stackstate-deps.json already pins STACKSTATE_INTEGRATIONS_VERSION 7.78.2-5, which contains that change. The embedded agent_requirements.in it ships no longer mentions vsphere-automation-sdk, and its full transitive closure resolves from public PyPI alone. So the index is dead weight, and not harmless dead weight: pip-compile echoes an authenticated index URL into agent_requirements-py3.txt, which is baked into the published image. That is the shape of the 2026-04-23 credential leak, fixed by STAC-24642 by moving auth into ~/.netrc. Removing the index removes the class of failure rather than mitigating it again, and takes the last PyPI read credential in the estate with it. Removing the env block also leaves this job with no secret in scope at all. The setup script, the tasks/omnibus.py block and the local-development references are left for a follow-up commit once a build confirms the resolve. Refs STAC-25553, STAC-25360 --- .github/workflows/build-deb.yml | 8 -------- 1 file changed, 8 deletions(-) diff --git a/.github/workflows/build-deb.yml b/.github/workflows/build-deb.yml index 6fbfe3a4179a..825037edee99 100644 --- a/.github/workflows/build-deb.yml +++ b/.github/workflows/build-deb.yml @@ -112,10 +112,6 @@ jobs: bazel-${{ matrix.arch }}- - name: Build DEB package with omnibus - env: - GITLAB_PACKAGE_REGISTRY_PYPI_SIMPLE_URL: ${{ vars.GITLAB_PACKAGE_REGISTRY_PYPI_SIMPLE_URL }} - GITLAB_PACKAGE_REGISTRY_USER: ${{ secrets.GITLAB_PACKAGE_REGISTRY_USER }} - GITLAB_PACKAGE_REGISTRY_READONLY_PASSWORD: ${{ secrets.GITLAB_PACKAGE_REGISTRY_READONLY_PASSWORD }} run: | set -eo pipefail export PATH="$PATH:/usr/local/go/bin" @@ -173,10 +169,6 @@ jobs: inv agent.version -u > version.txt cat version.txt - # Writes pip.conf + ~/.netrc for the private Python index. Hard-exits if - # the GITLAB_PACKAGE_REGISTRY_* variables above are unset. - source ./.gitlab-scripts/setup_artifact_registry.sh - export LD_LIBRARY_PATH="/opt/stackstate-agent/embedded/lib/python3.12/site-packages/psycopg2_binary.libs:/opt/stackstate-agent/embedded/lib" # --install-directory pins the omnibus paths to the BRANDED install dir. From 258b972200110f251d8e249c1b357f83ff763fc6 Mon Sep 17 00:00:00 2001 From: Remco Beckers Date: Thu, 27 Aug 2026 17:06:41 +0200 Subject: [PATCH 2/2] STAC-25553: remove the GitLab package registry plumbing left behind The DEB build no longer configures the index, so everything that existed to serve it is now unreferenced. Removing it in one pass rather than leaving a credential-shaped hole for someone to wire back up. tasks/omnibus.py loses the extra-index-url block. local.sh sourced the setup script from two places -- the container env passthrough and the in-container build path at line 199 -- and both go, along with the Makefile env passthrough and .gitlab-scripts/setup_artifact_registry.sh itself. Three of these were already broken and had been for some time, which is the argument for deleting rather than updating them: - Dockerfiles/local_builder/local_init.sh sourced .gitlab-scripts/setup_artifactory.sh, a filename that exists nowhere in this repository. - Development.md told a new engineer to export artifactory_user and artifactory_password, which setup_artifact_registry.sh never read -- it required GITLAB_PACKAGE_REGISTRY_USER and _READONLY_PASSWORD. The documented setup could not have worked as written. - .assistance/Vagrantfile exported the same two variables nothing consumes. omnibus/package-scripts/publish_image.sh had the URL as a dead assignment, never referenced in its 37 lines. The script itself appears to be unused -- nothing invokes it, and the only remaining mention is a comment in build-deb.yml explaining that the workflow inlined its docker build -- but deleting it is a separate claim, so only the assignment goes here. The --no-emit-{index-url,find-links} flags in the integrations recipe stay. They were the fix for the 2026-04-23 credential leak (STAC-24642) and cost nothing; only the comment changes, since it cited the deleted script. Refs STAC-25553, STAC-25360 --- .assistance/Vagrantfile | 3 -- .gitlab-scripts/setup_artifact_registry.sh | 41 ------------------- Development.md | 10 ----- Dockerfiles/local_builder/local_init.sh | 1 - Makefile | 3 -- local.sh | 4 -- .../stackstate-agent-integrations-py3.rb | 8 ++-- omnibus/package-scripts/publish_image.sh | 1 - tasks/omnibus.py | 13 ------ 9 files changed, 3 insertions(+), 81 deletions(-) delete mode 100644 .gitlab-scripts/setup_artifact_registry.sh diff --git a/.assistance/Vagrantfile b/.assistance/Vagrantfile index 154203b15e99..814206b97b2f 100644 --- a/.assistance/Vagrantfile +++ b/.assistance/Vagrantfile @@ -45,9 +45,6 @@ Vagrant.configure("2") do |config| echo "export GOLANG_VERSION=1.14" >> /home/vagrant/.env echo "export RVM_VERSION=1.29.12" >> /home/vagrant/.env echo "export RUBY_VERSION=2.4.2" >> /home/vagrant/.env - echo "export artifactory_user=$artifactory_user" >> /home/vagrant/.env - echo "export artifactory_password=$artifactory_password" >> /home/vagrant/.env - echo "export GITLAB_PACKAGE_REGISTRY_PYPI_SIMPLE_URL=gitlab.com/api/v4/projects/71271774/packages/pypi/simple" >> /home/vagrant/.env echo "export HOST_DIR=/home/vagrant/stackstate-agent-host" >> /home/vagrant/.env echo "export CI_PROJECT_DIR=/home/vagrant/stackstate-agent" >> /home/vagrant/.env echo "export C_MAKE_SETUP_PATH=~/cmake" >> /home/vagrant/.env diff --git a/.gitlab-scripts/setup_artifact_registry.sh b/.gitlab-scripts/setup_artifact_registry.sh deleted file mode 100644 index f13aeeb392fd..000000000000 --- a/.gitlab-scripts/setup_artifact_registry.sh +++ /dev/null @@ -1,41 +0,0 @@ -## python artifactory dependency -# pip.conf gets the URL only — no credentials interpolated into the URL. -# Auth goes into ~/.netrc instead, which pip / pip-compile / pip-tools all -# honour for HTTP basic auth without echoing credentials into output -# requirements files. See STAC-24642. - -missing= -for var in GITLAB_PACKAGE_REGISTRY_PYPI_SIMPLE_URL GITLAB_PACKAGE_REGISTRY_USER GITLAB_PACKAGE_REGISTRY_READONLY_PASSWORD; do - if [ -z "${!var}" ]; then - missing="$missing $var" - fi -done -if [ -n "$missing" ]; then - echo "ERROR: Required environment variables not set:$missing" >&2 - exit 1 -fi - -mkdir -p ~/.pip -cat > ~/.pip/pip.conf < ~/.netrc <`, # there's no need to refer to `pip`, the interpreter will pick the right script. # --no-emit-{index-url,find-links}: defence in depth so the resolved - # requirements file does not echo any auth-bearing URL from pip.conf - # back into the image. Source-side fix is in tasks/agent.py and - # .gitlab-scripts/setup_artifactory.sh (STAC-24642); these flags - # prevent future regressions reaching the image even if a pip.conf - # with credentials-in-URL is reintroduced upstream. + # requirements file does not echo any auth-bearing URL from pip.conf back + # into the image (STAC-24642). No authenticated index is configured any more + # (STAC-25553), so these flags now only guard against one being reintroduced. if windows? command "#{python} -m pip install --no-deps #{windows_safe_path(project_dir)}\\stackstate_checks_base" diff --git a/omnibus/package-scripts/publish_image.sh b/omnibus/package-scripts/publish_image.sh index 47a42eaac37e..1858c971bf67 100755 --- a/omnibus/package-scripts/publish_image.sh +++ b/omnibus/package-scripts/publish_image.sh @@ -8,7 +8,6 @@ DOCKERFILE_PATH="${3}" EXTRA_TAG="${4}-${ARCH}" REGISTRY="quay.io" ORGANIZATION="stackstate" -GITLAB_PACKAGE_REGISTRY_PYPI_SIMPLE_URL="https://gitlab.com/api/v4/projects/71271774/packages/pypi/simple" S6_ARCH="${5}" echo "IMAGE_TAG=${IMAGE_TAG}" diff --git a/tasks/omnibus.py b/tasks/omnibus.py index 0cd886d750ec..0274975fe9ae 100644 --- a/tasks/omnibus.py +++ b/tasks/omnibus.py @@ -261,19 +261,6 @@ def build( # If a python_mirror is set then use it for pip by adding it in the pip.conf file pip_index_url = f"[global]\nindex-url = {python_mirror}" if python_mirror else "" - # [sts] Add private GitLab PyPI registry as extra-index-url if credentials are available. - # Credentials are not included in the URL here, they are expected to be in ~/.netrc (see - # .gitlab-scripts/setup_artifact_registry.sh). Without this block, the omnibus-managed - # pip.conf is empty, pip only queries PyPI default, and STS-only packages like - # vsphere-automation-sdk==1.82.0 fail with "Could not find a version that satisfies". - artifact_registry_pypi_url = os.environ.get("GITLAB_PACKAGE_REGISTRY_PYPI_SIMPLE_URL") - if artifact_registry_pypi_url: - python_extra_mirror = f"https://{artifact_registry_pypi_url}" - if pip_index_url: - pip_index_url += f"\nextra-index-url = {python_extra_mirror}" - else: - pip_index_url = f"[global]\nextra-index-url = {python_extra_mirror}" - # We're passing the --index-url arg through a pip.conf file so that omnibus doesn't leak the token with open(pip_config_file, 'w') as f: f.write(pip_index_url)