Skip to content

chore(licensing): add per-module NOTICE-binary generation script and CI checks for detecting NOTICE-binary drifting #5417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Yicong-Huang merged 6 commits into from
Jun 10, 2026
+8,808 −4,932
Merged

chore(licensing): add per-module NOTICE-binary generation script and CI checks for detecting NOTICE-binary drifting #5417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5fd5b8d
chore(licensing): auto-generate per-module NOTICE-binary from jars' M…
Jun 6, 2026
980b041
Merge branch 'main' into feat/auto-generate-notice-binary
bobbai00 Jun 7, 2026
4e0ae68
chore(licensing): omit versions and per-jar listing from NOTICE-binary
Jun 7, 2026
df729a9
chore(licensing): address review feedback on NOTICE-binary tooling
Jun 7, 2026
319bb2f
Merge branch 'main' into feat/auto-generate-notice-binary
bobbai00 Jun 7, 2026
7319ca0
Merge branch 'main' into feat/auto-generate-notice-binary
bobbai00 Jun 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 33 additions & 9 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,12 +229,17 @@ jobs:
sbt scalafmtCheckAll \
"scalafixAll --check" \
WorkflowExecutionService/dist
- name: Unzip amber dist and check binary licenses
# Per-module LICENSE-binary files live at the repo root after #4668;
# the amber JVM dist is checked against amber/LICENSE-binary-java.
# The audit always runs (mirroring the previous 'if: always()' on its
# own step) and never fails the step; the binding check's exit code
# drives it.
- name: Unzip amber dist and check binary LICENSE + NOTICE
# Per-module LICENSE-binary files live at the repo root after #4668.
# One step covers the amber dist's third-party content end to end:
# * LICENSE-binary-java — check bundled jars against the claimed
# licenses (audit always runs and never fails the step);
# * NOTICE-binary — regenerate from the same unzipped lib dir
# via generate_notice_binary.py (plus amber/NOTICE-binary-python for
# the non-jar python attributions) and diff against the committed
# file. Drift means a dep changed without rerunning the generator —
# fix by running:
# ./bin/licensing/generate_notice_binary.py amber/NOTICE-binary /tmp/dists/amber-*/lib --extras amber/NOTICE-binary-python
run: |
set -euo pipefail
mkdir -p /tmp/dists
Expand All @@ -245,6 +250,12 @@ jobs:
--license-binary amber/LICENSE-binary-java \
/tmp/dists/amber-*/lib || check_exit=$?
./bin/licensing/audit_jar_licenses.py /tmp/dists/amber-*/lib || true

./bin/licensing/generate_notice_binary.py /tmp/notice-amber.txt /tmp/dists/amber-*/lib --extras amber/NOTICE-binary-python
if ! diff -u amber/NOTICE-binary /tmp/notice-amber.txt; then
echo "::error::amber/NOTICE-binary differs from generator output. Regenerate with bin/licensing/generate_notice_binary.py and commit."
check_exit=1
fi
exit "$check_exit"
- name: Create texera_db_for_test_cases
run: psql -h localhost -U postgres -v DB_NAME=texera_db_for_test_cases -f sql/texera_ddl.sql
Expand Down Expand Up @@ -589,9 +600,16 @@ jobs:
# Single sbt invocation so dist + test share compiled state. Use
# `jacoco` so the codecov upload step has a report to pick up.
run: sbt "${{ matrix.sbt_project }}/dist" "${{ matrix.sbt_project }}/jacoco"
- name: Unzip ${{ matrix.service }} dist and check binary licenses
# Each platform service has its own LICENSE-binary at the repo root
# after #4668; check this service's dist against just its own file.
- name: Unzip ${{ matrix.service }} dist and check binary LICENSE + NOTICE
# Each platform service has its own LICENSE-binary / NOTICE-binary at
# the repo root after #4668. One step covers both for this service's
# dist:
# * LICENSE-binary — check bundled jars against the claimed licenses
# (audit always runs and never fails the step);
# * NOTICE-binary — regenerate from the same unzipped lib dir via
# generate_notice_binary.py and diff against the committed file.
# Drift means a dep changed without rerunning the generator — fix:
# ./bin/licensing/generate_notice_binary.py ${{ matrix.service }}/NOTICE-binary /tmp/dists/${{ matrix.service }}-*/lib
run: |
set -euo pipefail
mkdir -p /tmp/dists
Expand All @@ -602,6 +620,12 @@ jobs:
--license-binary ${{ matrix.service }}/LICENSE-binary \
/tmp/dists/${{ matrix.service }}-*/lib || check_exit=$?
./bin/licensing/audit_jar_licenses.py /tmp/dists/${{ matrix.service }}-*/lib || true

./bin/licensing/generate_notice_binary.py /tmp/notice-${{ matrix.service }}.txt /tmp/dists/${{ matrix.service }}-*/lib
if ! diff -u ${{ matrix.service }}/NOTICE-binary /tmp/notice-${{ matrix.service }}.txt; then
echo "::error::${{ matrix.service }}/NOTICE-binary differs from generator output. Regenerate with bin/licensing/generate_notice_binary.py and commit."
check_exit=1
fi
exit "$check_exit"
- name: Upload ${{ matrix.service }} coverage to Codecov
# Per-service flag so each matrix entry has its own Codecov view
Expand Down
1 change: 1 addition & 0 deletions .licenserc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ header:
- '**/LICENSE-binary'
- '**/LICENSE-binary-*'
- '**/NOTICE-binary'
- '**/NOTICE-binary-*'
- '.dockerignore'
- '.gitattributes'
- '.github/PULL_REQUEST_TEMPLATE'
Expand Down
Loading
Loading