diff --git a/examples/multi-agent-notepad/.gitignore b/examples/multi-agent-notepad/.gitignore new file mode 100644 index 0000000000..6a5d5848e9 --- /dev/null +++ b/examples/multi-agent-notepad/.gitignore @@ -0,0 +1 @@ +run_demo.sh diff --git a/examples/multi-agent-notepad/README.md b/examples/multi-agent-notepad/README.md index 12d0676318..5cc1b3819d 100644 --- a/examples/multi-agent-notepad/README.md +++ b/examples/multi-agent-notepad/README.md @@ -3,67 +3,99 @@ # Multi-Agent Shared Notepad Demo -Launch multiple coding agents in parallel OpenShell sandboxes and let them use -a shared markdown notepad. This version uses Codex as the agent runtime and -GitHub as the durable notes backend. +Run multiple Codex coding agents in parallel OpenShell sandboxes, with a +GitHub repository as the durable shared notepad they coordinate through. + +## Why GitHub as a shared notepad? + +Long-running agents that produce artifacts — research notes, memory entries, +reports, decision logs — need somewhere durable to write them. The store has +to survive individual sandbox death, accept many concurrent writers safely, +and stay inspectable after the fact. GitHub provides all three for free: + +- **Durable across sandbox restarts.** A sandbox can crash mid-run; the + committed artifact persists. +- **Concurrency control built in.** Every PUT must include the current file + SHA, and the branch ref serializes commits. Racing writers see HTTP 409 + and retry instead of silently overwriting each other. +- **Auditable.** Every write is a commit with author, message, diff, and + timestamp. Free observability. +- **Reviewable.** Humans review agent output with the same PR/diff tooling + they already use for code. +- **No new infra.** Most teams already have a GitHub org. + +The pattern works best when artifacts are markdown or other text (so diffs +are useful) and when write rates are moderate (commits per minute, not per +second). For higher rates or structured queries, graduate to a real +datastore — see [Beyond map/reduce](#beyond-mapreduce-memory-architecture-variants) +below. + +## What this demo shows + +The simplest useful shape: **map/reduce**. N worker agents fan out and write +one note each; one synthesis agent reads them and writes a summary. -This example demonstrates three OpenShell ideas together: +```text +runs//notes/agent-1.md ← worker 1 writes +runs//notes/agent-2.md ← worker 2 writes +... +runs//summary.md ← synthesis agent writes +``` -- multiple isolated agents can run at the same time, -- agents can coordinate through durable shared notes without sharing a - filesystem, -- Codex OAuth can use provider-backed placeholders instead of storing real - OAuth tokens in the sandbox filesystem. +Each worker gets a different research angle on the same topic. Workers share +neither filesystem nor container — only the GitHub repository. -GitHub is the backing store in this example because it is familiar, durable, -branch-aware, and easy to inspect. The pattern is the important part: separate -coding agents communicate by reading and writing scoped markdown notes. +The demo also exercises two OpenShell features: -## Prerequisites +- **Provider-backed credentials.** Sandboxes get placeholders, not the real + Codex OAuth tokens or the GitHub token. The proxy resolves them at the + network boundary. +- **Scoped network policy.** Each sandbox can only `GET` and `PUT` paths under + `/repos///contents/runs//**`. -- OpenShell CLI from current `main` with provider env lookup and `--upload` - support. If it is not on `PATH`, set `OPENSHELL_BIN` to the binary path. -- A running OpenShell gateway: +## Files in this example - ```bash - openshell gateway start - ``` +- `demo.sh` — host orchestration. Validates env, creates providers, launches + sandboxes, waits for completion. Read this to understand how the run is + driven from the host. +- `runner.sh` — the script that runs **inside each sandbox**. Bootstraps + Codex OAuth, calls `codex exec`, and writes the result to GitHub. Read this + to understand the agent side. +- `policy.template.yaml` — the network policy applied to every sandbox in + the run. Renders to a concrete policy with the configured owner, repo, and + run id. +- `prompts/worker.md`, `prompts/synthesis.md` — agent instructions. -- Local Codex sign-in on the host machine: - - ```bash - codex login - codex login status - ``` +## Prerequisites -- `jq` on the host machine -- A disposable or demo-only GitHub repository to use as the shared notepad -- A GitHub token with permission to write repository contents +- OpenShell CLI from current `main` (or set `OPENSHELL_BIN` to the binary + path) +- A running OpenShell gateway: `openshell gateway start` +- Local Codex sign-in on the host: `codex login` +- `gh` (GitHub CLI) signed in, **or** a GitHub PAT with `contents:write` +- `jq` on the host +- A disposable or demo-only GitHub repository -Use an empty repository or one created specifically for this demo. The script -writes under `runs//` and will update files at those paths if they -already exist. Do not point the demo at a production repository unless you are -comfortable with it creating and updating files under that prefix. +The demo writes under `runs//`. Use a repository created specifically +for this demo, or one you're comfortable with the demo creating files in. -## Quick Start +## Quick start ```bash export DEMO_GITHUB_OWNER= export DEMO_GITHUB_REPO= -export DEMO_GITHUB_TOKEN= +export DEMO_GITHUB_TOKEN="$(gh auth token)" bash examples/multi-agent-notepad/demo.sh ``` -If you use the GitHub CLI, you can use your signed-in GitHub session: - -```bash -export DEMO_GITHUB_TOKEN="$(gh auth token)" -``` +`gh auth token` returns a token with whatever scopes you logged in with — +usually broader than `contents:write`. If you'd rather use a scope-limited +PAT, set `DEMO_GITHUB_TOKEN` to that value instead. By default the script launches five worker agents and one synthesis agent in -the OpenShell `base` sandbox image, where Codex is preinstalled. -To run a faster smoke test: +the OpenShell `base` image, where Codex is preinstalled. To run a faster +smoke test: ```bash export DEMO_AGENT_COUNT=2 @@ -80,64 +112,93 @@ export DEMO_RUN_ID="$(date +%Y%m%d-%H%M%S)" export DEMO_KEEP_SANDBOXES=0 ``` -`DEMO_RUN_ID` is used in sandbox names and policy paths, so keep it to -lowercase letters, numbers, and `-`. -Use a fresh `DEMO_RUN_ID` for each run unless you intentionally want to update -the files from a previous run. +`DEMO_RUN_ID` is used in sandbox names and policy paths, so keep it +lowercase letters, numbers, and `-`. Use a fresh `DEMO_RUN_ID` per run unless +you intentionally want to overwrite a previous run's files. -`DEMO_BRANCH` is used in GitHub API calls and output links. For this demo, use -a simple branch name containing only letters, numbers, `.`, `_`, and `-`. +`DEMO_BRANCH` may contain only letters, numbers, `.`, `_`, and `-`. -If a worker fails, the script prints the relevant log tail and keeps full logs -in a temporary directory. Set `DEMO_KEEP_SANDBOXES=1` when you want to inspect -the sandboxes after the run; temporary providers are still removed. +If a worker fails, the script prints the relevant log tail and keeps full +logs in a temporary directory. Set `DEMO_KEEP_SANDBOXES=1` to inspect +sandboxes after the run; temporary providers are still removed. -## What It Creates +## How credential protection works -The demo creates a small shared notepad for one multi-agent run. Each worker -writes a note, then the synthesis agent reads those notes and writes a summary: +The host script reads your local Codex sign-in and creates a temporary +OpenShell provider for the OAuth tokens. It also creates a temporary +provider for the GitHub token. Sandboxes receive provider placeholders, not +the real credential values. -```text -runs//notes/agent-1.md -runs//notes/agent-2.md -runs//notes/agent-3.md -runs//notes/agent-4.md -runs//notes/agent-5.md -runs//summary.md -``` - -Each worker gets a different research angle for the same topic. Workers never -share a filesystem or container. The GitHub repository is the shared notepad -and coordination layer. - -This is not a general-purpose agent memory system. It is a simple markdown -notepad that isolated agents can use to exchange findings. - -If files for the same `DEMO_RUN_ID` already exist, the demo updates them in -place. +When `codex` or `curl` inside a sandbox sends an authorized request, the +OpenShell proxy resolves the placeholder at the network boundary and +forwards the request upstream with the real credential. The credential +values never sit in the sandbox filesystem. -## How Credential Protection Works +## Network policy -The host script uses your local Codex sign-in to create a temporary OpenShell -provider for Codex OAuth. It also creates a temporary provider for the GitHub -token. Sandboxes receive provider placeholders, not the real credential values. +The script renders `policy.template.yaml` for the configured GitHub +repository and run id. The policy allows: -When Codex or `curl` sends an authorized request, the OpenShell proxy resolves -the placeholder at the network boundary and forwards the request upstream with -the real credential. The credential values do not need to be copied into the -sandbox filesystem. - -## Network Policy - -The script renders `policy.template.yaml` for the configured GitHub repository -and run id. The policy allows: - -- Codex traffic to OpenAI and ChatGPT endpoints used by the community base image -- limited Codex plugin metadata reads from `github.com/openai/plugins.git` -- GitHub REST `GET` and `PUT` calls scoped to: - - ```text - /repos///contents/runs//** - ``` +- Codex traffic to OpenAI and ChatGPT endpoints used by the community base + image +- Limited Codex plugin metadata reads from `github.com/openai/plugins.git` +- GitHub REST `GET` and `PUT` calls scoped to + `/repos///contents/runs//**` The policy does not grant broad GitHub API access. + +## Beyond map/reduce: memory architecture variants + +The pile-and-reduce shape in this demo is one of several useful patterns +for using a Git repository as durable agent memory. Each is a different +tradeoff between contention, complexity, and what you can ask of the +artifact afterwards. + +### Pile (this demo) + +Each agent writes its own file at a unique path. A reducer reads them all +and writes a summary. + +- **Best for:** parallel exploration with bounded fan-in (research, + multi-perspective analysis, code review across files). +- **Contention:** low. Different files have independent SHAs, so the only + source of 409s is branch-ref locking when many commits land at once. The + retry loop in `put_contents` handles it. +- **Extension:** hierarchical reduce — k workers per reducer, log(N) levels — + for runs that exceed what one synthesis agent can chew through. + +### Append journal + +A long-lived agent (or small team) appends entries to a shared `journal.md` +across many sessions. Useful as chronological memory: "decisions made", +"things learned", "open questions". + +- **Best for:** a single agent or a small group continuously writing an + ordered log. +- **Contention:** high if many writers; the GET-PUT-409-retry pattern is + load-bearing here. For high-write-rate journals, switch to the Git Data + API — create blobs concurrently, build one tree, commit once — to avoid + branch-ref serialization. +- **Extension:** split by month (`journal/2026-05.md`) to bound file size and + spread contention across files. + +### Indexed memory + +Agents read and write keyed entries — `memory/.md` per key. Higher- +level agents look up "what do we know about X?" by reading the relevant +file directly. + +- **Best for:** keyed memory that survives across runs and is queried by + topic. +- **Contention:** per-key, which is usually what you want. Hot keys still + benefit from the retry loop. +- **Extension:** maintain a `index.md` listing all keys. The index becomes + the hot spot — update it lazily, or rebuild it from a directory listing + when needed. + +### When to graduate beyond GitHub + +GitHub stops being the right backend when you need sustained write rates +above roughly 10/sec, sub-100ms reads, structured queries, or vector search. +At that point reach for Postgres, a KV store, or a vector DB. The shape of +the agent code stays similar — only the storage primitive changes. diff --git a/examples/multi-agent-notepad/demo.sh b/examples/multi-agent-notepad/demo.sh index c1777ab8aa..7a2e4bf2f3 100755 --- a/examples/multi-agent-notepad/demo.sh +++ b/examples/multi-agent-notepad/demo.sh @@ -100,7 +100,9 @@ validate_env() { [[ "$DEMO_GITHUB_OWNER" =~ ^[A-Za-z0-9_.-]+$ ]] || fail "DEMO_GITHUB_OWNER contains unsupported characters" [[ "$DEMO_GITHUB_REPO" =~ ^[A-Za-z0-9_.-]+$ ]] || fail "DEMO_GITHUB_REPO contains unsupported characters" [[ "$DEMO_BRANCH" =~ ^[A-Za-z0-9._-]+$ ]] || fail "DEMO_BRANCH may contain only letters, numbers, '.', '_', and '-'" + info "GitHub repo ${DEMO_GITHUB_OWNER}/${DEMO_GITHUB_REPO}@${DEMO_BRANCH} and token present" + info "checking OpenShell gateway is reachable..." "$OPENSHELL_BIN" status >/dev/null 2>&1 || fail "OpenShell gateway is not reachable; run: openshell gateway start" export CODEX_AUTH_ACCESS_TOKEN @@ -113,6 +115,7 @@ validate_env() { [[ -n "$CODEX_AUTH_ACCESS_TOKEN" ]] || fail "local Codex sign-in is missing an access token; run: codex login" [[ -n "$CODEX_AUTH_REFRESH_TOKEN" ]] || fail "local Codex sign-in is missing a refresh token; run: codex login" [[ -n "$CODEX_AUTH_ACCOUNT_ID" ]] || fail "local Codex sign-in is missing an account id; run: codex login" + info "local Codex OAuth tokens loaded from ~/.codex/auth.json" } render_policy() { @@ -126,212 +129,7 @@ render_policy() { write_runner() { cp "${PROMPTS_DIR}/worker.md" "${PROMPTS_UPLOAD_DIR}/worker.md" cp "${PROMPTS_DIR}/synthesis.md" "${PROMPTS_UPLOAD_DIR}/synthesis.md" - - cat > "$RUNNER_FILE" <<'EOF' -#!/usr/bin/env bash -set -euo pipefail - -MODE="$1" -OWNER="$2" -REPO="$3" -BRANCH="$4" -RUN_ID="$5" -AGENT_INDEX="${6:-0}" -AGENT_COUNT="${7:-0}" -TOPIC="${8:-}" - -api_url() { - printf 'https://api.github.com%s' "$1" -} - -github_request() { - local method="$1" - local path="$2" - local output="$3" - shift 3 - curl -sS \ - -X "$method" \ - -H "Accept: application/vnd.github+json" \ - -H "Authorization: Bearer ${DEMO_GITHUB_TOKEN}" \ - -H "X-GitHub-Api-Version: 2022-11-28" \ - "$@" \ - -o "$output" \ - -w "%{http_code}" \ - "$(api_url "$path")" -} - -render_template() { - local template="$1" - local slice="${2:-}" - node - "$template" "$AGENT_INDEX" "$AGENT_COUNT" "$TOPIC" "$slice" <<'NODE' -const fs = require("fs"); -const [templatePath, agentIndex, agentCount, topic, slice] = process.argv.slice(2); -let text = fs.readFileSync(templatePath, "utf8"); -text = text.replace(/^(?:\n)+\n?/, ""); -const replacements = { - "{{AGENT_INDEX}}": agentIndex, - "{{AGENT_COUNT}}": agentCount, - "{{TOPIC}}": topic, - "{{SLICE}}": slice, -}; -for (const [needle, value] of Object.entries(replacements)) { - text = text.split(needle).join(value); -} -process.stdout.write(text); -NODE -} - -bootstrap_codex_oauth() { - mkdir -p "$HOME/.codex" - node - <<'NODE' -const fs = require("fs"); -const path = `${process.env.HOME}/.codex/auth.json`; -const b64u = (obj) => Buffer.from(JSON.stringify(obj)).toString("base64url"); -const now = Math.floor(Date.now() / 1000); -const fakeIdToken = [ - b64u({ alg: "none", typ: "JWT" }), - b64u({ - iss: "https://auth.openai.com", - aud: "codex", - sub: "openshell-placeholder", - email: "placeholder@example.com", - iat: now, - exp: now + 3600, - }), - "placeholder", -].join("."); - -fs.writeFileSync(path, JSON.stringify({ - auth_mode: "chatgpt", - OPENAI_API_KEY: null, - tokens: { - id_token: fakeIdToken, - access_token: process.env.CODEX_AUTH_ACCESS_TOKEN, - refresh_token: process.env.CODEX_AUTH_REFRESH_TOKEN, - account_id: process.env.CODEX_AUTH_ACCOUNT_ID, - }, - last_refresh: new Date().toISOString(), -}, null, 2)); -NODE - chmod 600 "$HOME/.codex/auth.json" -} - -put_contents() { - local repo_path="$1" - local source_file="$2" - local message="$3" - local get_body put_body put_response status sha - get_body="$(mktemp)" - put_body="$(mktemp)" - put_response="$(mktemp)" - - status="$(github_request GET "/repos/${OWNER}/${REPO}/contents/${repo_path}?ref=${BRANCH}" "$get_body")" - if [[ "$status" == "200" ]]; then - sha="$(node -e 'const fs=require("fs"); const p=JSON.parse(fs.readFileSync(process.argv[1],"utf8")); process.stdout.write(p.sha || "");' "$get_body")" - elif [[ "$status" == "404" ]]; then - sha="" - else - echo "GitHub GET ${repo_path} failed with HTTP ${status}" >&2 - cat "$get_body" >&2 - return 1 - fi - - node - "$source_file" "$message" "$BRANCH" "$sha" > "$put_body" <<'NODE' -const fs = require("fs"); -const [file, message, branch, sha] = process.argv.slice(2); -const body = { - message, - branch, - content: fs.readFileSync(file).toString("base64"), -}; -if (sha) body.sha = sha; -process.stdout.write(JSON.stringify(body)); -NODE - - status="$(github_request PUT "/repos/${OWNER}/${REPO}/contents/${repo_path}" "$put_response" --data-binary "@${put_body}")" - if [[ "$status" != "200" && "$status" != "201" ]]; then - echo "GitHub PUT ${repo_path} failed with HTTP ${status}" >&2 - cat "$put_response" >&2 - return 1 - fi -} - -get_contents_file() { - local repo_path="$1" - local destination="$2" - local body status - body="$(mktemp)" - status="$(github_request GET "/repos/${OWNER}/${REPO}/contents/${repo_path}?ref=${BRANCH}" "$body")" - if [[ "$status" != "200" ]]; then - echo "GitHub GET ${repo_path} failed with HTTP ${status}" >&2 - cat "$body" >&2 - return 1 - fi - node - "$body" "$destination" <<'NODE' -const fs = require("fs"); -const [bodyPath, destination] = process.argv.slice(2); -const body = JSON.parse(fs.readFileSync(bodyPath, "utf8")); -fs.writeFileSync(destination, Buffer.from((body.content || "").replace(/\s/g, ""), "base64")); -NODE -} - -run_codex_to_file() { - local prompt_file="$1" - local output_file="$2" - codex exec \ - --skip-git-repo-check \ - --sandbox read-only \ - --ephemeral \ - --output-last-message "$output_file" \ - "$(cat "$prompt_file")" -} - -worker() { - local slice="$9" - local prompt_file output_file repo_path - bootstrap_codex_oauth - prompt_file="$(mktemp)" - output_file="$(mktemp)" - repo_path="runs/${RUN_ID}/notes/agent-${AGENT_INDEX}.md" - - render_template /sandbox/payload/prompts/worker.md "$slice" > "$prompt_file" - run_codex_to_file "$prompt_file" "$output_file" - put_contents "$repo_path" "$output_file" "Add agent ${AGENT_INDEX} note for ${RUN_ID}" - printf 'wrote %s\n' "$repo_path" -} - -synthesis() { - local notes_dir prompt_file output_file repo_path - bootstrap_codex_oauth - notes_dir="$(mktemp -d)" - prompt_file="$(mktemp)" - output_file="$(mktemp)" - repo_path="runs/${RUN_ID}/summary.md" - - for i in $(seq 1 "$AGENT_COUNT"); do - get_contents_file "runs/${RUN_ID}/notes/agent-${i}.md" "${notes_dir}/agent-${i}.md" - done - - render_template /sandbox/payload/prompts/synthesis.md "" > "$prompt_file" - { - printf '\n\n## Worker Notes\n\n' - for i in $(seq 1 "$AGENT_COUNT"); do - printf '\n\n---\n\n' - cat "${notes_dir}/agent-${i}.md" - done - } >> "$prompt_file" - - run_codex_to_file "$prompt_file" "$output_file" - put_contents "$repo_path" "$output_file" "Add multi-agent summary for ${RUN_ID}" - printf 'wrote %s\n' "$repo_path" -} - -case "$MODE" in - worker) worker "$@" ;; - synthesis) synthesis ;; - *) echo "unknown mode: $MODE" >&2; exit 2 ;; -esac -EOF + cp "${SCRIPT_DIR}/runner.sh" "$RUNNER_FILE" chmod +x "$RUNNER_FILE" } @@ -426,6 +224,7 @@ print_results() { } main() { + step "Validating Codex and GitHub credentials" validate_env render_policy write_runner diff --git a/examples/multi-agent-notepad/runner.sh b/examples/multi-agent-notepad/runner.sh new file mode 100755 index 0000000000..d04186248d --- /dev/null +++ b/examples/multi-agent-notepad/runner.sh @@ -0,0 +1,235 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# This script runs INSIDE each OpenShell sandbox. +# +# It is uploaded to /sandbox/payload/demo-runner.sh by demo.sh and invoked as +# the sandbox entrypoint. It receives positional arguments from `run_sandbox()` +# in demo.sh and reads DEMO_GITHUB_TOKEN from the environment, where the +# OpenShell proxy resolves the provider placeholder at the network boundary. +# +# Modes: +# worker — render the worker prompt for a slice, run codex, PUT the note. +# synthesis — read every worker note, run the synthesis prompt, PUT summary. + +set -euo pipefail + +MODE="$1" +OWNER="$2" +REPO="$3" +BRANCH="$4" +RUN_ID="$5" +AGENT_INDEX="${6:-0}" +AGENT_COUNT="${7:-0}" +TOPIC="${8:-}" + +api_url() { + printf 'https://api.github.com%s' "$1" +} + +github_request() { + local method="$1" + local path="$2" + local output="$3" + shift 3 + curl -sS \ + -X "$method" \ + -H "Accept: application/vnd.github+json" \ + -H "Authorization: Bearer ${DEMO_GITHUB_TOKEN}" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + "$@" \ + -o "$output" \ + -w "%{http_code}" \ + "$(api_url "$path")" +} + +render_template() { + local template="$1" + local slice="${2:-}" + node - "$template" "$AGENT_INDEX" "$AGENT_COUNT" "$TOPIC" "$slice" <<'NODE' +const fs = require("fs"); +const [templatePath, agentIndex, agentCount, topic, slice] = process.argv.slice(2); +let text = fs.readFileSync(templatePath, "utf8"); +text = text.replace(/^(?:\n)+\n?/, ""); +const replacements = { + "{{AGENT_INDEX}}": agentIndex, + "{{AGENT_COUNT}}": agentCount, + "{{TOPIC}}": topic, + "{{SLICE}}": slice, +}; +for (const [needle, value] of Object.entries(replacements)) { + text = text.split(needle).join(value); +} +process.stdout.write(text); +NODE +} + +bootstrap_codex_oauth() { + mkdir -p "$HOME/.codex" + node - <<'NODE' +const fs = require("fs"); +const path = `${process.env.HOME}/.codex/auth.json`; +const b64u = (obj) => Buffer.from(JSON.stringify(obj)).toString("base64url"); +const now = Math.floor(Date.now() / 1000); +const fakeIdToken = [ + b64u({ alg: "none", typ: "JWT" }), + b64u({ + iss: "https://auth.openai.com", + aud: "codex", + sub: "openshell-placeholder", + email: "placeholder@example.com", + iat: now, + exp: now + 3600, + }), + "placeholder", +].join("."); + +fs.writeFileSync(path, JSON.stringify({ + auth_mode: "chatgpt", + OPENAI_API_KEY: null, + tokens: { + id_token: fakeIdToken, + access_token: process.env.CODEX_AUTH_ACCESS_TOKEN, + refresh_token: process.env.CODEX_AUTH_REFRESH_TOKEN, + account_id: process.env.CODEX_AUTH_ACCOUNT_ID, + }, + last_refresh: new Date().toISOString(), +}, null, 2)); +NODE + chmod 600 "$HOME/.codex/auth.json" +} + +put_contents() { + local repo_path="$1" + local source_file="$2" + local message="$3" + local get_body put_body put_response status sha + local attempt=0 + local max_attempts=6 + local sleep_secs=1 + get_body="$(mktemp)" + put_body="$(mktemp)" + put_response="$(mktemp)" + + while (( attempt < max_attempts )); do + attempt=$((attempt + 1)) + + status="$(github_request GET "/repos/${OWNER}/${REPO}/contents/${repo_path}?ref=${BRANCH}" "$get_body")" + if [[ "$status" == "200" ]]; then + sha="$(node -e 'const fs=require("fs"); const p=JSON.parse(fs.readFileSync(process.argv[1],"utf8")); process.stdout.write(p.sha || "");' "$get_body")" + elif [[ "$status" == "404" ]]; then + sha="" + else + echo "GitHub GET ${repo_path} failed with HTTP ${status}" >&2 + cat "$get_body" >&2 + return 1 + fi + + node - "$source_file" "$message" "$BRANCH" "$sha" > "$put_body" <<'NODE' +const fs = require("fs"); +const [file, message, branch, sha] = process.argv.slice(2); +const body = { + message, + branch, + content: fs.readFileSync(file).toString("base64"), +}; +if (sha) body.sha = sha; +process.stdout.write(JSON.stringify(body)); +NODE + + status="$(github_request PUT "/repos/${OWNER}/${REPO}/contents/${repo_path}" "$put_response" --data-binary "@${put_body}")" + if [[ "$status" == "200" || "$status" == "201" ]]; then + return 0 + fi + + # Retry on 409 (concurrent writer) and 5xx (transient server error) + # with decorrelated jitter so racing workers don't retry in lockstep. + if [[ "$status" =~ ^(409|5[0-9][0-9])$ && $attempt -lt $max_attempts ]]; then + sleep "$(( sleep_secs + RANDOM % sleep_secs ))" + sleep_secs=$((sleep_secs * 2)) + continue + fi + + echo "GitHub PUT ${repo_path} failed with HTTP ${status} (attempt ${attempt}/${max_attempts})" >&2 + cat "$put_response" >&2 + return 1 + done +} + +get_contents_file() { + local repo_path="$1" + local destination="$2" + local body status + body="$(mktemp)" + status="$(github_request GET "/repos/${OWNER}/${REPO}/contents/${repo_path}?ref=${BRANCH}" "$body")" + if [[ "$status" != "200" ]]; then + echo "GitHub GET ${repo_path} failed with HTTP ${status}" >&2 + cat "$body" >&2 + return 1 + fi + node - "$body" "$destination" <<'NODE' +const fs = require("fs"); +const [bodyPath, destination] = process.argv.slice(2); +const body = JSON.parse(fs.readFileSync(bodyPath, "utf8")); +fs.writeFileSync(destination, Buffer.from((body.content || "").replace(/\s/g, ""), "base64")); +NODE +} + +run_codex_to_file() { + local prompt_file="$1" + local output_file="$2" + codex exec \ + --skip-git-repo-check \ + --sandbox read-only \ + --ephemeral \ + --output-last-message "$output_file" \ + "$(cat "$prompt_file")" +} + +worker() { + local slice="$1" + local prompt_file output_file repo_path + bootstrap_codex_oauth + prompt_file="$(mktemp)" + output_file="$(mktemp)" + repo_path="runs/${RUN_ID}/notes/agent-${AGENT_INDEX}.md" + + render_template /sandbox/payload/prompts/worker.md "$slice" > "$prompt_file" + run_codex_to_file "$prompt_file" "$output_file" + put_contents "$repo_path" "$output_file" "Add agent ${AGENT_INDEX} note for ${RUN_ID}" + printf 'wrote %s\n' "$repo_path" +} + +synthesis() { + local notes_dir prompt_file output_file repo_path + bootstrap_codex_oauth + notes_dir="$(mktemp -d)" + prompt_file="$(mktemp)" + output_file="$(mktemp)" + repo_path="runs/${RUN_ID}/summary.md" + + for i in $(seq 1 "$AGENT_COUNT"); do + get_contents_file "runs/${RUN_ID}/notes/agent-${i}.md" "${notes_dir}/agent-${i}.md" + done + + render_template /sandbox/payload/prompts/synthesis.md "" > "$prompt_file" + { + printf '\n\n## Worker Notes\n\n' + for i in $(seq 1 "$AGENT_COUNT"); do + printf '\n\n---\n\n' + cat "${notes_dir}/agent-${i}.md" + done + } >> "$prompt_file" + + run_codex_to_file "$prompt_file" "$output_file" + put_contents "$repo_path" "$output_file" "Add multi-agent summary for ${RUN_ID}" + printf 'wrote %s\n' "$repo_path" +} + +case "$MODE" in + worker) worker "${9:-}" ;; + synthesis) synthesis ;; + *) echo "unknown mode: $MODE" >&2; exit 2 ;; +esac