DIPs testing#67
Draft
MoonBoi9001 wants to merge 89 commits into
Draft
Conversation
MoonBoi9001
force-pushed
the
samuel/dips-dev-environment
branch
3 times, most recently
from
cb0c65b to
39e989f
Compare
MoonBoi9001
changed the title
MoonBoi9001
force-pushed
the
samuel/dips-dev-environment
branch
from
39e989f to
28089bd
Compare
MoonBoi9001
force-pushed
the
samuel/dips-dev-environment
branch
from
7c68181 to
0345ff8
Compare
MoonBoi9001
force-pushed
the
samuel/dips-dev-environment
branch
2 times, most recently
from
962e3f7 to
972b4af
Compare
Mount local checkouts of contracts, indexer-rs, dipper, iisa, and the eligibility-oracle-node so the stack runs your branches end-to-end. Also adds a separate eligibility-oracle overlay and aligns the base compose file with the dev overrides. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add dev run-dips.sh entrypoints for indexer-agent and indexer-service so they consume mounted source. Refresh dipper, iisa, eligibility-oracle, tap-agent, and tap-escrow-manager run scripts to drive the end-to-end DIPs pipeline against horizon contracts. Drop the iisa local-scoring stub set (replaced by API-push design from the cronjob). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add scripts for sending indexing requests, deploying test subgraphs, generating extra indexers, monitoring the DIPs pipeline, checking subgraph sync, and snapshotting network status. These drive the end-to-end testing flow against the local stack. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add reusable .claude/skills entries for the recurring DIPs testing operations (fresh deploy, add indexers, deploy subgraphs, send indexing request, network status). Add BUGS.md to log every issue surfaced during end-to-end testing, CLAUDE.md for project-level guidance, and TESTING-STATUS.md to track progress. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switch the env file to the testing-targeted config: enable the indexing-payments profile, point all *_SOURCE_ROOT at local checkouts, pin contracts to mb9/dips-local-testing-fixes, and add the x402 receiver wallet for the gateway. Refresh .gitignore alongside. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Base stack now brings up every service from pinned commits or images. Per-service dips-*.yaml overlays opt individual components into a source-mount mode that builds in-container from a local checkout. dips.yaml stays as the mount-everything preset. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MoonBoi9001
force-pushed
the
samuel/dips-dev-environment
branch
from
9dd0111 to
77db3a0
Compare
The DIPs source-mount overlays were a parallel build pipeline that duplicated the image-only path. They hard-coded Mac host paths in the env file and broke on any non-Mac clone. Drop them; rely on the pinned image versions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cronjob image is published to GHCR by the upstream subgraph-dips-indexer-selection workflow. Pull it via image: instead of cloning the private source and building locally, so fresh deployments work without GitHub auth in the build container. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The eligibility-oracle-node repo is private. Cloning at build time fails on machines without GitHub auth. Each developer drops a local clone at the gitignored containers/oracles/eligibility- oracle-node/source/ path; the Dockerfile COPYs from there. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
graph-contracts no longer writes tap-contracts.json — TAP-related addresses (GraphTallyCollector, PaymentsEscrow) live in horizon.json now. wait_for_config blocked 300s waiting for the missing file; dipper read TAPVerifier from it. Both updated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
indexer-agent crashes at SubgraphClient.create when the spec has tapSubgraph or indexingPaymentsSubgraph as empty objects (truthy in JS). Restore both endpoints. The TAP subgraph isn't deployed on this branch, but a stale URL still lets the agent start. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The agent's @semiotic-labs/tap-contracts-bindings library has no chainId 1337 baked in. Without a tap-contracts.json address book the binding library rejects Network.create and the management API never starts. Stub it from horizon.json: TAPVerifier <- GraphTallyCollector, Escrow <- PaymentsEscrow. Addresses aren't exercised on the DIPs path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Rust tap-agent and indexer-service crash on startup with "missing field query_url for default.subgraphs.escrow" — the schema hard-requires this section even though semiotic/tap subgraph isn't deployed on this branch. Stale URL satisfies the schema; queries against it fail gracefully and the DIPs flow doesn't use this path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extras now use the same build context, image versions, healthchecks, and run.sh as the primary indexer-agent and indexer-service. Drops the dockerfile_inline wrapper, the host source-mount volumes, and the run-dips.sh entrypoint override. Per-indexer identity and hostnames flow in via compose environment overrides. Also fixes the generator's ENV_FILE path from a non-existent .environment to .env, so the COMPOSE_FILE entry is actually written. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dips overlay was removed in commit 325ec70; the add-indexers skill still chained it into every docker compose invocation, which would fail on a fresh clone with "no such file or directory". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Old skill was a soft reset (containers + volumes only) plus several references that no longer hold on this branch — the dips compose overlay was deleted, tap-escrow-manager was renamed, the TAP subgraph isn't deployed any more, and run.sh is no longer volume- mounted. The new skill targets the lnet-test VM, wipes containers, volumes, networks, all images, and the clone itself, then re-clones from origin, repopulates eligibility-oracle-node/source via rsync from the Mac, runs build --pull, brings up the stack, and streams per-service health to the user. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The old skill ran every command (docker, curl, python) against the local docker daemon. That doesn't work on the Mac+VM setup where docker lives on lnet-test and the generator script lives on the Mac. Rewrite makes the split explicit: generator runs on Mac, the yaml and updated .env are scp'd to the VM, all docker / docker-pause / curl-localhost commands are SSH-wrapped. Drop stale claims about flock-serialized cargo builds (no Rust compile happens any more — extras use primary's thin-wrapper Dockerfile), the legacy TAP subgraph, the DOCKER_DEFAULT_PLATFORM= prefix (VM is amd64-native), and the orphan reference to /fresh-deploy's down -v handling. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The contract-address helper chained an overlay file that was removed earlier on this branch, breaking the script with "no such file or directory" on a fresh clone. Let docker compose read the overlay list from .env. Also drop an unused import. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The scripts hit localhost-only endpoints and shell out to cast and the graph CLI, so they must run on the VM via SSH. Document the one-time install of foundry from a release tarball, plus Node 22 from NodeSource since the apt default is too old for graph CLI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dipper PR #626 single-sources chain_id and the RecurringCollector address, leaving the two chain_client keys unread on newer builds. They must stay while DIPPER_VERSION predates that PR (older builds require them), so a note marks exactly when to drop them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Newer pinned contracts moved RecurringAgreementManager into the GIP-0088 issuance deploy that graph-contracts wasn't running, so the indexing-payments subgraph had no address and failed, blocking the indexer. Deploy it, grant the payer its role, and wire in the address. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The pinned indexer-rs requires subgraphs.indexing_payments whenever DIPs is enabled, but the generated config didn't set it, so indexer-service exited on startup once it finally ran. Point the DIPs config at the local indexing-payments subgraph's query URL. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The pinned dipper build exits on startup because its config still uses pricing and chain keys it no longer accepts. Swap them for the per-agreement monthly GRT ceiling it now expects, and drop the indexer-server and tap-signer blocks it stopped reading. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The newer dipper build requires the recurring-agreement-manager contract address in its payments config, and its chain-client section no longer accepts two address keys it now reads elsewhere. Emit the address and drop the two keys so the service starts cleanly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ten tracked bugs had their fixes verified present in the exact dipper, indexer, contracts and iisa versions the stack now pins, so the entries were purely historical. Drop them and keep the six local-network config notes plus the two still-open performance items. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
With the resolved bugs gone, the eight remaining entries are renumbered one to eight. Two entries referenced others by number; both are rewritten to read on their own. One was inlined because its target was deleted, the other repointed to its new number. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Renumbering the bug tracker left a code comment and a skill note pointing at numbers that no longer match. Both already explain the reason inline, so drop the numbers and keep the wording; the agent run.sh comments are also trimmed to the project's three-line limit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The pipeline monitor's summary counted unique status values, so three indexers all accepting on-chain were reported as "1 accepted". Count the agreements directly so the summary matches the per-indexer lines printed above it. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The merged DIPs log viewer lived only as an untracked file on the test VM, so losing it meant rebuilding from memory. This commits it: it folds every DIPs container's logs into one filtered stream, auto-attaches new indexers, and drops graph-node's repetitive per-block lines. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The merged DIPs log view tails every indexer, so each one's routine reward-collection and allocation chatter showed up six times over, burying the real DIPs events. This drops that chatter and collapses the per-agent status heartbeats to reprint only when their counts change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The previous filter dropped indexing-reward collection and allocation actions as routine chatter, but on DIPs allocations they are real events showing the indexer working and earning. This restores them; the per-agent idle heartbeats stay deduped to print only on change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The send-indexing-request guide told you to build a dipper-cli binary on the Mac and tunnel to the VM, but we now run the pinned dipper-cli container on the VM directly. This rewrites the steps to match, dropping the Mac build, the SSH tunnel, and its teardown. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
When a DIPs agreement is accepted, the indexer's graph-node syncs the newly assigned subgraph to chain head, emitting a Scanning/Scanned line per block range. In a recent capture these were 2316 lines, 88% of the window, with no DIPs signal — the merged viewer now filters them out. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The start-indexing-extra init container deposited 2 GRT of query-fee escrow per extra indexer. The gateway-side escrow manager already funds that same escrow for every indexer with an active allocation, so the deposit was redundant and is removed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The start-indexing-extra init container sent 1 ETH to each extra indexer for gas. Those accounts are anvil junk-mnemonic accounts the chain already pre-funds, so raising anvil to `--accounts 20` covers them and the transfer is dropped. Operators still get their ETH. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
The gateway image cloned edgeandnode/gateway and ran a debug cargo build on every cold deploy (~minutes). It now layers run.sh onto the prebuilt ghcr.io/edgeandnode/graph-gateway image that production runs (v27.6.0, a release build), cutting one Rust compile from the deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KJmMVG736T3xefovirWryK
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dev environment for end-to-end DIPs testing against Horizon contracts. Sharing as a draft.