Skip to content

fix(providers): validate L7 endpoint constraints in profile lint - #2

Closed
gracesmith6504 wants to merge 1 commit into
mainfrom
fix/1714-profile-lint-l7-validation
Closed

fix(providers): validate L7 endpoint constraints in profile lint#2
gracesmith6504 wants to merge 1 commit into
mainfrom
fix/1714-profile-lint-l7-validation

Conversation

@gracesmith6504

@gracesmith6504 gracesmith6504 commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

Profile lint (openshell provider profile lint) only validated host and port on endpoints, ignoring L7 fields like protocol, access, rules, and deny_rules. This meant profiles could pass lint but fail at sandbox startup with errors such as "protocol requires rules or access to define allowed traffic."

This adds L7 semantic checks to validate_profile_set() in the providers crate, matching the runtime validation logic in validate_l7_policies().

Before (broken profile passes lint silently):

$ openshell provider profile lint -f test-profile.yaml
Provider profile lint passed.

After (lint catches the missing constraint):

$ openshell provider profile lint -f test-profile.yaml
Provider profile diagnostics:
  error test-profile.yaml profile=opencode-openrouter field=endpoints[0] protocol requires rules or access to define allowed traffic
Error:   × provider profile lint failed

Related Issue

Fixes NVIDIA#1714

Changes

  • Added is_known_l7_protocol() and is_jsonrpc_family() helpers
  • Added 9 error checks to the endpoint validation loop in validate_profile_set(), matching the semantics from validate_l7_policies():
    1. Unknown protocol (not in rest/websocket/graphql/sql/json-rpc/mcp)
    2. Rules and access mutually exclusive
    3. JSON-RPC family + access rejection
    4. JSON-RPC requires explicit rules
    5. Protocol requires rules or access (non-mcp)
    6. MCP requires rules when allow_all_known_mcp_methods is false
    7. Rules must contain at least one allow entry (would deny all traffic)
    8. Deny rules require protocol
    9. Deny rules require base allow set
  • Fixed existing profile_yaml_round_trip_preserves_full_network_policy_fields test (had access + rules together, now mutually exclusive)
  • Added 11 new tests covering all validation paths including issue bug(providers): provider v2 profile lint does not validate L7 endpoint constraints (protocol/access/rules) NVIDIA/OpenShell#1714 reproduction

Testing

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (not applicable)

Summary by CodeRabbit

  • Improvements
    • Expanded endpoint configuration validation across multiple protocol families, including stricter checks for rules, access, and deny_rules.
    • Added clearer, per-endpoint diagnostics (now scoped to each endpoint entry) and broadened protocol consistency validation (including JSON-RPC and MCP-specific requirements).
  • Bug Fixes
    • Corrected/strengthened handling of empty or incompatible rule configurations and invalid rules/access combinations.
  • Tests
    • Updated fixtures and assertions and added new test coverage for unknown protocols, missing required fields, mutual exclusivity, and JSON-RPC/MCP constraints.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0dd3fb05-e7f7-453a-96a8-14046d55a72f

📥 Commits

Reviewing files that changed from the base of the PR and between 90f7188 and 6b3c3ad.

📒 Files selected for processing (1)
  • crates/openshell-providers/src/profiles.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/openshell-providers/src/profiles.rs

📝 Walkthrough

Walkthrough

Profile endpoint validation now enforces supported protocols, access-control combinations, JSON-RPC/MCP rules, and deny-rule semantics. Fixtures and tests cover GraphQL and REST round-tripping plus regression scenarios including issue 1714.

Changes

Endpoint Validation

Layer / File(s) Summary
Endpoint validation rules
crates/openshell-providers/src/profiles.rs
Validates protocol allowlists, rules/access combinations, JSON-RPC and MCP requirements, effective allow entries, and deny_rules.
Round-trip fixture coverage
crates/openshell-providers/src/profiles.rs
Separates GraphQL and REST endpoint configurations and verifies REST rules after YAML round-tripping.
Validation regression tests
crates/openshell-providers/src/profiles.rs
Covers invalid protocols, missing or conflicting access controls, deny rules, JSON-RPC/MCP restrictions, empty rules, and issue 1714 behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

  • grace-issue-alerts issue 26: Covers the expanded validate_profile_set L7 endpoint validation implemented here.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and clearly summarizes the main change: added L7 endpoint constraint validation to profile lint.
Linked Issues check ✅ Passed The PR implements the requested L7 endpoint lint checks for #1714 within openshell-providers, matching the issue's validation gap.
Out of Scope Changes check ✅ Passed The changes appear in scope for the validation fix, with supporting test and fixture updates only.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/1714-profile-lint-l7-validation

Comment @coderabbitai help to get the list of available commands.

@gracesmith6504

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@gracesmith6504
gracesmith6504 marked this pull request as ready for review July 14, 2026 12:40

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/openshell-providers/src/profiles.rs`:
- Around line 1465-1490: Update the validation conditions around the
GraphQL-specific warning and JSON-RPC-specific warning so unset protocols also
trigger diagnostics when their respective fields are populated. Preserve the
existing protocol-specific exclusions and warning messages for supported
protocols.
- Around line 1403-1419: The protocol validation in the profile validation flow
must emit only one diagnostic when json-rpc lacks rules and access. Narrow the
generic check to exclude json-rpc (or otherwise combine the overlapping
conditions), and apply the same deduplication to the deny_rules base-allow
validation so each misconfiguration produces a single diagnostic while
preserving the existing specific json-rpc message.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2478dc25-9f3e-4224-a111-0a124ca66240

📥 Commits

Reviewing files that changed from the base of the PR and between e3d26dd and 86b0524.

📒 Files selected for processing (1)
  • crates/openshell-providers/src/profiles.rs

Comment thread crates/openshell-providers/src/profiles.rs
Comment thread crates/openshell-providers/src/profiles.rs Outdated
@gracesmith6504
gracesmith6504 force-pushed the fix/1714-profile-lint-l7-validation branch 2 times, most recently from e766b94 to 90f7188 Compare July 14, 2026 13:34
Profile lint (openshell provider profile lint) only validated host and
port on endpoints, ignoring L7 fields like protocol, access, rules, and
deny_rules. This meant profiles could pass lint but fail at sandbox
startup with errors such as "protocol requires rules or access to define
allowed traffic."

Add L7 semantic checks to validate_profile_set() in the providers crate,
matching the runtime validation logic in validate_l7_policies(). The new
checks cover:

- Unknown protocol values
- Protocol requiring rules or access
- Mutual exclusion of rules and access
- JSON-RPC family restrictions (no access presets, requires explicit rules)
- MCP protocol requiring rules when allow_all_known_mcp_methods is false
- Rules must contain at least one allow entry (would deny all traffic)
- Deny rules requiring protocol and base allow set

Fixes NVIDIA#1714

Signed-off-by: Grace Smith <grasmith@redhat.com>
@gracesmith6504
gracesmith6504 force-pushed the fix/1714-profile-lint-l7-validation branch from 90f7188 to 6b3c3ad Compare July 14, 2026 13:39

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
crates/openshell-providers/src/profiles.rs (1)

1388-1448: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Diagnostic duplication still present despite prior "addressed" marker.

This reproduces the previously-flagged overlap: the generic check at Line 1397 (!protocol.is_empty() && protocol != "mcp" && !has_rules && access.is_empty()) excludes mcp but not json-rpc, so a json-rpc endpoint missing rules triggers both the Line 1388-1395 message and the Line 1397-1404 message. Likewise, the deny_rules base-allow check at Line 1441-1448 is a strict subset of the Line 1397 condition (it only adds has_deny_rules), so it always co-fires alongside the generic message whenever it fires. This is directly reproduced by validate_profile_set_rejects_deny_rules_without_base_allow (Line 2918-2942 below), which uses .any() and therefore doesn't surface the duplicate.

♻️ Suggested consolidation
-            if protocol == "json-rpc" && !has_rules {
-                diagnostics.push(ProfileValidationDiagnostic::error(
-                    source,
-                    profile_id,
-                    &loc,
-                    format!("protocol {protocol} requires explicit rules with allow.method"),
-                ));
-            }
-
-            if !protocol.is_empty() && protocol != "mcp" && !has_rules && access.is_empty() {
-                diagnostics.push(ProfileValidationDiagnostic::error(
-                    source,
-                    profile_id,
-                    &loc,
-                    "protocol requires rules or access to define allowed traffic",
-                ));
-            }
+            let missing_base_allow =
+                !protocol.is_empty() && protocol != "mcp" && !has_rules && access.is_empty();
+            if protocol == "json-rpc" && missing_base_allow {
+                diagnostics.push(ProfileValidationDiagnostic::error(
+                    source,
+                    profile_id,
+                    &loc,
+                    format!("protocol {protocol} requires explicit rules with allow.method"),
+                ));
+            } else if missing_base_allow && has_deny_rules {
+                diagnostics.push(ProfileValidationDiagnostic::error(
+                    source,
+                    profile_id,
+                    &loc,
+                    "deny_rules require rules or access to define the base allow set",
+                ));
+            } else if missing_base_allow {
+                diagnostics.push(ProfileValidationDiagnostic::error(
+                    source,
+                    profile_id,
+                    &loc,
+                    "protocol requires rules or access to define allowed traffic",
+                ));
+            }

Then drop the now-redundant standalone block at Line 1441-1448.

Please confirm whether commit e766b94 actually addressed this, or only partially (e.g. only the mcp exclusion), since the currently-shown logic still exhibits the overlap.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/openshell-providers/src/profiles.rs` around lines 1388 - 1448,
Consolidate overlapping validation diagnostics in the profile validation flow:
update the generic non-empty protocol check to exclude json-rpc as well as mcp,
so json-rpc-specific errors remain the sole diagnostic when rules are missing;
remove the redundant has_deny_rules base-allow block because it is fully covered
by the generic condition. Preserve the existing diagnostic messages and behavior
for unrelated protocols.
🧹 Nitpick comments (1)
crates/openshell-providers/src/profiles.rs (1)

1330-1346: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Repeated diagnostic boilerplate; consider a local helper.

Every branch in this loop repeats diagnostics.push(ProfileValidationDiagnostic::error(source, profile_id, &loc, ...)). A small closure (e.g. let mut push_error = |msg: String| diagnostics.push(ProfileValidationDiagnostic::error(source, profile_id, &loc, msg));) would cut the repetition. Given the line-range diff notes flag this endpoint block as high complexity, extracting the L7 checks (Lines 1330-1449) into a dedicated validate_endpoint_l7(...) helper would also make the rules independently testable and keep validate_profile_set focused on cross-cutting profile concerns.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/openshell-providers/src/profiles.rs` around lines 1330 - 1346, Reduce
repeated diagnostic construction in the endpoint validation loop by adding a
local error-pushing closure that captures source, profile_id, and loc, then use
it for each endpoint diagnostic. Extract the L7 validation checks from
validate_profile_set into a dedicated validate_endpoint_l7 helper so those rules
are independently testable while preserving existing diagnostics and behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@crates/openshell-providers/src/profiles.rs`:
- Around line 1388-1448: Consolidate overlapping validation diagnostics in the
profile validation flow: update the generic non-empty protocol check to exclude
json-rpc as well as mcp, so json-rpc-specific errors remain the sole diagnostic
when rules are missing; remove the redundant has_deny_rules base-allow block
because it is fully covered by the generic condition. Preserve the existing
diagnostic messages and behavior for unrelated protocols.

---

Nitpick comments:
In `@crates/openshell-providers/src/profiles.rs`:
- Around line 1330-1346: Reduce repeated diagnostic construction in the endpoint
validation loop by adding a local error-pushing closure that captures source,
profile_id, and loc, then use it for each endpoint diagnostic. Extract the L7
validation checks from validate_profile_set into a dedicated
validate_endpoint_l7 helper so those rules are independently testable while
preserving existing diagnostics and behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 65071edc-9fc7-4dfe-8884-fe42884f56ba

📥 Commits

Reviewing files that changed from the base of the PR and between e766b94 and 90f7188.

📒 Files selected for processing (1)
  • crates/openshell-providers/src/profiles.rs

@gracesmith6504

Copy link
Copy Markdown
Owner Author

Superseded by #4 which uses the shared policy layer approach recommended by the triage agent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(providers): provider v2 profile lint does not validate L7 endpoint constraints (protocol/access/rules)

1 participant