feat(observability): export gateway traces over OTLP

[krishicks]

Summary

Add an opt-in OTLP/gRPC trace exporter to the gateway. Export is enabled by the presence of an [openshell.gateway.otlp] table with an endpoint; there is no separate toggle.

Related Issue

#2507

Changes

Instrumented:

• Inbound request server spans, named for the RPC ($service/$method) or {method} {path} for plain HTTP. They continue valid W3C traceparent context when present and start a new trace otherwise. gRPC spans also carry rpc.system, rpc.service, rpc.method, and trailer-derived rpc.grpc.status_code.
• Compute driver calls (create, delete, list, get, validate, watch) as client spans anchored on the ComputeDriver contract.
• Store reads and writes as children of the current request or loop span.
• Work with no inbound request: the sandbox reconcile sweep, provider credential refresh tick, and driver watch events. Each roots one operation trace so its store calls do not arrive as anonymous single-span traces.

This is deliberately not exhaustive. Auth, policy evaluation, and middleware remain uninstrumented, as do store lifecycle calls (ping, close) that a readiness poll would turn into a span per tick. The aim is a useful trace tree at a reviewable size; coverage can grow against real traces.

Design notes:

• The TOML table owns whether and where to export. The SDK OTEL_* variables own how; sampling, batching, and limits are not mirrored into gateway config.
• The OpenTelemetry layer exports spans only. Existing tracing events remain on the stdout and sandbox-log paths and are not copied into trace payloads.
• Telemetry never blocks the gateway. A malformed endpoint logs an error and disables export rather than failing startup, and buffered spans are drained during graceful shutdown.
• Failed spans carry error status without a separate error.type attribute. Request spans use HTTP status and gRPC response trailers; driver spans use the returned gRPC status; autonomous loop spans record failed results explicitly. Store spans exempt UniqueViolation and Conflict, because those errors report expected contention such as a held lease or an optimistic-concurrency retry.
• The compute driver is reachable only through TracedDriver::call, so a call cannot skip its span. This is the client half of a client/server pair and the single place to inject context if drivers move out of process.
• Tests share one process-wide subscriber and in-memory exporter because tracing caches callsite interest globally.

Inbound W3C trace context is propagated into gateway request spans. Context is not yet injected into outbound driver calls, so a future out-of-process driver would still need propagation at the TracedDriver seam.

The Helm chart is intentionally unchanged, so OTLP export cannot yet be enabled on a chart-deployed gateway.

Testing

• mise run pre-commit passes
• Unit tests added/updated
• E2E tests added/updated (if applicable)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2534.docs.buildwithfern.com/openshell

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[pimlock]

LGTM! I left couple comments, but these can be follow ups.