Skip to content

Add Exie AI assistant - #2430

Open
ejsmith wants to merge 24 commits into
mainfrom
feature/exie-assistant
Open

Add Exie AI assistant#2430
ejsmith wants to merge 24 commits into
mainfrom
feature/exie-assistant

Conversation

@ejsmith

@ejsmith ejsmith commented Aug 3, 2026

Copy link
Copy Markdown
Member

What changed

  • Add Exie, an in-app AI assistant that opens from the main toolbar without replacing the user's current stack or event workflow.
  • Stream conversations through the Exceptionless API with Markdown formatting, links, copy/feedback actions, expandable tool activity, suggested follow-ups, and current page/detail-sheet context.
  • Let Exie investigate through the Exceptionless MCP tools, including the existing low-risk write tools, with server-enforced organization boundaries and exact user authorization for mutations.
  • Add feature configuration for OpenRouter-compatible chat completions. The feature defaults on when an API key is configured, can be disabled completely, and keeps credentials in deployment secrets/environment variables.
  • Restrict hosted access to Medium plans and above while keeping development mode available for testing. Plan options control concurrency, per-minute turns, monthly tokens, and monthly cost.
  • Make conversations and limits web-farm safe through the shared cache. Persist monthly per-organization usage, cost, outcome, tool, and limit statistics; emit aggregate OpenTelemetry metrics; and expose a global-admin usage report in System → Exie.
  • Include the assistant and admin usage API contracts in OpenAPI and the HTTP samples.

Why

Exceptionless users should be able to investigate errors, stacks, and events conversationally without leaving their current workflow. The first production version also needs predictable provider spend, plan-based access, web-farm correctness, operational visibility, and a complete off switch for self-hosted installations.

Plan limits

Plan Concurrent turns Turns/minute Monthly tokens Monthly cost
Medium 2 10 25M $5
Large 3 15 50M $10
Extra Large 5 25 100M $20
Enterprise 10 50 250M $50
Unlimited 20 100 500M $100

Free and Small plans receive the upgrade experience. Annual plans use the corresponding monthly allowance.

Operational impact

  • Organization index version increases to store rolling assistant usage history.
  • Usage is aggregated through the existing distributed usage pipeline and retained for 366 days.
  • Monthly limit counters rehydrate from durable organization usage after shared-cache loss, and provider requests reserve conservative usage until actual provider totals arrive.
  • GET /api/v2/admin/assistant-usage powers the global-admin monthly totals, per-organization consumption, plan utilization, and blocked-attempt reporting in System → Exie.
  • No API key or provider credential is committed.
  • No existing public API or WebSocket contract is intentionally broken.

Validation

  • npm run validate — formatting, ESLint, and svelte-check passed with 0 errors and 0 warnings.
  • Focused Svelte/Vitest coverage — 11 files, 20 tests passed.
  • Clean dotnet build ./Exceptionless.slnx --no-restore --configuration Release — passed with 0 warnings and 0 errors.
  • Focused Release regression coverage for Exie, MCP, usage persistence, OpenAPI, and endpoint manifests — 193 tests passed.
  • Live GitHub checks — Docker build, client validation/tests, Playwright E2E, and the full .NET test suite with coverage all passed.
  • All 19 review threads were answered and resolved.
  • git diff --check passed.

@ejsmith
ejsmith marked this pull request as ready for review August 3, 2026 05:15

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be0b03b29c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Api/Endpoints/AssistantEndpoints.cs
Comment thread src/Exceptionless.Web/Api/Endpoints/AssistantEndpoints.cs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8f9ec4b33

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Api/Endpoints/AssistantEndpoints.cs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fadb9a1993

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantUsageService.cs
Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs
Comment thread src/Exceptionless.Web/Api/Endpoints/AdminEndpoints.cs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2d6f0fb55a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs Outdated
Comment thread src/Exceptionless.Web/Assistant/AssistantAccessService.cs

@niemyjski niemyjski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and sweet feature, left some comments mostly on magic strings.

Also, what do we want to do for /docs/ and or running locally, would it detect ollama automatically?

Comment thread src/Exceptionless.Core/Configuration/AssistantOptions.cs Outdated
Comment thread src/Exceptionless.Core/Repositories/Configuration/Indexes/OrganizationIndex.cs Outdated
Comment thread src/Exceptionless.Core/Services/UsageService.cs Outdated
Comment thread src/Exceptionless.Web/Assistant/AssistantToolResultSerializer.cs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b083c501cd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c993cd38c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantAccessService.cs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd4d2054cf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Mcp/ExceptionlessMcpTools.cs
Comment thread src/Exceptionless.Web/Assistant/AssistantUsageService.cs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c367aa420

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs Outdated
Comment thread src/Exceptionless.Web/Mcp/ExceptionlessMcpTools.cs Outdated

ejsmith commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

@niemyjski Regarding /docs/ and local/self-hosted use: I kept provider discovery explicit in this PR. Exie uses the configured OpenAI-compatible Assistant:Endpoint and Assistant:Model, and it only becomes available when it is configured; it does not probe for or silently switch to Ollama. A local Ollama instance can be used by pointing those settings at Ollama's OpenAI-compatible chat-completions endpoint (with a non-empty local placeholder key if the endpoint does not require auth). I think the right follow-up is a short dedicated Exie/self-hosting docs page with OpenRouter and Ollama examples, rather than putting this in the Kubernetes topic. The automatic-detection behavior is intentionally out of scope because it would make production provider selection surprising.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 825fe1e5aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantUsageService.cs Outdated
Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs Outdated
Comment thread src/Exceptionless.Web/Api/Endpoints/AssistantEndpoints.cs Outdated

@niemyjski niemyjski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really solid iteration overall: the assistant-specific services, organization boundary, focused tests, and published API contracts make this large feature much easier to reason about. I kept this pass intentionally focused and added one new inline blocker that matches the current test-api failure. The other high-value concerns are already captured in the three existing open threads; for the reservation issue, the cleanest structural fix would be one lifecycle object that owns reserve, reconcile, and conservative fallback persistence rather than another branch in the stream loop. I did not add cosmetic or broad refactor comments.

Comment thread src/Exceptionless.Core/Services/UsageService.cs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a774eb4e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs Outdated
Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs Outdated
Comment thread src/Exceptionless.Web/Assistant/AssistantUsageService.cs Outdated
Comment thread src/Exceptionless.Core/Services/UsageService.cs
# Conflicts:
#	src/Exceptionless.Web/ClientApp/package-lock.json
#	src/Exceptionless.Web/ClientApp/src/routes/(app)/stack/[stackId=objectid]/+page.svelte

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 08182c3d5c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs
Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs
Comment thread src/Exceptionless.Web/Assistant/AssistantUsageService.cs Outdated
Comment thread src/Exceptionless.Web/Assistant/AssistantUsageService.cs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a41aa87723

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs
Comment thread src/Exceptionless.Web/Assistant/AssistantUsageService.cs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 41cb1bad91

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ce0e65456c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs Outdated
Comment thread src/Exceptionless.Web/Assistant/AssistantService.cs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f466ad820a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

<Button
class="h-auto min-h-7 gap-1.5 px-2 py-1 text-left text-xs whitespace-normal"
disabled={suggestionsDisabled}
onclick={() => onSuggestedAction?.(action.prompt)}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not treat model-suggested prompts as write approval

When a provider-generated suggestion has a benign label but a hidden mutation prompt, clicking the button submits action.prompt as an ordinary user message even though only the label is displayed. Fresh evidence beyond the earlier write-confirmation concern is that prompt-injected tool data can therefore produce a button labeled “View details” whose prompt says “Please ignore this stack”; the server then treats that generated text as the user's exact approval and can execute the write. Display and confirm the full mutation or mark suggested prompts as model-generated so they cannot authorize write tools.

Useful? React with 👍 / 👎.

Comment on lines +652 to +655
bool explicitlyRequested = remove
? ContainsAny(message, "remove link", "delete link", "remove reference", "delete reference")
: ContainsAny(message, "add link", "attach link", "add reference", "attach reference");
if (!explicitlyRequested)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require affirmative commands for reference-link writes

When the latest message merely asks about adding a link, such as “Can you explain how to add link https://example.test to this stack?”, the general intent check accepts Can you, and this substring matcher authorizes add_stack_reference_link because the URL and current-stack reference also match. Fresh evidence after the status, snooze, and critical-command hardening is that reference-link writes still lack their affirmative-command check, so a provider-emitted call can mutate the stack in response to an explanatory question.

Useful? React with 👍 / 👎.

Comment on lines +109 to +110
await using var providerRequest = await assistantUsageService.StartProviderRequestAsync(request.OrganizationId, providerInputCharacters);
using var response = await SendRequestAsync(messages, options, allowTools, request, cancellationToken);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid charging rejected provider requests

When the provider immediately rejects the request with a non-success status—for example because the API key is revoked or the account is rate-limited—the usage lifecycle has already reserved the maximum prompt and completion allowance. SendRequestAsync throws without a usage frame, so disposal converts that reservation into durable conservative usage even though no generation occurred; repeated 401/400/429 responses can exhaust an organization's monthly Exie allowance during a configuration error or provider outage. Only finalize fallback usage after the provider has accepted or begun streaming the request.

Useful? React with 👍 / 👎.

@github-actions

Copy link
Copy Markdown

Code Coverage

Package Line Rate Branch Rate Complexity Health
Exceptionless.Core 75% 67% 10113
Exceptionless.AppHost 38% 40% 143
Exceptionless.Insulation 37% 35% 286
Exceptionless.Web 84% 68% 7867
Summary 78% (24900 / 31956) 67% (11659 / 17449) 18409

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants