-
Notifications
You must be signed in to change notification settings - Fork 1
Feature/integrations framework #23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
b1120b2
feff4f0
de1bbf7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| """Integrations management and dispatch API routes.""" | ||
|
|
||
| from typing import Any | ||
|
|
||
| from fastapi import APIRouter, Depends, HTTPException, status | ||
| from pydantic import BaseModel | ||
| from sqlalchemy import select | ||
| from sqlalchemy.ext.asyncio import AsyncSession | ||
|
|
||
| from docsgraph_server.db.models import DbIntegrationConfig | ||
| from docsgraph_server.db.session import get_session | ||
| from docsgraph_server.integrations.engine import IntegrationManager | ||
|
|
||
| router = APIRouter(prefix="/integrations", tags=["integrations"]) | ||
|
|
||
|
|
||
| class IntegrationConfigUpdate(BaseModel): | ||
| """Pydantic model for updating an integration configuration.""" | ||
|
|
||
| enabled: bool | ||
| config_json: dict[str, Any] | ||
|
|
||
|
|
||
| class IntegrationStatusResponse(BaseModel): | ||
| """Pydantic model for integration status responses.""" | ||
|
|
||
| id: str | ||
| name: str | ||
| enabled: bool | ||
| config_json: dict[str, Any] | ||
|
|
||
|
|
||
| class DispatchTestRequest(BaseModel): | ||
| """Pydantic model for triggering a test event dispatch.""" | ||
|
|
||
| event_type: str | ||
| payload: dict[str, Any] | ||
|
|
||
|
|
||
| @router.get("", response_model=list[IntegrationStatusResponse]) | ||
| async def list_integrations( | ||
| db: AsyncSession = Depends(get_session), # noqa: B008 | ||
| ) -> list[dict[str, Any]]: | ||
| """List all registered integrations and their current configurations.""" | ||
| manager = IntegrationManager() | ||
| await manager.load_configurations(db) | ||
|
|
||
| response = [] | ||
| for integration_id, integration in manager.registry.items(): | ||
| result = await db.execute( | ||
| select(DbIntegrationConfig).where(DbIntegrationConfig.id == integration_id) | ||
| ) | ||
| db_cfg = result.scalar_one_or_none() | ||
|
|
||
| response.append( | ||
| { | ||
| "id": integration_id, | ||
| "name": integration.get_name(), | ||
| "enabled": db_cfg.enabled if db_cfg else False, | ||
| "config_json": db_cfg.config_json if db_cfg else {}, | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Secret fields should never come back out. The straightforward fix is to keep them out of the response, for example redacting known-sensitive keys or, better, splitting secrets into their own column so a response model cannot include them by accident. Storing them encrypted at rest would be a sensible follow-up, since right now anyone with read access to |
||
| } | ||
| ) | ||
| return response | ||
|
|
||
|
|
||
| @router.post("/{integration_id}/configure", response_model=IntegrationStatusResponse) | ||
| async def configure_integration( | ||
| integration_id: str, | ||
| req: IntegrationConfigUpdate, | ||
| db: AsyncSession = Depends(get_session), # noqa: B008 | ||
| ) -> dict[str, Any]: | ||
| """Configure and enable/disable a specific integration.""" | ||
| manager = IntegrationManager() | ||
| if integration_id not in manager.registry: | ||
| raise HTTPException( | ||
| status_code=status.HTTP_404_NOT_FOUND, | ||
| detail=f"Integration '{integration_id}' not found", | ||
| ) | ||
|
|
||
| result = await db.execute( | ||
| select(DbIntegrationConfig).where(DbIntegrationConfig.id == integration_id) | ||
| ) | ||
| db_cfg = result.scalar_one_or_none() | ||
|
|
||
| if not db_cfg: | ||
| db_cfg = DbIntegrationConfig( | ||
| id=integration_id, | ||
| enabled=req.enabled, | ||
| config_json=req.config_json, | ||
| ) | ||
| db.add(db_cfg) | ||
| else: | ||
| db_cfg.enabled = req.enabled | ||
| db_cfg.config_json = req.config_json | ||
|
|
||
| await db.commit() | ||
|
|
||
| await manager.load_configurations(db) | ||
| integration = manager.registry[integration_id] | ||
|
|
||
| return { | ||
| "id": integration_id, | ||
| "name": integration.get_name(), | ||
| "enabled": db_cfg.enabled, | ||
| "config_json": db_cfg.config_json, | ||
| } | ||
|
|
||
|
|
||
| @router.post("/dispatch") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All three routes in this module are unauthenticated. These are administrative operations and should require an authenticated caller holding an admin permission. Separately, this endpoint is described as dispatching a test event but is indistinguishable from a real one to the receiver; adding a flag in the signed body marking test deliveries would save someone a confusing incident. |
||
| async def dispatch_event( | ||
| req: DispatchTestRequest, | ||
| db: AsyncSession = Depends(get_session), # noqa: B008 | ||
| ) -> dict[str, Any]: | ||
| """Dispatch a test event to all active integrations.""" | ||
| manager = IntegrationManager() | ||
| await manager.load_configurations(db) | ||
| outcomes = await manager.dispatch(req.event_type, req.payload) | ||
| return {"outcomes": outcomes} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| """create_integrations_table. | ||
|
|
||
| Revision ID: 03a58b392db2 | ||
| Revises: 0001 | ||
| Create Date: 2026-08-26 10:37:36.116178 | ||
| """ | ||
|
|
||
| from collections.abc import Sequence | ||
|
|
||
| import sqlalchemy as sa | ||
| from alembic import op | ||
|
|
||
| # revision identifiers, used by Alembic. | ||
| revision: str = "03a58b392db2" | ||
| down_revision: str | None = "0001" | ||
| branch_labels: str | Sequence[str] | None = None | ||
| depends_on: str | Sequence[str] | None = None | ||
|
|
||
|
|
||
| def upgrade() -> None: | ||
| """Create integration_configs table.""" | ||
| # ### commands auto generated by Alembic - please adjust! ### | ||
| op.create_table( | ||
| "integration_configs", | ||
| sa.Column("id", sa.String(length=100), nullable=False), | ||
| sa.Column("enabled", sa.Boolean(), nullable=False), | ||
| sa.Column("config_json", sa.JSON(), nullable=False), | ||
| sa.PrimaryKeyConstraint("id"), | ||
| ) | ||
| # ### end Alembic commands ### | ||
|
|
||
|
|
||
| def downgrade() -> None: | ||
| """Drop integration_configs table.""" | ||
| # ### commands auto generated by Alembic - please adjust! ### | ||
| op.drop_table("integration_configs") | ||
| # ### end Alembic commands ### |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| """SQLAlchemy ORM models for external integrations configuration.""" | ||
|
|
||
| from typing import Any | ||
|
|
||
| from sqlalchemy import JSON, Boolean, String | ||
| from sqlalchemy.orm import Mapped, mapped_column | ||
|
|
||
| from docsgraph_server.db.base import Base | ||
|
|
||
|
|
||
| class DbIntegrationConfig(Base): | ||
| """SQLAlchemy model for storing external integration configurations.""" | ||
|
|
||
| __tablename__ = "integration_configs" | ||
|
|
||
| id: Mapped[str] = mapped_column(String(100), primary_key=True) | ||
| enabled: Mapped[bool] = mapped_column(Boolean, default=False, nullable=False) | ||
| config_json: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict, nullable=False) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # docsgraph-server External Integrations Framework | ||
|
|
||
| The integrations framework enables `docsgraph-server` to connect with third-party external services (such as cloud storage, single sign-on, and event notifier channels) without modifying core server logic. | ||
|
|
||
| --- | ||
|
|
||
| ## 1. Core Interfaces | ||
|
|
||
| All integrations must conform to the `Integration` protocol: | ||
|
|
||
| ```python | ||
| from typing import Any, Protocol | ||
|
|
||
|
|
||
| class Integration(Protocol): | ||
| def get_id(self) -> str: | ||
| """Return the unique string identifier for the integration (e.g. 'webhook_notifier').""" | ||
| ... | ||
|
|
||
| def get_name(self) -> str: | ||
| """Return a user-friendly name for configuration UIs.""" | ||
| ... | ||
|
|
||
| def configure(self, config: dict[str, Any]) -> None: | ||
| """Apply a dynamic configuration dictionary containing options and credentials.""" | ||
| ... | ||
|
|
||
| def is_enabled(self) -> bool: | ||
| """Return whether the integration is configured and active.""" | ||
| ... | ||
|
|
||
| async def execute(self, event_type: str, payload: dict[str, Any]) -> dict[str, Any]: | ||
| """Perform the integration execution on a dispatched event.""" | ||
| ... | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## 2. Reference Implementation: Webhook Event Notifier | ||
|
|
||
| The `WebhookNotifierIntegration` is pre-registered and provides a secure push mechanism for external systems. | ||
|
|
||
| ### Features | ||
| - **HMAC Signatures**: Supports payload authentication. When `secret_key` is supplied, payloads are signed via HMAC-SHA256 and sent in the `X-Docsgraph-Signature` HTTP header. | ||
| - **Async Execution**: Non-blocking network I/O powered by `httpx.AsyncClient`. | ||
|
|
||
| --- | ||
|
|
||
| ## 3. Configuration & Database Mapping | ||
| Integrations are configured at runtime (without changing core code or rebooting the server) via database records in the `integration_configs` table: | ||
|
|
||
| - `id` (PK): The unique integration ID matching `get_id()`. | ||
| - `enabled`: Global toggle to enable/disable. | ||
| - `config_json`: JSON object containing connection parameters, credentials, or URLs. | ||
|
|
||
| API routes under `/api/v1/integrations` allow listing, configuring, and testing integrations. | ||
|
|
||
| --- | ||
|
|
||
| ## 4. Failure Isolation & Graceful Degradation | ||
| To prevent failing external integrations from blocking or breaking core operations, all execution dispatches are isolated. The `IntegrationManager` catches exceptions on execution, logs them, and returns an outcome status without propagation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor:
load_configurationson line 46 has already selected every row fromintegration_configs, and then this queries the same table once per integration inside the loop. Havingload_configurationsreturn the configs it loaded, or keeping them on the manager, removes the extra round trips.