Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/sbom-compliance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:

steps:
- name: Checkout Code
uses: actions/checkout@v4
uses: actions/checkout@v6

- name: Generate SBOM (${{ matrix.component }})
uses: anchore/sbom-action@v0
Expand All @@ -41,22 +41,22 @@ jobs:
upload-artifact: false

- name: Scan SBOM for Vulnerabilities
uses: anchore/scan-action@v6
uses: anchore/scan-action@v7
id: scan
with:
sbom: ${{ matrix.component }}-sbom.spdx.json
fail-build: false # Don't block builds yet, just report
severity-cutoff: high

- name: Upload SBOM Artifact
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: ${{ matrix.component }}-sbom
path: ${{ matrix.component }}-sbom.spdx.json
retention-days: 5

- name: Upload Vulnerability Report
uses: github/codeql-action/upload-sarif@v3
uses: github/codeql-action/upload-sarif@v4
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/stack-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ jobs:
uses: docker/setup-buildx-action@v3

- name: Cache base image
uses: actions/cache@v4
uses: actions/cache@v5
id: base-cache
with:
path: /tmp/base-image.tar.zst
Expand Down Expand Up @@ -280,7 +280,7 @@ jobs:
run: npm ci

- name: Cache Playwright browsers
uses: actions/cache@v4
uses: actions/cache@v5
id: playwright-cache
with:
path: ~/.cache/ms-playwright
Expand Down
2 changes: 1 addition & 1 deletion backend/.env.test
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ KUBERNETES_CONFIG_PATH=/app/kubeconfig.yaml
KUBERNETES_CA_CERTIFICATE_PATH=/app/certs/k8s-ca.pem
K8S_POD_CPU_LIMIT=1000m
K8S_POD_MEMORY_LIMIT=128Mi
K8S_POD_CPU_REQUEST=200m
K8S_POD_CPU_REQUEST=50m

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

# First, check the current state and history of the .env.test file
git log --oneline -n 20 -- backend/.env.test

Repository: HardMax71/Integr8sCode

Length of output: 135


🏁 Script executed:

# Get the git diff to see the actual change
git show HEAD:backend/.env.test | head -20

Repository: HardMax71/Integr8sCode

Length of output: 793


🏁 Script executed:

# Look for all environment files to understand the structure
fd ".env" backend/ -type f

Repository: HardMax71/Integr8sCode

Length of output: 236


🏁 Script executed:

# Search for references to K8S_POD_CPU_REQUEST in the codebase
rg "K8S_POD_CPU_REQUEST" -C 3

Repository: HardMax71/Integr8sCode

Length of output: 2815


🏁 Script executed:

# Search for test configuration files and timeout settings
fd -e yaml -e yml -e py -e js -e ts backend/ | grep -iE "(test|spec|e2e|config)" | head -20

Repository: HardMax71/Integr8sCode

Length of output: 403


🏁 Script executed:

# Look for e2e test configuration and timeout settings
rg -i "timeout|e2e" backend/ --type yaml -C 2 | head -50

Repository: HardMax71/Integr8sCode

Length of output: 307


🏁 Script executed:

# Check git history for K8S_POD_CPU_REQUEST changes
git log -p --all -- backend/.env.test | grep -A 5 -B 5 "K8S_POD_CPU_REQUEST" | head -40

Repository: HardMax71/Integr8sCode

Length of output: 48


🏁 Script executed:

# Look for all .env files to compare
fd ".env" backend/

Repository: HardMax71/Integr8sCode

Length of output: 48


🏁 Script executed:

# Check if there's a production .env or other env files with different values
git ls-files | grep -E "\.env"

Repository: HardMax71/Integr8sCode

Length of output: 110


🏁 Script executed:

# Specifically check the previous version of .env.test before the recent commit
git show HEAD~1:backend/.env.test 2>/dev/null | grep "K8S_POD_CPU_REQUEST" || echo "File or line not found in previous version"

Repository: HardMax71/Integr8sCode

Length of output: 108


🏁 Script executed:

# Check the .env file (non-test) for comparison
cat backend/.env | grep -A 2 -B 2 "K8S_POD"

Repository: HardMax71/Integr8sCode

Length of output: 345


🏁 Script executed:

# Look at the full git history of the .env.test file to see when this setting was added
git log --oneline backend/.env.test | head -10

Repository: HardMax71/Integr8sCode

Length of output: 135


🏁 Script executed:

# Get the complete diff of the recent commit that mentions cpu requests
git show 94f34ab

Repository: HardMax71/Integr8sCode

Length of output: 50382


🏁 Script executed:

# Check if K8S_POD_CPU_REQUEST was previously in .env.test at all
git log --all -S "K8S_POD_CPU_REQUEST" -- backend/.env.test

Repository: HardMax71/Integr8sCode

Length of output: 2057


Verify that tests run reliably with the new low CPU request.

This introduces K8S_POD_CPU_REQUEST=50m in the test environment—significantly lower than the production value of 200m. Given that the PR includes e2e test fixes and timeout adjustments, confirm that the reduced CPU allocation doesn't cause test flakiness or timeouts under typical CI load.

🧰 Tools
🪛 dotenv-linter (4.0.0)

[warning] 13-13: [UnorderedKey] The K8S_POD_CPU_REQUEST key should go before the K8S_POD_MEMORY_LIMIT key

(UnorderedKey)

🤖 Prompt for AI Agents
In `@backend/.env.test` at line 13, The test env lowers K8S_POD_CPU_REQUEST to 50m
which may cause CI flakiness; either restore the test default to the
production-like value (set K8S_POD_CPU_REQUEST back to 200m) or make the test
harness/CI override configurable so tests can run with higher CPU under load;
after changing the K8S_POD_CPU_REQUEST value run the e2e suite repeatedly in CI
(or locally in the same container resource profile) to confirm no timeouts or
flakiness.

K8S_POD_MEMORY_REQUEST=128Mi
K8S_POD_EXECUTION_TIMEOUT=5
K8S_NAMESPACE=integr8scode
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,13 +170,13 @@ async def get_event_stats(self, hours: int = 24) -> EventStatistics:
top_users = [UserEventCount(**doc) for doc in top_users_result if doc["user_id"]]

# Execution duration pipeline
exec_time_field = S.field(ExecutionDocument.resource_usage.execution_time_wall_seconds)
exec_time_field = S.field(ExecutionDocument.resource_usage.execution_time_wall_seconds) # type: ignore[union-attr]
exec_pipeline = (
Pipeline()
.match({
ExecutionDocument.created_at: {"$gte": start_time},
ExecutionDocument.status: "completed",
ExecutionDocument.resource_usage.execution_time_wall_seconds: {"$exists": True},
ExecutionDocument.resource_usage.execution_time_wall_seconds: {"$exists": True}, # type: ignore[union-attr]
})
.group(by=None, query={"avg_duration": S.avg(exec_time_field)})
)
Expand Down
4 changes: 2 additions & 2 deletions backend/app/db/repositories/admin/admin_user_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from datetime import datetime, timezone

from beanie.odm.operators.find import BaseFindOperator
from beanie.operators import Or, RegEx
from beanie.operators import Eq, Or, RegEx

from app.core.security import SecurityService
from app.db.docs import (
Expand Down Expand Up @@ -50,7 +50,7 @@ async def list_users(
)

if role:
conditions.append(UserDocument.role == role)
conditions.append(Eq(UserDocument.role, role))

query = UserDocument.find(*conditions)
total = await query.count()
Expand Down
11 changes: 5 additions & 6 deletions backend/app/db/repositories/event_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
from typing import Any, Mapping

from beanie.odm.enums import SortDirection
from beanie.operators import GTE, LT, LTE, In, Not, Or, RegEx
from beanie.odm.operators.find import BaseFindOperator
from beanie.operators import GTE, LT, LTE, Eq, In, Not, Or, RegEx
from monggregate import Pipeline, S

from app.core.tracing import EventAttributes
Expand Down Expand Up @@ -96,11 +97,9 @@ async def get_events_by_type(
async def get_events_by_aggregate(
self, aggregate_id: str, event_types: list[EventType] | None = None, limit: int = 100
) -> list[DomainEvent]:
conditions = [
EventDocument.aggregate_id == aggregate_id,
In(EventDocument.event_type, list(event_types)) if event_types else None,
]
conditions = [c for c in conditions if c is not None]
conditions: list[BaseFindOperator] = [Eq(EventDocument.aggregate_id, aggregate_id)]
if event_types:
conditions.append(In(EventDocument.event_type, list(event_types)))
docs = (
await EventDocument.find(*conditions).sort([("timestamp", SortDirection.ASCENDING)]).limit(limit).to_list()
)
Expand Down
43 changes: 23 additions & 20 deletions backend/app/db/repositories/saga_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

from beanie.odm.enums import SortDirection
from beanie.odm.operators.find import BaseFindOperator
from beanie.operators import GT, LT, In
from beanie.operators import GT, LT, NE, Eq, In
from monggregate import Pipeline, S

from app.db.docs import ExecutionDocument, SagaDocument
Expand All @@ -14,19 +14,24 @@
class SagaRepository:
def _filter_conditions(self, saga_filter: SagaFilter) -> list[BaseFindOperator]:
"""Build Beanie query conditions from SagaFilter."""
conditions = [
SagaDocument.state == saga_filter.state if saga_filter.state else None,
In(SagaDocument.execution_id, saga_filter.execution_ids) if saga_filter.execution_ids else None,
SagaDocument.context_data["user_id"] == saga_filter.user_id if saga_filter.user_id else None,
SagaDocument.saga_name == saga_filter.saga_name if saga_filter.saga_name else None,
GT(SagaDocument.created_at, saga_filter.created_after) if saga_filter.created_after else None,
LT(SagaDocument.created_at, saga_filter.created_before) if saga_filter.created_before else None,
]
conditions: list[BaseFindOperator] = []
if saga_filter.state:
conditions.append(Eq(SagaDocument.state, saga_filter.state))
if saga_filter.execution_ids:
conditions.append(In(SagaDocument.execution_id, saga_filter.execution_ids))
if saga_filter.user_id:
conditions.append(Eq(SagaDocument.context_data["user_id"], saga_filter.user_id))
if saga_filter.saga_name:
conditions.append(Eq(SagaDocument.saga_name, saga_filter.saga_name))
if saga_filter.created_after:
conditions.append(GT(SagaDocument.created_at, saga_filter.created_after))
if saga_filter.created_before:
conditions.append(LT(SagaDocument.created_at, saga_filter.created_before))
if saga_filter.error_status is True:
conditions.append(SagaDocument.error_message != None) # noqa: E711
conditions.append(NE(SagaDocument.error_message, None))
elif saga_filter.error_status is False:
conditions.append(SagaDocument.error_message == None) # noqa: E711
return [c for c in conditions if c is not None]
conditions.append(Eq(SagaDocument.error_message, None))
return conditions

async def upsert_saga(self, saga: Saga) -> bool:
existing = await SagaDocument.find_one(SagaDocument.saga_id == saga.saga_id)
Expand Down Expand Up @@ -55,11 +60,9 @@ async def get_saga(self, saga_id: str) -> Saga | None:
async def get_sagas_by_execution(
self, execution_id: str, state: SagaState | None = None, limit: int = 100, skip: int = 0
) -> SagaListResult:
conditions = [
SagaDocument.execution_id == execution_id,
SagaDocument.state == state if state else None,
]
conditions = [c for c in conditions if c is not None]
conditions: list[BaseFindOperator] = [Eq(SagaDocument.execution_id, execution_id)]
if state:
conditions.append(Eq(SagaDocument.state, state))

query = SagaDocument.find(*conditions)
total = await query.count()
Expand Down Expand Up @@ -135,10 +138,10 @@ async def get_saga_statistics(self, saga_filter: SagaFilter | None = None) -> di
states[doc["_id"]] = doc["count"]

# Average duration for completed sagas
completed_conditions = [
completed_conditions: list[BaseFindOperator] = [
*conditions,
SagaDocument.state == SagaState.COMPLETED,
SagaDocument.completed_at != None, # noqa: E711
Eq(SagaDocument.state, SagaState.COMPLETED),
NE(SagaDocument.completed_at, None),
]
duration_pipeline = (
Pipeline()
Expand Down
4 changes: 2 additions & 2 deletions backend/app/db/repositories/user_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from datetime import datetime, timezone

from beanie.odm.operators.find import BaseFindOperator
from beanie.operators import Or, RegEx
from beanie.operators import Eq, Or, RegEx

from app.db.docs import UserDocument
from app.domain.enums.user import UserRole
Expand Down Expand Up @@ -38,7 +38,7 @@ async def list_users(
)

if role:
conditions.append(UserDocument.role == role)
conditions.append(Eq(UserDocument.role, role))

query = UserDocument.find(*conditions)
total = await query.count()
Expand Down
14 changes: 8 additions & 6 deletions backend/app/db/repositories/user_settings_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
from datetime import datetime

from beanie.odm.enums import SortDirection
from beanie.operators import GT, LTE, In
from beanie.odm.operators.find import BaseFindOperator
from beanie.operators import GT, LTE, Eq, In

from app.db.docs import EventDocument, UserSettingsDocument, UserSettingsSnapshotDocument
from app.domain.enums.events import EventType
Expand Down Expand Up @@ -37,13 +38,14 @@ async def get_settings_events(
sort_order: SortDirection = SortDirection.ASCENDING,
) -> list[DomainUserSettingsChangedEvent]:
aggregate_id = f"user_settings_{user_id}"
conditions = [
EventDocument.aggregate_id == aggregate_id,
conditions: list[BaseFindOperator] = [
Eq(EventDocument.aggregate_id, aggregate_id),
In(EventDocument.event_type, [str(et) for et in event_types]),
GT(EventDocument.timestamp, since) if since else None,
LTE(EventDocument.timestamp, until) if until else None,
]
conditions = [c for c in conditions if c is not None]
if since:
conditions.append(GT(EventDocument.timestamp, since))
if until:
conditions.append(LTE(EventDocument.timestamp, until))

find_query = EventDocument.find(*conditions).sort([("timestamp", sort_order)])
if limit:
Expand Down
10 changes: 5 additions & 5 deletions backend/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ dependencies = [
"backoff==2.2.1",
"blinker==1.9.0",
"Brotli==1.2.0",
"cachetools==6.2.0",
"cachetools==6.2.4",
"certifi==2024.8.30",
"charset-normalizer==3.4.0",
"click==8.1.7",
Expand All @@ -37,7 +37,7 @@ dependencies = [
"fastavro==1.12.1",
"fonttools==4.61.1",
"frozenlist==1.7.0",
"google-auth==2.45.0",
"google-auth==2.47.0",
"googleapis-common-protos==1.70.0",
"greenlet==3.3.0",
"grpcio==1.76.0",
Expand All @@ -52,13 +52,13 @@ dependencies = [
"Jinja2==3.1.6",
"kiwisolver==1.4.9",
"kubernetes==31.0.0",
"limits==3.13.0",
"limits==5.6.0",
"markdown-it-py==4.0.0",
"MarkupSafe==3.0.2",
"mdurl==0.1.2",
"msgpack==1.1.0",
"multidict==6.7.0",
"oauthlib==3.2.2",
"oauthlib==3.3.1",
"opentelemetry-api==1.22.0",
"opentelemetry-exporter-otlp==1.22.0",
"opentelemetry-exporter-otlp-proto-common==1.22.0",
Expand Down Expand Up @@ -140,7 +140,7 @@ dev = [
"hypothesis==6.103.4",
"iniconfig==2.0.0",
"matplotlib==3.10.8",
"mypy==1.17.1",
"mypy==1.19.1",
"mypy_extensions==1.1.0",
"pipdeptree==2.23.4",
"pluggy==1.5.0",
Expand Down
Loading
Loading