diff --git a/biome.json b/biome.json index 517be7287..b7c87298d 100644 --- a/biome.json +++ b/biome.json @@ -1,7 +1,10 @@ { "$schema": "https://biomejs.dev/schemas/2.4.12/schema.json", "vcs": { "enabled": true, "clientKind": "git", "useIgnoreFile": true }, - "files": { "ignoreUnknown": false, "includes": ["**", "!**/*.css"] }, + "files": { + "ignoreUnknown": false, + "includes": ["**", "!**/*.css", "!**/design/**", "!**/.worktrees/**"] + }, "formatter": { "enabled": true, "indentStyle": "tab", @@ -92,7 +95,14 @@ "useGetterReturn": "error" } }, - "includes": ["**", "**/dist", "**/.eslintrc.cjs", "!**/*.css"] + "includes": [ + "**", + "**/dist", + "**/.eslintrc.cjs", + "!**/*.css", + "!**/design/**", + "!**/.worktrees/**" + ] }, "javascript": { "formatter": { "quoteStyle": "double" } }, "overrides": [ diff --git a/design/DESIGN.md b/design/DESIGN.md new file mode 100644 index 000000000..8450dcb04 --- /dev/null +++ b/design/DESIGN.md @@ -0,0 +1,156 @@ +# OpenScreen Design System + +> Category: Video & Productivity Tools +> Surface: Desktop web (Electron) +> Source: tokens extracted from `openscreen-editor-2.html` (canonical) + +A high-fidelity, professional dark-and-light desktop application design system +optimized for video recording, post-processing editing, and timeline +manipulation. Mint is the single brand accent; red is reserved strictly for +REC / cut / skip / trim / transcript-highlight states; amber covers warnings. + +--- + +## Color Palette + +### Light theme (`:root`) + +| Role | Token | Hex | Notes | +|---|---|---|---| +| Canvas | `--bg` | `#fafbfc` | Off-white, never pure white; soft radial gradient | +| Panel | `--surface` | `#ffffff` | Raised panels on canvas | +| Card | `--surface-1` | `#f7f8fa` | Tier 1 raised | +| Tier 2 | `--surface-2` | `#f3f5f8` | Raised on panel | +| Hover | `--surface-3` | `#eef0f3` | Active/hover state | +| Popover | `--surface-hi` | `#ffffff` | Highest elevation | +| Border | `--border` | `#e7e9ee` | Soft hairline | +| Border soft | `--border-soft` | `#f1f2f5` | Subtle dividers | +| Border hi | `--border-hi` | `#d1d5db` | Emphasis lines | +| Foreground | `--fg` | `#1f2937` | Slate, not pure black | +| Foreground emphasis | `--fg-emphasis` | `#111827` | Headlines | +| Muted | `--muted` | `#6b7280` | Body / labels | +| Meta | `--meta` | `#9ca3af` | Timestamps, captions | +| Brand (mint) | `--accent` | `#10b981` | Primary action, focus ring, toggle-on | +| Brand glow | `--brand-glow` | `rgba(16,185,129,0.35)` | Focus / hover | +| Danger | `--danger` | `#ef4444` | REC, skip, trim, transcript highlight | +| Warning | `--warn` | `#f59e0b` | Soft amber | +| Success | `--success` | `#10b981` | Alias of brand | + +### Dark theme (`:root[data-theme="dark"]`) + +| Role | Token | Hex | Notes | +|---|---|---|---| +| Canvas | `--bg` | `#0a0d12` | Near-black with faint blue cast | +| Panel | `--surface` | `#14181f` | Tier 1 panel | +| Card | `--surface-1` | `#14181f` | Same as panel base | +| Tier 2 | `--surface-2` | `#1c2029` | Raised | +| Hover | `--surface-3` | `#252a35` | Hover/active | +| Popover | `--surface-hi` | `#2e3440` | Highest | +| Border | `--border` | `#252a35` | Reads on canvas | +| Foreground | `--fg` | `#e6e9ef` | Off-white, never pure `#ffffff` | +| Muted | `--muted` | `#8b95a3` | Tuned for AA on dark | +| Brand (mint) | `--accent` | `#10b981` | Same hue, brighter tone on `--brand-lo: #34d399` | +| Danger | `--danger` | `#f87171` | Softer red for dark bg | + +### Traffic lights (macOS chrome) + +| Token | Hex | +|---|---| +| `--light-red` | `#ff5f57` | +| `--light-yellow` | `#febc2e` | +| `--light-green` | `#28c840` | + +--- + +## Typography + +- **Display:** system-ui stack — `system-ui, -apple-system, "Segoe UI", "Helvetica Neue", Arial, sans-serif` +- **Body:** system-ui stack (same as display) +- **Mono:** `ui-monospace, "SF Mono", Menlo, Monaco, Consolas, monospace` + +### Type scale (desktop editor, base 13px) + +| Token | Size | Use | +|---|---|---| +| `--fs-app` | 13px | Base UI | +| `--fs-app-sm` | 12px | Secondary | +| `--fs-app-lg` | 14px | Primary UI | +| `--fs-title` | 16px | Panel titles | +| `--fs-section` | 11px | Section headers (uppercase, tracked) | +| `--fs-display` | 24px | Hero numerals | + +Nothing below 11px. + +--- + +## Layout + +- **Radius:** 8px standard; `--r-xs: 4px`, `--r-sm: 6px`, `--r-md: 8px`, `--r-lg: 12px`, `--r-pill: 9999px` +- **Border weight:** 1px +- **Spacing:** 4px baseline grid (`--sp-1` through `--sp-6`) +- **Editor wireframe:** + - Titlebar: 34px + - Left utility rail: 48px + - Left panel: 320px + - Right properties: 320px + - Resize handle: 6px + - Bottom timeline/toolbar: 224px + +### Posture rules + +- **One accent, used at most twice per screen.** Default budget is eyebrow + primary CTA. +- **Red is reserved** for REC, cut, skip, trim, and transcript highlight only. It is not a brand color. +- **Mint is the single brand color** for active state, focus ring, toggle-on, and brand mark. +- **Off-white canvas, tiered surfaces.** Never use pure `#000` or pure `#fff` for editorial chrome. +- **Soft elevation, slate-based shadows.** Two `--elev-card` and `--elev-pop` are enough. +- **System-ui type, mono numerics.** No web fonts loaded; ships the declared fallback stack. + +--- + +## Motion + +- `--motion-fast: 120ms` — hover, focus ring +- `--motion-base: 180ms` — state transitions +- `--ease: cubic-bezier(0.2, 0, 0, 1)` — standard easing + +--- + +## Voice & Tone + +- **Adjectives:** high-fidelity, professional, calm, restrained. +- **Tone:** a confident dark-themed tool. Marketing prose is sparse; product UI carries the work. +- **Messaging pillar:** video recording, post-processing editing, and timeline manipulation — same three jobs the editor does. + +### Vocabulary + +- **Use:** Record, Trim, Cut, Skip, Timeline, Clip, Track, Transcript. +- **Avoid:** playful emoji feature labels (✨ 🚀 🎯), "AI-powered" in product chrome, generic SaaS copy ("supercharge your workflow"). + +--- + +## Imagery + +- **Style:** schematic, palette-derived. Scene illustration in preview is grayscale made from the same neutrals — not literal artwork. +- **Treatment:** replace placeholder scenes with real project footage when shipping. +- **Avoid:** stock photography, decorative illustration in tool chrome, hand-drawn mascots. + +--- + +## Files in this directory + +| File | Role | +|---|---| +| `openscreen-editor-2.html` | Latest editor (light + dark themes in one file). Canonical source of tokens. | +| `openscreen-editor.html` | First editor pass. Light-only. | +| `editor.html` | Earlier experimental editor with red REC accent. | +| `openscreen-landing.html` | Marketing landing page. | +| `index.html` | Launcher / overview. | +| `DESIGN.md` | This document. | + +--- + +## Open questions / follow-ups + +- `editor.html` uses a different red-dominant accent and predates the mint brand color. Keep as historical reference; do not import its tokens into new work. +- Landing page (`openscreen-landing.html`) is light-only; consider a dark variant once product photography is sourced. +- No logo asset is committed yet. Brand mark should be a simple wordmark or geometric mark in mint on the dark canvas. diff --git a/design/openscreen-editor.html b/design/openscreen-editor.html new file mode 100644 index 000000000..8a740b1c5 --- /dev/null +++ b/design/openscreen-editor.html @@ -0,0 +1,4686 @@ + + + + + + + OpenScreen — Éditeur + + + +
+ + +
+ + + + OpenScreen + + + + demo2 + + + + Enregistré · il y a 2 s + + + + + + + + + + + + + + + + +
+ + +
+ + + + + + + + + + + +
+ + +
+
+
+ + 00:00:12.4 + 1920 × 1080 · 60 fps +
Aperçu · 16:9
+
+ +
+
+
+ +
+ + +
+ + +
+ + + + + + + + + + + + + + + + + + + +
+ + + + +
+ + +
+ + +
+ + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/architecture/ai-edition-collision-analysis.md b/docs/architecture/ai-edition-collision-analysis.md new file mode 100644 index 000000000..f469b8925 --- /dev/null +++ b/docs/architecture/ai-edition-collision-analysis.md @@ -0,0 +1,355 @@ +# AI Edition Merge — Collision Analysis & Edge Cases + +> Cross-references `ai-edition-merge-plan.md`, `openscreen-inventory.md`, `axcut-inventory.md`. Every collision listed here must be resolved before its owning phase starts. +> +> **Severity legend:** 🔴 breaks the merge · 🟠 will cause user-visible regressions · 🟡 polish / nice-to-have. + +--- + +## 1. Schema collisions (Phase 0) + +### 1.1 Both projects have `version: 2` but mean different things 🔴 +- OpenScreen: `projectPersistence.ts:65` — `PROJECT_VERSION = 2`, denotes the `EditorProjectData` envelope. +- axcut: `axcut-schema/src/index.ts:1` — `axcutSchemaVersion = 2`, denotes the `AxcutDocument` shape. +- **Resolution:** bump everything to v3. Single `SCHEMA_VERSION = 3` constant lives in `src/lib/ai-edition/schema/`. Migration is bidirectional: v2 OpenScreen ↔ v3 ↔ v2 axcut. Document the migration paths in `migrateDocument(value, fromVersion)`: + - `fromVersion = 2, source = 'openscreen'` → wrap `EditorProjectData` into `AxcutDocument` (single asset, single clip spanning full duration, trimRegions inverted to skipRanges). + - `fromVersion = 2, source = 'axcut'` → pass-through (already valid). + - `fromVersion = 3` → no-op. +- Auto-detect source by sniffing for `media.screenVideoPath` (OpenScreen) vs `assets[]` (axcut). No source-of-origin field needed in v3. + +### 1.2 Axcut's merged primary `transcript` vs per-asset `transcripts[]` 🟠 +- `axcut-schema/src/index.ts:96-100` — `documentSchema` carries **both** a single merged `transcript` and a `transcripts[]` array. +- `document-service.ts:73-83` — every `updateTranscript` call merges all per-asset transcripts into the single primary via `mergeTranscripts`. +- The merge exists because axcut's `VirtualPreview` only consumes one transcript. +- **Collision:** with multi-asset projects, the merge is lossy — you can't tell which words came from which asset. +- **Resolution:** Phase 1 stores only `transcripts[]`; `VirtualPreview` (Phase 2) reads the per-clip `clip.wordRefs[]` instead of a single primary. Drop the `transcript` (singular) field in v3. + +### 1.3 OpenScreen's `trimRegions` are inverses of axcut's `skipRanges` 🟠 +- OpenScreen: `types.ts:211` — `TrimRegion { id, startMs, endMs }` — regions are **kept**. +- axcut: `axcut-schema/src/index.ts:79` — `skipRangeSchema` — regions are **removed**. +- **Resolution:** migration from v2 OpenScreen inverts: each `trimRegion` becomes a `skipRange` pair (everything outside trim = kept, inside trim = skipped). For the simple case of a single trim region `[a, b]`, you get `skipRanges = [{startSec: 0, endSec: a}, {startSec: b, endSec: duration}]`. +- **Edge case:** when an OpenScreen project has **zero trim regions** (entire recording kept), the migrated project has `skipRanges: []` and **one clip spanning the full duration**. This is the "single-take" baseline — should be the most common migration result. + +### 1.4 OpenScreen has fields axcut lacks 🔴 +| OpenScreen field | Map to AxcutDocument v3 | +|---|---| +| `editor.trimRegions[]` | invert into `skipRanges[]` | +| `editor.zoomRegions[]` | new `zoomRanges[]` field in `documentSchema` | +| `editor.annotationRegions[]` | new `annotations[]` field in `documentSchema` | +| `editor.speedRegions[]` | already in axcut as `timeline.speedRanges[]` (rangeSchema shape) | +| `editor.{wallpaper, shadowIntensity, showBlur, motionBlurAmount, borderRadius, padding, cropRegion, aspectRatio, webcamLayoutPreset, webcamMaskShape, webcamMirrored, webcamReactiveZoom, webcamSizePreset, webcamPosition, cursorTheme}` | new `legacyEditor` envelope; applied at export time | +| Cursor telemetry (`useScreenRecorder.ts` writes `.cursor.json` next to WebM) | **NOT** in document; stays as sidecar asset metadata. Add `asset.cursorTelemetryPath?` to `assetSchema`. | + +### 1.5 Schema fields need asset-scoped addressing 🟠 +- In a single-asset OpenScreen project, `zoomRegion.startMs/endMs` are timeline-absolute. +- In a multi-asset project, a zoom region applies to **one source asset** within one **clip**. The simplest model is: zoom regions are clip-scoped, not document-scoped. +- **Resolution:** in v3, `zoomRanges[]`, `annotations[]`, and `speedRanges[]` all carry `clipId?` (optional — null means "apply globally if there's exactly one clip, otherwise error"). OpenScreen's migrated projects (one clip) keep their regions with `clipId: null`. Multi-clip projects require `clipId`. +- This needs a `resolveRegionScope(document, region)` helper. + +### 1.6 axcut's `clip_` id generation is fragile 🟡 +- `timeline.ts:50-66` — auto-generated ids `clip_1, clip_2, ...`. New clips scan for the next free index. Could collide with user-provided ids. +- **Resolution:** use `crypto.randomUUID()` for all auto-generated clip ids (matching `TimelineEditor.tsx:17` pattern OpenScreen already uses). Update `retimeClips` to skip the numeric generation. + +--- + +## 2. Process model collisions (Phases 0, 6) + +### 2.1 Python worker + Faster-Whisper → drop entirely 🔴 +- axcut spawns Python as a child process via `PythonWorker` (`apps/server/src/services/python-worker.ts:14`). On Windows, finding `python` on PATH is unreliable; on macOS, codesigning a Python interpreter is a nightmare. +- OpenScreen's `transformers.js` Whisper (`src/lib/captioning/transcribe.ts:44`) runs in a Web Worker, already bundled via `caption-assets/`. +- **Resolution:** axcut's `enqueueTranscription` becomes a renderer-side call to `transcribeMono16kToSegments`. `enqueueAssetIngest` (which calls probe + proxy + transcribe) becomes: renderer reads metadata via `mediabunny` probe; transcribes via Whisper; updates the document. +- **Edge case:** axcut's Whisper defaults to `medium` (1.5 GB); OpenScreen bundles `tiny` (~75 MB). Need a user setting `transcriptionModel` (default `tiny`, opt-in to `medium`). The model download flow must happen in the renderer (transformers.js) — no Python download server. + +### 2.2 Fastify + SSE → Electron IPC + `webContents.send` 🔴 +- axcut: Fastify listens on `127.0.0.1:4010`, SSE stream at `GET /api/projects/:id/events`. +- OpenScreen: Electron main process + IPC handlers. +- **Resolution:** every axcut route becomes a `native-bridge:invoke` domain action (or a new `ai-edition:invoke` channel). The EventBus stays in the main process; fan-out happens via `webContents.send('ai-edition:event', { projectId, event })`. Renderer subscribes via a new `useAiEditionEvents(projectId)` hook. +- **Edge case:** SSE supports auto-reconnect; raw `webContents.send` does not. If the renderer reloads (e.g. user opens DevTools and hits Cmd+R), it must re-subscribe on mount. Plan accordingly in `VideoEditor.tsx`'s `useEffect`. +- **Edge case:** `webContents.send` only reaches the window it targets. Multiple editor windows? OpenScreen currently has one editor window; verify the renderer doesn't open multiple before assuming fan-out works. + +### 2.3 SQLite location `.axcut-data/metadata.sqlite` → `app.getPath('userData')` 🔴 +- axcut hardcodes the data dir under the repo. +- OpenScreen uses `app.getPath('userData')` (the OS-standard user data location: `~/Library/Application Support/OpenScreen` on macOS, `%APPDATA%/openscreen` on Windows). +- **Resolution:** move SQLite to `app.getPath('userData')/ai-edition/metadata.sqlite`. WAL mode is fine for Electron. +- **Edge case:** `better-sqlite3` is a native module. OpenScreen already has `@electron/rebuild` in devDeps — verify it's wired in `electron-builder.json5` rebuild step. If not, add `npm rebuild better-sqlite3` to the build script. + +### 2.4 axcut's `queueMicrotask` job runner → real cancellation 🔴 +- axcut `job-service.ts:18-22` schedules work via `queueMicrotask(() => void runX(...))`. No way to cancel. +- **Edge case:** user closes the editor mid-ingest. axcut's Fastify stays alive, the job keeps running, the renderer never sees the result. User is confused. +- **Resolution:** main-process job runner tracks `Set` per window. When the editor window closes, mark all its jobs as `cancelled` and abort the underlying `Worker.terminate()` (caption) or `AbortController.abort()` (HTTP fetch for transcription). + +### 2.5 axcut credentials in plain JSON → `safeStorage` 🟠 +- axcut: `llm-credentials.json` is plain JSON, written under `.axcut-data/`. +- **Security regression:** OpenScreen should not store credentials as plain JSON. Use Electron's `safeStorage.encryptString()` (OS keychain: Keychain on macOS, DPAPI on Windows, libsecret on Linux). +- **Resolution:** `LlmConfigService` (Phase 7) encrypts keys with `safeStorage` before write; decrypts on read. Keys migrate: on first run, detect plain JSON and re-encrypt. +- **Edge case:** `safeStorage.isEncryptionAvailable()` may return false on Linux without a desktop session. Fallback: `electron-store` with a user-set passphrase, or warn + opt-out. + +--- + +## 3. UX / UI collisions (Phases 1, 5) + +### 3.1 Single recording vs multi-take workflow 🔴 +- Today: `useScreenRecorder.ts:621` `stopRecording` → `switchToEditor()` is unconditional. +- Plan said: stay-in-recorder with prompt. **Conflict with the existing flow.** Need a decision: + - **Stay-in-recorder** (Screen Studio feel): every stop → toast "Saved to project X. Record another or open editor?" → two buttons. + - **Auto-open-editor** (today's feel): every stop → editor opens. The user's mental model matches existing. +- **Recommendation:** default to **stay-in-recorder** when `aiEdition` flag is on and there's an active project; default to **auto-open-editor** when there's no active project. The HUD needs a "Switch project" / "New project" affordance. +- **Edge case:** if the user starts recording without an active project, axcut's flow says "create a new project first." OpenScreen today has no project concept on the HUD. Need a quick-create button. +- **Edge case:** what if recording fails (stream error)? Plan path: don't create the asset; don't switch window; show toast. + +### 3.2 `Ctrl+C / Ctrl+V` clip duplicate collides with Electron Edit menu 🔴 +- Inventory §9 already flagged this. **Resolution:** + - Override the application menu's `Edit > Copy` / `Edit > Paste` roles to call `webContents.send('editor:clipboard', { op: 'copy' | 'paste' })` when the editor window is focused. The editor's React handler intercepts and triggers clip duplicate. + - In text inputs (``, `