Tracking Issue: NLnet grant milestones

[Keavon]

The ongoing development milestones tracked here are graciously funded with support of the NLNet NGI0 Commons Fund More details: https://nlnet.nl/project/Graphite/

Within the scope of this project, Graphite will be integrating keyframe animation to bring vector graphics to life with motion, expanding upon existing support for live playback of mathematically-driven motion. Also, new and updated tool controls and panel layouts will make the interface more familiar, intuitive, and accessible to users of all skill levels. Furthermore, development of a new document file format will usher Graphite out of its experimental era by supporting backwards-compatible save files that can be relied upon to open correctly after many new application versions.

Floating menus system support for context menus and tooltips

Many current features in Graphite are buried beihind keyboard shortcuts and lack discoverability. This set of improvements will make the interface more accessible by surfacing functionality through right-click context menu actions and tooltips, helping introduce functionality and shortcuts all throughout the editor.

• Refactor floating menu system to be controlled fully from the Rust side of the codebase (Make font selection show a live preview on hover; move its code to the backend #3487)

  • OUTCOME: The lingering discrepency in the separation of concerns between the content handling (the Rust backend) and visual presentation (the web frontend) has been removed by reimplementing all such functionality in Rust. Across 31 changed files, this data flow has been moved so that the former nonstandard usage of font loading and presentation employs the newly developed plumbing and state management systems for floating menus. Instead of having a font component that duplicates most functionality of the floating menu component, the latter is now used consistently throughout. A significant portion of the work involved handling nuanced bugs until functional parity was reached, and then surpassed. The new system supports a feature that users will appreciate, where font choices are previewed live on the text object.

    Before and after comparison

    Before:

    change_previous.mp4

    After:

    change_after.mp4

• Replace plaintext tooltips with custom-styled floating menu tooltips supporting rich styling and hotkey hints (Replace text-only tooltips with custom richly styled tooltips #3436, Improve tooltip docs with Markdown styling and refined math node explanations #3488, commit 415d4b5)

  • OUTCOME: Throughout the editor, all widgets may not specify header and body text (either being optional) as well as a keyboard or mouse input hint to enhance the discoverability of such input shortcuts. Furthermore, the text supports basic Markdown syntax parsing for displaying bold, italic, and code.

    Before and after comparison

    Before:

    

    After:

    

    Before:

    

    After:

    

• Write new tooltip descriptions for prominent tools, buttons, nodes, and commonly-used functionality throughout the editor to improve clarity and discoverability (Improve tooltip docs with Markdown styling and refined math node explanations #3488, Add tooltip documentation to the Text node and tidy up node catalog categorization #3645, Make the node macro include doc comments when hovering a node's function #3979)

• Implement support for general context menus

• Replace the node graph's existing hacky placeholder context menus with the new general ones

• Populate the editor with context menus on layers, objects in the viewport, and other prominent UI areas most expected by users to have them

Tool control bar

Graphite's current tool control bar UI dates back to the application's earliest code and has never offered a clear, intuitive way to adjust tool settings and object styling. This task will redesign its behavior to align its conceptual model with user expectations, providing a clearer way to adjust the appearance of existing selected objects and of new objects that will be drawn with the active tool.

• Design and finalize plan for new, intuitive UI behavior to reduce confusion about how tools, drawing, and layer styling concepts interact
  • Design mockups for how object styling controls and working colors will look and behave
  • Design mockups for how tool modes and each tool's control bar UI will look and behave
• Implement redesigned tool control bar UI for object styling/appearance (Add Gradient tool control bar buttons, Reverse Stops and Reverse Direction #3830, Fix virtual scrolling MenuList font family dropdowns shrinking when wider content is unmounted #3987, New nodes: 'Gradient Type' and 'Spread Method', and add Gradient tool support for controlling these nodes #4084, Make the Text tool control bar font family, style, and size sync with the selected layer #4118, Make the drawing tools' Weight control sync with the selected layer #4120, Make the Shape tool's modes' parameter controls sync with the selected shape layer #4121, Redesign how the control bar handles fill and stroke colors #4137, Replace the control bar's stroke weight with a full stroke properties popover #4145, Fix hover transfer bugs occurring with the color picker popover #4146)
• Implement redesigned working colors UI and its interactions with selected objects and tools (Add support for gradients with midpoints and add draggable diamonds to the color picker dialog #3813, Make the Gradient tool default to the primary/secondary working colors as its initial gradient #4119)

Node Properties panel

The existing node system in Graphite exposes some moderately technical parameters for adjusting the graphical properties of its operations. But as the node graph evolved with more support for more expressive functionality, the panel's UI has lagged behind and no longer provides the most user-friendly way to understand and control these parameters. This task will revamp the panel to provide an easier to use and understand interface for this commonly used functionality.

• Refine the existing design mockup for the node Properties panel
• Implement the redesigned panel UI to make existing graphics editing functionality more accessible and user-friendly by not requiring the node graph (Add the re-labeled node display name to its Properties panel section header #3930, Use the SpectrumInput widget for the adjustment node Properties panel layouts #4105, Implement dragging to reorder Properties panel node sections #4299, Add persistence to the collapsed state of Properties panel node sections #4300)
• Support explicitly choosing the desired data types of node parameters in a user-friendly way
  • Refactor internal type handling system to support custom application-specific data types like percentages and angles
  • Clean up the user-facing naming of existing data types to be more consistent and intuitive, like DVec2 (internal) to Vec2 (user-facing) (2fa958a, Make the node graph use Table<GradientStops> instead of GradientStops #3837, Migrate remaining node graph data types from Vec to Table #4067, Rename the Table type's "rows" -> "items" and "columns" -> "attributes" everywhere #4130, Rename the "Table" type to "List" everywhere #4133, Simplify and standardize how data types are presented in user-facing strings #4147)
  • Integrate a context menu for viewing and selecting a compatible data type of a node parameter
  • Clean up existing nodes to cleanly utilize this new capability in place of prior workarounds

Object Properties panel

In addition to the technical node parameters, Graphite users also needs a more designer-friendly way to adjust common visual properties of selected objects in the viewport. This goes hand-in-hand with the revamped node Properties panel, the redesigned tool control bar, and the new Timeline panel's keyframe editor, since these four features provide connected conceptual models for adjusting the parameters of graphics in the document. This task involves designing and building this new panel so Graphite feels more like a nontechnical design tool by default.

• Refactor the UI widget layout description format system so it is flexible enough to represent the new UI requirements (Refactor the old menu bar plumbing to use standard TextButtons #3444, Restyle and refactor shortcut labels to send hints bar and welcome screen layouts from Rust #3447, commit 4581689, commit 3c4ad8b, Unwrap the Layout enum to replace it with the WidgetLayout struct now called Layout #3448, commit 2ee8e56, commit 74d9c91, Move the MenuBarMessage up from within PortfolioMessage to the root Message #3459, commit 63217fe, commit e99f30e, Refactor messages.ts by removing class-transformer and JS classes #3858, Refactor the TypeScript data flow for full type safety and auto-generation of Rust types #3865)
  • OUTCOME: The messy widget system that evolved organically into the state it was in has been overhauled with a major refactor coming in at +7741 and -9217 lines of additions/removals, with changes affecting every component and layout in use. The new system removes the complex dance required to implement new widgets and layouts involving more than the simplest case of horizontal lists of widgets in a single row, which allows subsequent sub-milestones to implement the needed layout styles required by the design for a clear and intuitive user experience. All functionality has remained the same insofar as the existing layouts remain unaltered, but the code is now able to cleanly represent those current, as well as future, designs, with significantly reduced maintenance burden and complexity.
• Design a mockup for new this new panel with contextual areas for common object types like primitive shapes, text, vector, and raster
• Implement this new panel with support for editing common properties of selected objects (like colors, dimensions, and fonts) in a non-technical way

Layers panel

The Layers panel is an important tool for organization and goes alongside the properties panels in exposing common functionality. This task involves minor improvements to the existing panel to improve discoverability and visual clarity of some features, as well as fixing layout issues that arise at smaller window sizes.

• Build a design mockup for the Layers panel and add new minor design UI/UX refinements
• Improve the existing panel with minor UI/UX enhancements, like making clipping masks more discoverable (Add Layers panel support for displaying multiple groups with instances of the same children layers #3982, Implement dragging the visibility/lock icons in the Layers panel to toggle each #4152, Implement dragging layers into the group/new/delete buttons in the Layers panel #4153, Add Alt+drag layer duplication to the Layers panel #4216)
• Fix widgets overlapping each other in the Layers panel and throughout the editor at smaller window sizes

Timeline panel basic animation controls

Bringing animation to Graphite requires a new panel for managing playback time. This is the first step in building the full keyframe animation system, setting out a way to control playback and export.

• Rendering multiple frames for animation export to PNG, JPEG, and SVG image sequences (Add animated GIF file export to Graphene CLI #3846)
• Initial UI for the Timeline panel with playback controls, time display, and playhead within the time ruler
• Playback controls for framerate, scrubbing, and looping animation time ranges
• Modes for both continuous seconds and discrete frames and conversions between them

Timeline panel dope sheet editor

The dope sheet is a common, intuitive way to visualize and edit keyframes for animation. This task involves extending the basic Timeline panel with functionality for driving node parameters with keyframes.

• Base infrastructure and plumbing for the Timeline panel's connection to the node graph
• UI for dope sheet channels and keyframes and hook up its basic functionality for scalar values
• Driving node parameter values with animation curves data created in the Timeline panel

Timeline panel advanced features and animation curves editor

Building upon the dope sheet editor, this task involves adding more advanced features that animators expect for precise control over motion and timing. This task puts the finishing touches on the initial animation system.

• Animatable composite data types like Vec2 decomposable into multiple scalar channels that can be controlled by curves
• Modes for both manual keying and auto keying
• Design a mockup for the Timeline panel's curves editor view
• UI for the curves editor view according to standard animation software conventions

Stable document format history management

Graphite has been built without a permanent document file format which it requires to be depended on for production use and adopted as an open standard in the graphics ecosystem. This task involves the initial research and development for the core building blocks of this newly designed format.

• Design the structure of the new replacement format (Add document-format RFC draft #4264)
  • OUTCOME: The design of the new .gdd format is now published in the repository as an RFC at node-graph/rfcs/document-format.md. It specifies a flat registry of nodes, slots, and attributes, the deltas that record authoring history, the content-addressable resource model, and the layout of the file on disk. The central decision is that the on-disk layout no longer mirrors the editor's in-memory Rust types, so internal refactors of those types stop forcing breaking changes to the file format. The RFC was written against the working implementation rather than ahead of it, so its contents were repeatedly corrected to match how the shipped code actually behaves.
• Build the on-disk representation of the node graph data and of deltas for authoring history (Add graph-storage crate #4198, Add document-format crate #4234)
  • OUTCOME: Two new crates hold the data model of the format. The document/graph-storage crate stores a document as a flat registry together with the deltas that produced it, replacing the recursive in-memory network with a shape that can be diffed, appended to, and merged. The document/format crate wraps this in a typed handle for .gdd files and covers the serialization codecs, the layout of paths inside the document, and the persistence of session state such as the working copy and the history cursor. Both crates ship with tests that assert a document survives conversion into storage and back without changing.
• Add support for reusable node definitions by "aliasing" another node in the document
• Research and integrate a CRDT algorithm for applying graph data deltas to sync up concurrent offline edits to the same file (Add graph-storage crate #4198, Add History type and merge-commit model to graph-storage #4238)
  • OUTCOME: Authoring history is now an operation-based CRDT rather than a linear undo stack. Every edit becomes a delta stamped with a Lamport clock so that concurrent edits can be ordered causally, and deletions are recorded as timestamped tombstones so that an edit to a node deleted elsewhere resolves the same way on both peers. The committed deltas form an append-only graph held in topological order by a dedicated History type, which merges two divergent histories into the same result on either side by sorting them canonically. Delta computation was also made deterministic, so that two peers performing the same edit produce identical deltas, which is a requirement for them to ever converge.
• Additional time for inevitable but not-yet-known research challenges (Drop MemoHash's std Hash impl and route cache-identity hashing through CacheHash #4209, Add graph-storage crate #4198)
  • OUTCOME: The unforeseen challenge was that Graphite's memoization layer treated two different notions of sameness as one. MemoHash used a cached hash for both its identity and its equality, so values that had round-tripped through the new storage format could compare as equal while differing in meaning, which caused stale cached renders to be reused. The fix gives MemoHash an equality based on the value it wraps and routes cache identity through a separate CacheHash. The remaining time went into hardening the storage crates against malformed files, replacing unchecked casts and panicking code paths with checked conversions and typed errors.

Stable document integration of new storage layer with existing code

After building the new document storage format in isolation, this task involves integrating it with the existing codebase so that Graphite can start using it for representing documents in memory and authoring history.

• Conversion from current in memory document representation to new storage format (Add graph-storage crate #4198)
  • OUTCOME: The editor's in-memory node network is lowered into the flat storage registry by a dedicated conversion. Because the runtime representation allows shapes that the storage format has to reject, the conversion validates as it goes and checks references that cross network boundaries instead of assuming they are valid. Scope injections, which previously existed only as runtime state, are now captured so that they survive being saved.
• Conversion from storage format to in memory representation (Add graph-storage crate #4198)
  • OUTCOME: The inverse conversion rebuilds the editor's runtime network from the stored registry, restoring nested networks, node identities, and input connections. The two directions are tested together over a set of documents, so that loading a saved file produces a graph indistinguishable from the one that was saved.
• Translation of in-memory graph modification deltas to storage deltas and vice versa (Add graph-storage crate #4198, Add History type and merge-commit model to graph-storage #4238, Integrate document-format storage into the editor #4265)
  • OUTCOME: Edits made through the editor's existing graph modification paths are diffed into storage deltas and applied back in the other direction, which keeps both representations in step without rewriting the editor around the new format. Attribute changes are split into individually addressable diffs rather than replacing a whole node, so that deltas stay small and concurrent edits to different properties of the same node can still be merged. Utilities for diffing two networks and two resource registries were added as well, since divergence between the representations is otherwise hard to locate.
• Hooking up the current history system to use the new storage representation (Integrate document-format storage into the editor #4265, Add support for opening GDD documents from the CLI #4262)
  • OUTCOME: Across 31 changed files, the editor now opens, edits, autosaves, and closes .gdd documents alongside the legacy .graphite format, and the frontend recognizes the new file extension. Undo and redo are driven by a cursor over the stored delta history instead of the previous snapshot stack, and the working copy is written asynchronously so that saving does not interrupt editing. The same storage path is exercised outside the editor by graphene-cli, which can now open both legacy and .gdd documents. End-to-end tests cover the full round trip through a file and back.

Stable document asset database + on-disk storage

Once the new document storage layer is integrated, this task involves additional requirements for handling embedded and linked assets like image files, as well as extending the new system to write to, and be read from, actual files on disk. At this point, it can replace the existing bulky, fragile JSON serialization files in use today.

• Create a content-addressable asset database to store resources like imported images (Implement content-addressed resource storage for raster images #4148, Migrate fonts to be stored as resources #4165, Define data types for ID-based resource handling #4168, Use ResourceId for referring to Resources #4176, Make Resource aware of its own hash #4187, Introduce network request handler #4194, Resolve scopes before flattening network #4235, Rework clipboard handling to carry embedded resources across documents #4296, Add document-format crate #4234, Integrate document-format storage into the editor #4265)
  • OUTCOME: Everything content-addressable in a document is now a resource identified by the hash of its contents, including raster images, fonts, embedded WASM, and proto-node declarations. An identical asset is therefore stored only once no matter how many times the document references it. Each resource carries an ordered list of the places its bytes can be obtained from, such as bytes embedded in the document, a path on disk, a URL, or a system font. That list is resolved in order on load and converges under concurrent edits in the same way the rest of the document does.
• Allow referencing nodes from other files
• Design and implement compressed container format and metadata storage (Add document-container crate #4191, Add document-format crate #4234)
  • OUTCOME: The document/container crate provides the virtual filesystem that a .gdd file is written into. It exposes one interface with three interchangeable backends, one held in memory for tests, one using memory-mapped files on native platforms, and one using OPFS in the browser. Documents serialize to and from streaming zip and xz archives, and the codec is detected automatically on load so that the choice of compression is not baked into the format. Making the backends behave identically took much of the effort, including validating paths so that symlinks are rejected and coalescing appends on OPFS to avoid quadratic copying.
• Design and implement template for document format migrations

Stable document format serialization and backwards-compatibility

After the new document format is fully integrated and able to read and write files, this task involves building the systems required to ensure that files remain backwards-compatible as Graphite continues to develop over time. This will ensure users can rely on their files opening correctly in future versions of the application, easing adoption for production use of Graphite.

• Develop a system for providing an upgrade path from old to new versions of nodes and their usages in the document
• Implement a system for testing a collection of example files for backwards-compatibility failures
• Build a way for users to choose whether specific assets should be embedded in the document file or linked from disk