diff --git a/CanvasTextShaping/CanvasTextShaping.md b/CanvasTextShaping/CanvasTextShaping.md new file mode 100644 index 0000000..197cde5 --- /dev/null +++ b/CanvasTextShaping/CanvasTextShaping.md @@ -0,0 +1,474 @@ +# Canvas Text Shaping + +The text support in HTML canvas is very limited. Because 2D canvas is used nowadays to support complex text scenarios such as word processor, the need for complex text support became evident. + +Other alternatives, such as combining HTML-in-Canvas with SVG, are not sufficient for sophisticated clients. While HTML-in-Canvas can render complex scripts perfectly, it lacks basic text-processing functionality such as caret positioning, +hit-testing, and calculating selection rectangles. Implementing a text-on-path editor like this [demo](https://demos.skia.org/demo/canvas_edit/) page is best done using the `HTMLCanvasElement` API, but that would require adding text shaping support to `CanvasText`. + +## API purpose + +Extend the capabilities of `CanvasRenderingContext2D` to support text shaping and layout. This would additionally enable precise caret positioning, hit testing and text selection rectangles calculations. + +## Summary of the Google proposal + +Google has presented this [proposal](https://github.com/fserb/canvas2D/blob/master/spec/enhanced-textmetrics.md). Their proposal addresses a single problem: calculating the [grapheme cluster boundaries](https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries). The cluster boundaries will be respected in displaying, hit-testing, caret positioning and selection rectangles calculations. This allows drawing clusters in isolation. + +The TextMetrics is extended in this proposal to allow calculating the text clusters. A clusters holds but does not expose: its text and its font. This how their APIs can be used to draw text cluster by cluster. + +``` +let tm = ctx.measureText(text); +let clusters = tm.getTextClusters(); + +for(let cluster of clusters) + ctx.fillTextCluster(cluster, 0, 0); +``` + +## Problems with the Google proposal + +This proposal addresses a single problem and does not state how it can be extended in the future. Our takes on this proposal are: + +1. Is not extensible. It does everything in one step which is `getTextClusters()` using the current selected style. It does not allow the client to intervene before the text shaping to achieve custom display for example. +2. It does not support processing rich text where multiple fonts/colors are applied to a line of text. +3. It extends `TextMetrics` and makes it own the text and the shaping info of the measured text. Currently `TextMetrics` just returns the geometry of the measured text. We think CanvasText is more appropriate place to be extended for text-shaping. +4. Is not efficient when displaying text with the same font/color cluster by cluster. + +## WebKit proposal + +This proposal presents different levels of text shaping. You can deal with text, `TextRun` or `GlyphRun`. It also allows the client to change the style before the actual text-shaping. This makes the interface simple and extensible. It addresses most of the text-shaping functionalities. Most of these functionalities are straightforward to implement. A few of them need the system frameworks to be implemented. + +## IDL changes + +The IDL changes below can be grouped into four categories: + +1. GlyphRun Geometry APIs +2. CanvasText text shaping APIs +3. CanvasText GlyphRun drawing APIs +4. Canvas GlyphRuns APIs +5. CanvasText text Geometry APIs + +``` +interface TextRun { + readonly attribute unsigned long start; + readonly attribute DOMString text; +}; + +interface GlyphRun { + readonly attribute TextRun textRun; + + sequence clusters(); + TextMetrics textMetrics(); + unsigned long xToCharPos(double x); + double charPosToX(unsigned long charPos); + DOMRectReadOnly selectionRect(unsigned long start, unsigned long end); +}; + +callback StyleCallback = undefined (TextRun textRun); + +interface mixin CanvasText { + // ... extended from current CanvasText. + + // Text breaking - find the breaking opportunities in text + sequence breakText(DOMString text); + + // Text analysis - `segments` is used to apply custom slicing + sequence analyzeText(DOMString text, optional sequence segments = {}); + + // TextRuns layout - returns the visual order to display TextRuns + sequence layoutTextRuns(sequence textRuns); + + // TextRun shaping. + GlyphRun shapeTextRun(TextRun run); + + // GlyphRun drawing + double fillGlyphRun(GlyphRun run, double x, double y); + double strokeGlyphRun(GlyphRun run, double x, double y); + + // GlyphRuns justification, drawing and geometry. + sequence justifyGlyphRuns(sequence glyphRuns, double width); + + undefined fillGlyphRuns(sequence glyphRuns, double x, double y, optional StyleCallback? styleCallback); + undefined strokeGlyphRuns(sequence glyphRuns, double x, double y, optional StyleCallback? styleCallback); + + unsigned long glyphRunsXToCharPos(sequence glyphRuns, double x); + double glyphRunsCharPosToX(sequence glyphRuns, unsigned long charPos); + sequence glyphRunsSelectionRects(sequence glyphRuns, unsigned long start, unsigned long end); + + // Text extensions + unsigned long textXToCharPos(DOMString text, double x); + double textCharPosToX(DOMString text, unsigned long charPos); + sequence textSelectionRects(DOMString text, unsigned long start, unsigned long end); +} +``` + +## Definitions + +**TextRun:** `analyzeText()` breaks down the text into a series of "runs" based on changes in the text engine required or the text direction. Each of these subdivisions is represented by a `TextRun`. `analyzeText()` returns a sequence of `TextRuns` sorted in the visual order. + +**GlyphRun:** Using the current selected font**,** `shapeTextRun()` applies contextual shaping, ligatures, and character-to-glyph translations required for complex scripts (such as Arabic, Hebrew, or Indic languages). `shapeTextRun()` returns a `GlyphRun` which can be used directly and bypass the shaping step when processing the text.. + +## Drawing text + +**This polyfill shows how the existing `fillText()` API can be implemented using the proposed shaping APIs.** + +Loop through all the `GlyphRuns` in their visual order. Display the `GlyphRun` . Advance the x-position by the width of the `GlyphRun` . + +``` +CanvasText.prototype.fillGlyphRuns = function(glyphRuns, x, y, styleCallback) +{ + for (`const`` `glyphRun of glyphRuns) { + styleCallback?.(glyphRun.textRun); + x += this.fillGlyphRun(glyphRun, x, y); + } +} + +CanvasText.prototype.fillText = function(text, x, y) +{ + const textRuns = this.analyzeText(text); + const visualOrder = this.layoutTextRuns(textRuns); + const glyphRuns = visualOrder.map(index => this.shapeTextRun(textRuns[index])); + this.fillGlyphRuns(glyphRuns, x, y); +} +``` + +This diagram below shows how the text **Hello أهلا ١٢٣ World!** is processed to be displayed: +![Drawing Text](drawing-text.png) +## Hit-testing + +**This polyfill shows how t`extXToCharPos()` API can be implemented using the proposed shaping APIs.** + +Loop through all the `GlyphRuns` in their visual order. If a `GlyphRun` contains the x-position call `xToCharPos()` for this `GlyphRun` and pass the relative x-position. Return the retuned `charPos` relative to the beginning of the text + +``` +CanvasText.prototype.glyphRunsXToCharPos = function(glyphRuns, x) +{ + for (const glyphRun of glyphRuns) { + const width = glyphRun.textMatrics().width; + const runStart = glyphRun.textRun.start; + if (x < width) + return runStart + glyphRun.xToCharPos(x); + x -= width; + } +} + +CanvasText.prototype.textXToCharPos = function(text, x) +{ + const textRuns = this.analyzeText(text); + const visualOrder = this.layoutTextRuns(textRuns); + const glyphRuns = visualOrder.map(index => this.shapeTextRun(textRuns[index])); + return this.glyphRunsXToCharPos(glyphRuns, x, y); +} +``` + +## Getting the caret position + +**This polyfill shows how `textCharPosToX()` API can be implemented using the proposed shaping APIs.** + +Loop through all the `GlyphRuns` in their visual order. If a `GlyphRun` contains the `charPos` , get the x-position of `charPos` within this `GlyphRun`. Return the x-position relative to the beginning of the text. + +``` +CanvasText.prototype.glyphRunsCharPosToX = function(textRuns, glyphRuns, charPos) +{ + let x = 0; + + for (const glyphRun of glyphRuns) { + const width = glyphRun.textMatrics().width; + const runStart = glyphRun.textRun.start; + const runEnd = runStart + glyphRun.textRun.text.length; + + if (charPos >= start && charPos < end) + return x + glyphRun.charPosToX(charPos - runStart); + x += width; + } + + return x; +} + +CanvasText.prototype.textCharPosToX = function(text, charPos) +{ + const textRuns = this.analyzeText(text); + const visualOrder = this.layoutTextRuns(textRuns); + const glyphRuns = visualOrder.map(index => this.shapeTextRun(textRuns[index])); + return this.glyphRunsCharPosToX(glyphRuns, charPos); +} +``` + +## Getting the selection rects + +**This polyfill shows how `textSelectionRects()` API can be implemented using the proposed shaping APIs.** + +Loop through all the `GlyphRuns` in their visual order. If a `GlyphRun` interests with the selection range, get the rectangle of this intersection. Try to combine the new rectangle with the last calculated rectangle if they are adjacent horizontally. Otherwise append a new rectangle. + +``` +CanvasText.prototype.glyphRunsSelectionRects = function(glyphRuns, start, end) +{ + const rects = []; + + for (const glyphRun of glyphRuns) { + const runStart = glyphRun.textRun.start; + const runEnd = runStart + glyphRun.textRun.text.length; + + // Intersect [start, end] with TextRun range + const rangeStart = Math.max(runStart, start); + const rangeEnd = Math.min(runEnd, end); + if (rangeStart >= rangeEnd) + continue; + + const rect = glyphRun.selectionRect(rangeStart, rangeEnd); + + // Try to combine the new rect with the last rect. + // Otherwise add a new rect. + if (rects.length && rects[rects.length - 1].right == rect.left) + rects[rects.length - 1].right = rect.right; + else + rects[rects.length] = DOMRect.fromRect(rect); + } + + return rects; +} + +CanvasText.prototype.textSelectionRects = function(text, start, end) +{ + const textRuns = this.analyzeText(text); + const visualOrder = this.layoutTextRuns(textRuns); + const glyphRuns = visualOrder.map(index => this.shapeTextRun(textRuns[index])); + return this.glyphRunsSelectionRects(glyphRuns, start, end); +} +``` + +## Example 1: Drawing Colored Clusters + +**This example shows how the Goole examples in their [proposal](https://github.com/fserb/canvas2D/blob/master/spec/enhanced-textmetrics.md) can be implemented using the proposed shaping APIs.** + +Loop through all the `GlyphRuns` in their visual order. For every `GlyphRun` get its `clusters()` . Loop through all the clusters in their visual order , set the correct color, draw the cluster and advance the x-position by the width of this cluster. + +``` +// Google example +function fillClustersWithColors(ctx, text, x, y, colors) +{ + const textRuns = ctx.analyzeText(text); + const visualOrder = this.layoutTextRuns(textRuns); + const glyphRuns = visualOrder.map(index => this.shapeTextRun(textRuns[index])); + const clusters = glyphRuns.map(glyphRun => glyphRun.clusters()).flat(); + + // Make sure to color each cluster differently. + ctx.fillGlyphRuns(clusters, x, y, (textRun) => { + ctx.fillStyle = colors[textRun.start % colors.length]; + }); +} +``` + +This method can be called like this + +``` +ctx.font = '60px serif'; +ctx.textAlign = 'left'; +ctx.textBaseline = 'middle'; + +const text = 'Colors 🎨 are 🏎️ fine!'; +const colors = ['orange', 'navy', 'teal', 'crimson']; +fillClustersWithColors(ctx, text, 0, 0, colors); +``` + +The result of this should look like this. Notice the letters ‘f’ and ‘I’ are displayed by one glyph and considered one cluster. +![Drawing Colored Clusters Display](drawing-colored-clusters-display.png) +## Example 2: Drawing Justified Colored Words + +**This example shows how a line of text can be displayed justified and style each word with a different color.** + +Pass the indices of the spaces to `analyzeText()`. Shape all `TextRuns` and call `justifyGlyphRuns()`. Loop through all the justified `GlyphRuns` in their visual order. For every `GlyphRun` set the correct color, then draw it and advance the x-position by its width. + +``` +// Drawing justified colored words +function fillJustifiedColoredWords(ctx, text, x, y, width, colors) +{ + // Enforce word boundaries segmentation. + const segments = ctx.breakText(text); + const textRuns = ctx.analyzeText(text, segments); + + // Create justified GlyphRuns. + const visualOrder = this.layoutTextRuns(textRuns); + const glyphRuns = visualOrder.map(index => this.shapeTextRun(textRuns[index])); + const justifiedGlyphRuns = ctx.justifyGlyphRuns(glyphRuns, width); + + // Make sure to color each justified word differently. + let index = 0; + ctx.fillGlyphRuns(justifiedGlyphRuns, x, y, (textRun) => { + ctx.fillStyle = colors[index++ % colors.length]; + }); +} +``` + +This method can be called like this + +``` +const text = 'Hello أهلا وسهلا ١٢٣ عالم World!'; +const colors = ['orange', 'navy', 'teal', 'crimson']; +fillJustifiedColoredWords(ctx, text, 0, 0, 500, colors); +``` + +The result of using this function can be something like this screenshot +![Drawing Colored Justified Words Display](drawing-colored-justified-words-display.png) +## Example 3: Drawing Styled Text + +**This example shows how a multi-styles line can be displayed using the proposed shaping APIs.** + +Pass the indices of the styles to `analyzeText()`. Loop through all the `TextRuns` in their visual order. Select the desired style before getting the shaping for each sliced `TextRun` . Shape each one of them by calling `shapeTextRun()`. Display the `GlyphRun` and advance the x-position by its width. + +Styles is an array of structures; each structure has an index which indicates where this style starts to apply in the text. + +``` +// Drawing rich text +function fillTextWithStyles(ctx, text, x, y, styles) +{ + const segments = styles.map(style => style.start); + const textRuns = ctx.analyzeText(text, segments); + const visualOrder = ctx.layoutTextRuns(textRuns); + + for (const index of visualOrder) { + const textRun = textRuns[index]; + + // Select the style in the context before shaping + const j = styles.findLast(style => style.start <= textRun.start); + ctx.font = styles[j].font; + ctx.fillStyle = styles[j].color; + + const glyphRun = ctx.shapeTextRun(textRun); + x += ctx.fillGlyphRun(glyphRun, x, y); + } +} +``` + +This function can be called like this: + +``` +const text = 'Hello أهلا ١٢٣ World!'; +const styles = [ + { charPos: 0, font: '32px Times', color: 'black' }, + { charPos: 8, font: '32px Times', color: 'red' }, + { charPos: 17, font: '32px Times', color: 'black' } +]; +fillTextWithStyles(ctx, text, 0, 0, styles); +``` + +The result of this should look like this. Notice the letters ‘ل’ and ‘أ’ are displayed by one glyph and considered one cluster. +![Drawing Text with Styles Display](drawing-text-with-styles-display.png) +The following diagram below shows the steps which should be taken place to process this scenario: +![Drawing Text with Styles](drawing-text-with-styles.png) +## Example 4: Drawing Wrapped Justified Styled Text + +**This example shows how styled text can be wrapped and justified using the proposed shaping APIs.** + +First the word break opportunities in the text and merge them with the style segments. Pass the combined sorted segmentation to `analyzeText()` . Loop through the `TextRuns` in their logical order. Shape and measure all the `GlyphRuns` of a word. Try adding the word to the current line. If it does not fit, justify and draw the current line and advance to the next line. Add the `GlyphRuns` of the word to the current line and move to the next word. The last line should not be justified when it is drawn. + +``` +function glyphRunsInVisualOrder(ctx, glyphRuns) + { + // Reorder the GlyphRuns in their TextRuns' visul order. + const textRuns = glyphRuns.map(glyphRun => glyphRun.textRun); + const visualOrder = ctx.layoutTextRuns(textRuns); + return visualOrder.map(index => glyphRuns[index]); +} + +function fillGlyphRunsWithStyles(ctx, glyphRuns, x, y, styles) + { + // Fill GlyphRuns given in logical order. + const orderedGlyphRuns = glyphRunsInVisualOrder(ctx, glyphRuns); + ctx.fillGlyphRuns(orderedGlyphRuns, x, y, (textRun) => { + const j = styles.findLast(style => style.start <= textRun.start); + ctx.font = styles[j].font; + ctx.fillStyle = styles[j].color; + }); +} + +function fillJustifiedGlyphRunsWithStyles(ctx, glyphRuns, x, y, styles, width) +{ + // Justify and fill GlyphRuns given in logical order. + const justifiedGlyphRuns = ctx.justifyGlyphRuns(glyphRuns, width); + fillGlyphRunsWithStyles(justifiedGlyphRuns, x, y, styles); +} + +function fillWrappedJustifiedTextWithStyles(ctx, text, x, y, styles, lineWidth, lineHeight) +{ + const wordBreaks = ctx.breakText(text); + const styleBreaks = styles.map(style => style.start); + const segments = wordBreaks.concat(styleBreaks).sort((a, b) => a - b); + const textRuns = ctx.analyzeText(text, segments); + + let lineGlyphRuns = []; + let lineMeasuredWidth = 0; + let wordGlyphRuns = []; + let wordMeasuredWidth = 0; + let wordIndex = 0; + + for (const textRun of textRuns) { + // Check whether textRun belongs to the current word. + if (textRun.start > wordBreaks[wordIndex]) { + // Check whether the current word can fit in the current line. + if (lineMeasuredWidth + wordMeasuredWidth > lineWidth) { + fillJustifiedGlyphRunsWithStyles(ctx, lineGlyphRuns, x, y, styles, lineWidth); + + // Move to the next line. + y += lineHeight; + lineGlyphRuns = []; + lineMeasuredWidth = 0; + } + + // Add the last word to the current line. + lineGlyphRuns.concat(wordGlyphRuns); + lineMeasuredWidth += wordMeasuredWidth; + + // Move to the next word. + ++wordIndex; + wordGlyphRuns = []; + wordMeasuredWidth = 0; + } + + // Select the font in the context before shaping + const j = styles.findLast(style => style.start <= textRun.start); + ctx.font = styles[j].font; + + const glyphRun = ctx.shapeTextRun(textRun); + const metrics = glyphRun.textMetrics(); + + // Add the current GlypRun to the current word. + wordGlyphRuns[wordGlyphRuns.length] = glyphRun; + wordMeasuredWidth += metrics.width; + } + + // Last line should not be justified. + fillGlyphRunsWithStyles(ctx, lineGlyphRuns, x, y, styles); +} +``` + +This method can be called like this + +``` +const text = 'The red fox jumped' + + ' الثعلب الأحمر قفز فوق الكلب البني الكسلان' + + ' over the lazy brown dog.'; +const styles = [ + { charPos: 0, font: '32px Times', color: 'black' }, + { charPos: 4, font: 'bold 32px Times', color: 'red' }, + { charPos: 8, font: '32px Times', color: 'black' }, + { charPos: 27, font: 'bold 32px Times', color: 'red' }, + { charPos: 34, font: '32px Times', color: 'black' }, + { charPos: 48, font: 'bold 32px Times', color: 'brown' }, + { charPos: 54, font: 'bold 32px Times', color: 'blue' }, + { charPos: 58, font: '32px Times', color: 'black' }, + { charPos: 76, font: 'bold 32px Times', color: 'brown' }, + { charPos: 82, font: '32px Times', color: 'black' } +]; +fillWrappedJustifiedTextWithStyles(ctx, text, 0, 0, styles, 480, 40); +``` + +The result of using this function can be something like this screenshot +![Drawing Wrapped Justified Text with Styles](drawing-wrapped-justified-text-with-styled-display.png) +## Conclusion + +This proposal has the following advantages: + +1. It is efficient since it does not require processing the text cluster by cluster. A whole Latin line with the same style will be displayed as one `TextRun`. +2. It splits the text processing into separate operations. This separation allows custom layout and shaping. +3. The polyfills above are simple and do not depend on the system frameworks. They are more efficient if they are implemented in native code. +4. Advanced features such as line breaking can be added later. + diff --git a/CanvasTextShaping/drawing-colored-clusters-display.png b/CanvasTextShaping/drawing-colored-clusters-display.png new file mode 100644 index 0000000..744c5e8 Binary files /dev/null and b/CanvasTextShaping/drawing-colored-clusters-display.png differ diff --git a/CanvasTextShaping/drawing-colored-justified-words-display.png b/CanvasTextShaping/drawing-colored-justified-words-display.png new file mode 100644 index 0000000..6b83ca9 Binary files /dev/null and b/CanvasTextShaping/drawing-colored-justified-words-display.png differ diff --git a/CanvasTextShaping/drawing-text-with-styles-display.png b/CanvasTextShaping/drawing-text-with-styles-display.png new file mode 100644 index 0000000..e144792 Binary files /dev/null and b/CanvasTextShaping/drawing-text-with-styles-display.png differ diff --git a/CanvasTextShaping/drawing-text-with-styles.png b/CanvasTextShaping/drawing-text-with-styles.png new file mode 100644 index 0000000..bd5c2bc Binary files /dev/null and b/CanvasTextShaping/drawing-text-with-styles.png differ diff --git a/CanvasTextShaping/drawing-text.png b/CanvasTextShaping/drawing-text.png new file mode 100644 index 0000000..9f52b46 Binary files /dev/null and b/CanvasTextShaping/drawing-text.png differ diff --git a/CanvasTextShaping/drawing-wrapped-justified-text-with-styled-display.png b/CanvasTextShaping/drawing-wrapped-justified-text-with-styled-display.png new file mode 100644 index 0000000..da5ca36 Binary files /dev/null and b/CanvasTextShaping/drawing-wrapped-justified-text-with-styled-display.png differ