feat: 会话标签页显示实时 Token 输出速度 - #432
Conversation
|
感谢这个 PR,功能很实用,实现也比我预期的扎实 👏 我在本地把 本地验证(全绿)
读代码时确认没问题的几处这几个判断都很准,特意说一下,看得出是真的读过 connection store 的 reducer 才动手的:
1. 冷启动会从 0 慢慢爬上来,前 2–3 秒明显偏低这条是实际能看到的。 于是 seeding 之后紧接着那次 observe 通常是 0,而 稳态是准的,说明时间常数式 EWMA 的选型没问题 👍,就是短回合可能全程都在爬坡。加个 bias correction( 2. 日文假名和韩文谚文没算进 CJK
3. 每次通知都会全量重扫已累积文本
日常回合完全无感,只有很长的回合、尤其分屏多个 tab 同时在流的时候才会显出来,所以不当阻塞项。好在修起来很干净: 4. 静默间隙读数会冻住
5. 小 nit
再次感谢 🙏 上面这些都不影响合并,我先合了,后续再跟进。 |
Follow-up to the token-output-speed badge, closing the review notes on it. Bias-correct the smoother. Every wire envelope dispatches EVENT_APPLIED and notifies subscribers before the 16ms-batched content delta lands, so the first sample after a turn begins routinely carries no new tokens. Cold-starting the EWMA from that sample opened the badge at 0.0 and crawled toward the true rate over several seconds; tracking the accumulated weight and dividing it back out makes the reading the turn's cumulative average until TAU has elapsed, and an exponential window after. A 300ms warmup gate keeps the first reading off the screen until it covers a meaningful slice of wall clock. On a 60 tok/s stream the opening reading goes from 0.0 to 57.6. Measure incrementally. Re-scanning the whole accumulated turn on every notification was O(turn) per event on the synchronous dispatch path, and both the CJK match and the whitespace strip allocated. TokenCountAccumulator keeps each block's consumed length and measures only what was appended, and countChars is an allocation-free char-code scan. 73-315x less work over a turn, with results identical to the whole-string measure. Re-seat on a live message id change, since a snapshot hydration replaces the message rather than appending to it. Widen the dense-script class. Hiragana, katakana, Hangul and CJK punctuation (。、) all fell into the latin bucket, under-reporting Japanese and Korean output by more than 2x. Astral characters now score once via their high surrogate, which keeps the count exactly additive across a suffix split. Add a heartbeat so the rate decays through a silent gap instead of freezing: the store only notifies on wire activity, and a quiet tool, a retry backoff or a blocking permission prompt would otherwise pin the last reading. Also: tabular-nums so the digits stop shimmering, the accessible name moved off the bare wrapper span onto the icon to match ComposerContextUsage, and drop the screenshot that nothing in the repo referenced.
|
上面那些后续项我直接在这个分支上补了一个 commit(695d53c3),你原来的提交没动 🙂 如果哪里改得不合你意,尽管推翻或者让我调整。 对应关系: 1️⃣ 冷启动爬坡 — 给 EWMA 加了偏差校正(跟踪累计权重再除回去,就是 Adam 那套),并加了 300ms 的预热闸门。这样在 τ 之前读数是这一回合的累计均值,之后才退化成指数窗口。按真实的 2️⃣ 假名 / 谚文 — 字符类改写成 charCode 区间判断,补上平假名、片假名、谚文,以及之前也漏掉的 3️⃣ 全量重扫 — 新增
(整个回合累计的 CPU;和整串测量的数值误差是 0.0,不是近似。) 4️⃣ 静默间隙冻住 — 加了 500ms 心跳,回合开始时启动、回合结束/卸载时停掉,让注释里写的那个衰减真正成立。 5️⃣ nit — 额外补了一个: 测试从 16 个加到 36 个,每个新增用例都做了变异验证(去掉心跳只挂衰减那条、去掉偏差校正只挂冷启动那几条、去掉重新播种只挂 hydration 那条)。这个分支上 |
|
赞,大佬辛苦了 |
功能
每个会话标签页在生成过程中,于 composer 下方状态栏、上下文窗口百分比指示器之前,显示实时 Token 输出速度(如
12.4 tok/s);生成结束自动隐藏。实现方式
text+thinking内容(只统计根 agent,排除子代理输出)ComposerContextUsage相同模式);无后端改动,桌面 / Web / 服务器 / 移动端共用同一套代码改动文件
src/lib/token-speed.ts— 字符换算 + 平滑 tracker(纯函数,已单测)src/components/chat/token-output-speed.tsx— 徽标组件src/components/chat/message-input.tsx— 在ComposerContextUsage前渲染src/lib/token-speed.test.ts、src/components/chat/token-output-speed.test.tsx验证
tsc --noEmit通过vitest run全量通过:281 个文件 / 3632 个测试