Conversation
Parallelise the file-processing passes over a modpack archive and remove redundant CurseForge API round trips. Measured on a 481 MB CurseForge modpack (5574 override entries, 241 manifest mods, 28 threads) using the launcher's own instrumentation, comparing against the unmodified code on the same machine: overrides hash scan 2425 ms -> 687-726 ms overrides extraction 7925 ms -> 3336-4692 ms install stage 7215 ms -> 3337-4697 ms curse.completion 10627 ms -> 1432-3318 ms The whole install window went from 43.0 s to 21.1 s when assets and libraries are already present; the cold run also takes 43.0 s, but now covers a full asset and library download that the original measurement did not pay for. What changed: - Unzipper extracts an archive's entries across several readers, one per worker, because a ZipArchiveReader seeks through a single channel and cannot serve two threads. Entry counts below a threshold stay on the calling thread. - MinecraftInstanceTask hashes the overrides in parallel the same way. - CurseCompletionTask resolves addon class ids and file names in batches instead of one request per manifest entry. - ZipFileTree caches its flat entry lookup, which the parallel passes share. - PerfLog is the opt-in (hmcl.perf) instrumentation behind these numbers. Correctness: - Parallel extraction is verified against the sequential path to produce byte-identical output (5574 files, 589529257 bytes). - The charset the parallel readers decode entry names with is resolved by CompressingUtils.resolveZipEncoding, which reproduces openZipFileWithPossibleEncoding exactly, including its UTF-8 fallback branch, so a parallel extraction writes the same file names as before. Both were compared over 42 archive/charset combinations. - Unit tests cover the touched parsing paths.
Reuse the outputs of a previously built loader when the same loader version is installed again. A loader installer only declares outputs for a few of its processors, so a launcher that trusts the profile cannot tell whether the remaining steps still need to run and repeats all of them on every install, even when their outputs are already sitting in the shared libraries directory. Record the SHA-1 of every file a processor writes, along with the installer profile and the loader manifest, so a repeat installation skips both the installer download and every processor. A record is discarded as soon as any output is missing or its checksum no longer matches. Move the CurseForge manifest lookup off the end of the installation and next to the downloads. Resolving the file names and the addon class ids only depends on the manifest, and both are latency bound, so they no longer sit on the critical path behind a saturated download. Both lookups are also batched: one POST /v1/mods/files and one POST /v1/mods answer for the whole manifest instead of one request per entry, which matters because the request budget is shared with every other CurseForge call. Share decoded mod icons across list rebuilds, keyed by path, size and modification time, so reloading the mod list no longer reopens every jar to decode an icon it had already decoded. Remove the PerfLog instrumentation that was added to measure the changes above. It is not part of the upstream code and would not have a place there.
Contributor
|
内存占用有测试吗？感觉HMCL现在不缺加载的时间，需要优化的是下载和内存为主 |
Author
内存占用没有测试 这个把整合包安装时间缩短了一半左右(冷安装和温安装都是) modlist加载是顺手加速的,能增加用户体验 |
Member
|
这是 GPT 写的吗？看起来有大量过度抽象和无意义修改，会增加很多维护成本。 请你检查其中有意义的部分，把有意义的优化逐个分别作为单独的 PR 提交，而不是用 AI 一次性弄出这么一大坨出来，看的很头疼。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
perf: 加速整合包安装、加载器复用与模组列表刷新
分支：perf/modpack-install-optimizations
这个 PR 做什么
四块改动，都围绕最大复用：
实测数据
测试环境：1.20.1 + Forge 47.3.22，同一个 CurseForge 整合包 (落幕曲1.5.3)（481 MB，5574 个 overrides 条目，241 个 manifest 条目，268 个模组），同一台机器。与未改动的代码在同一台机器、相同负载下、同一个整合包上对比。
整合包安装
CurseForge 元数据提前
241 个条目的名称解析合并为 1 次
POST /v1/mods/files，class id 合并为 1 次POST /v1/mods。加载器产物复用
同一个整合包连续安装两次：
Executing external processor条数forge-installer出现安装 Forge阶段「改动前」列指没有构建记录时每一次安装都会发生的事——也就是本 PR 之前的行为。第二次安装全程没有下载安装器，也没有启动任何 processor 进程。
模组列表刷新（268 个模组）
上游每次刷新都重新解析全部 268 个 jar；本 PR 按
(size, lastModified)缓存，文件未改动时每个文件只做一次stat。首次刷新两者区间重叠，不构成明确提升——首次刷新的收益来自并行解析，而上游那一组数据自身波动就有 3 倍，覆盖了这个量级。列表界面本身在刷新时仍会整体重建（
setAll），但图标改为按(路径, 大小, 修改时间)缓存在页面上，重建后的条目直接复用上一轮已解码的图，不再重新打开 jar。实测这一项占原刷新开销的绝大部分（29 个可见图标、589.6 ms）。实现摘要
解压与哈希并行化。
Unzipper把一个归档的条目分散到多个 reader 上，每个 worker 一个，因为ZipArchiveReader通过单一 channel 寻址，无法同时服务两个线程。条目数低于阈值时留在调用线程。MinecraftInstanceTask以同样方式并行哈希 overrides。ZipFileTree缓存扁平条目索引供这些并行流程共用。CurseForge 批量请求。
POST /v1/mods与POST /v1/mods/files一次可答多个 id，而整合包 manifest 恰好逐条指名了它钉住的文件。逐条请求时即使服务端很快也不划算，因为信号量是与所有 CurseForge 调用共用的，几百个条目大部分时间在排队等许可而不是在跟服务端交换。元数据提前。 名称解析与 class id 查询只依赖 manifest，不碰实例里的任何文件，因此排在游戏下载与解压旁边，而不是排在它们之后。这两项是延迟受限而非带宽受限，与一个已经打满带宽的下载共用链路不额外花费时间；反之放在下载之后，每一次往返都落在关键路径上。
加载器产物复用。 加载器安装器只为少数 processor 声明
outputs校验和。Forge 1.20.1-47.3.22 中jarsplitter带outputs，而installertools、ForgeAutoRenamingTool、binarypatcher一个都不带。只信 profile 的启动器无法判断这些步骤是否还需要执行，于是每次安装都重跑一遍——哪怕上一次的产物还躺在共享的 libraries 目录里。本 PR 补上缺失的部分：记录每个 processor 刚写出的文件的校验和，同时保存安装器 profile 和加载器 manifest 原文，重复安装因此连安装器 JAR 都不必下载和解包。构建键为
forge|<版本>|client/neoforge|<版本>|client。加载器版本串已经同时确定了 MC 版本和加载器，且在拉取安装器之前就已知:这正是让重复安装能省掉下载的原因。NeoForge 必须自己指定键：它报给 patch 的版本来自安装器 profile，而「要不要下载安装器」这个判断必须发生在读到 profile 之前。模组列表缓存。 解析只读文件、不碰管理器状态，因此可以并发且不持锁地跑，结果之后再单线程合并。reader 划分只取决于扩展名和实例的加载器集合，每次刷新算一次而不是每个文件算一次。非模组文件在解析前就被过滤掉，解析阶段因此不必对它们做
stat。正确性
CompressingUtils.resolveZipEncoding精确复现openZipFileWithPossibleEncoding，包括其 UTF-8 回退分支，在 42 组归档/字符集组合上验证过。checked=6 mismatched=0）。任一产物缺失、校验和不符、format版本不符或键不匹配，均判定为无效记录并退回完整构建。.hmcl/loader-build/下，路径按仓库根目录相对存储，与读取时的工作目录无关。CurseCompletionTask在拿不到提前解析结果时（修复入口）仍会自行查询，行为与改动前一致。未验证
NeoForge 没有端到端实测——测试环境里只有 Forge/CurseForge 的整合包，没有 NeoForge 整合包。