Read source and config files as UTF-8 regardless of locale - #479
Open
sinsoku wants to merge 1 commit into
Open
Conversation
`File.read` without an encoding follows `Encoding.default_external`, which is US-ASCII when `LANG` is not set (e.g., ruby-lsp started from a GUI editor). With a `.rbs` file containing non-ASCII characters, `RBS::Parser` then raises `ArgumentError` (invalid byte sequence in US-ASCII). This is not a `RBS::ParsingError` and is not rescued, so one such file aborts `add_workspace` and `batch` entirely. `typeprof.conf.jsonc` has the same problem. Ruby files are mostly fine because Prism ignores the encoding of the source string. They still fail when `Encoding.default_internal` is set (e.g., `ruby -U`), since `File.read` then transcodes from US-ASCII. So this reads them as UTF-8 too. The rbs gem has read signature files as UTF-8 for the same reason since ruby/rbs#493.
sinsoku
added a commit
to sinsoku/ruby-lsp-typeprof
that referenced
this pull request
Sep 9, 2026
TypeProf reads source and config files with `File.read`, which follows `Encoding.default_external`. When `LANG` is not set (e.g. ruby-lsp launched from a GUI editor), that is US-ASCII and a non-ASCII `.rbs` file raises `ArgumentError` in `add_workspace`. `activate` rescued it, so code lens and document symbol silently disappeared. Changing `Encoding.default_external` is process-wide, but ruby-lsp runs add-on hooks sequentially on a single worker thread, so no other add-on runs while it is temporarily switched. TypeProf will read files as UTF-8 itself once ruby/typeprof#479 is released, at which point this workaround can be dropped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
File.readwithout an encoding followsEncoding.default_external, which is US-ASCII whenLANGis not set (e.g., ruby-lsp started from a GUI editor). With a.rbsfile containing non-ASCII characters,RBS::Parserthen raisesArgumentError(invalid byte sequence in US-ASCII). This is not aRBS::ParsingErrorand is not rescued, so one such file abortsadd_workspaceandbatchentirely.typeprof.conf.jsonchas the same problem.Ruby files are mostly fine because Prism ignores the encoding of the source string. They still fail when
Encoding.default_internalis set (e.g.,ruby -U), sinceFile.readthen transcodes from US-ASCII. So this reads them as UTF-8 too.The rbs gem has read signature files as UTF-8 for the same reason since ruby/rbs#493.
Verification
env -u LANG bundle exec rake testpasses. Before the fix, the new tests fail withArgumentError: invalid byte sequence in US-ASCII.🤖 Generated with Claude Code