Skip to content

GH-50707: [C++][Gandiva] fix out-of-bounds read in mask utf8proc length - #50709

Merged
raulcd merged 2 commits into
apache:mainfrom
Arawoof06:mask-utf8proc-iterate-length
Aug 14, 2026
Merged

GH-50707: [C++][Gandiva] fix out-of-bounds read in mask utf8proc length#50709
raulcd merged 2 commits into
apache:mainfrom
Arawoof06:mask-utf8proc-iterate-length

Conversation

@Arawoof06

@Arawoof06 Arawoof06 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

The Gandiva mask stubs walk a utf8 string calling utf8proc_iterate(data + bytes_read, data_len, ...), passing the total data_len as the remaining byte count even though the pointer has already advanced by bytes_read. utf8proc_iterate reads up to strlen bytes, so a value ending in a truncated multi-byte glyph (for example a lead byte 0xF0 as the final byte of an exactly sized value buffer) makes it read continuation bytes past data + data_len. This is reachable from mask(), mask_first_n() and mask_last_n() on untrusted string columns. gdv_mask_last_n_utf8_int32 happens to be shielded by its utf8proc_decompose pre-pass, but gdv_mask_first_n_utf8_int32 and mask_utf8_utf8_utf8_utf8 have no such guard and over-read.

What changes are included in this PR?

Pass data_len - bytes_read (the real remaining length) at all four utf8proc_iterate call sites so utf8proc reports the truncated glyph instead of reading past the buffer. mask_utf8_utf8_utf8_utf8 also lacked the char_len < 0 check that the other two paths already have, so I added it there to reject the now-detected invalid input rather than advancing by a negative length.

Are these changes tested?

Yes. TestMaskTruncatedUtf8NoOverread feeds a value whose reported length stops one byte short of a complete euro sign; before the change both functions consumed the out-of-range byte and returned a masked result, after it they report the truncated input. The existing mask tests still pass.

Are there any user-facing changes?

No change for valid utf8. A value that ends in a truncated multi-byte glyph now surfaces an invalid-utf8 error instead of being silently masked using bytes past its end.

This PR contains a "Critical Fix". The wrong length argument lets utf8proc_iterate read past the end of an exactly sized input buffer.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #50707 has been automatically assigned in GitHub to PR creator.

@raulcd raulcd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you rebase, please? The CI failures are fixed on main and should be fixed on your PR.

@dmitry-chirkov-dremio @lriggs @akravchukdremio @xxlaykxx could you take a look at this PR, please?

@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Jul 31, 2026
EXPECT_EQ(expected, std::string(result, out_len));
}

TEST(TestGdvFnStubs, TestMaskTruncatedUtf8NoOverread) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth having a simple test for gdv_mask_last_n_utf8_int32 as well

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added one. last_n's utf8proc_decompose pre-pass rejects the truncated glyph before the iterate loop, so the test just confirms it errors on that input instead of over-reading.

@Arawoof06
Arawoof06 force-pushed the mask-utf8proc-iterate-length branch from 2e9ab83 to 1869ef3 Compare August 11, 2026 08:32
@Arawoof06

Copy link
Copy Markdown
Contributor Author

Rebased on main to pick up the CI fixes, and added a test covering gdv_mask_last_n_utf8_int32 on the truncated input per @lriggs.

@github-actions github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 11, 2026
@lriggs

lriggs commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Looks good

Comment thread cpp/src/gandiva/gdv_function_stubs.cc
@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 13, 2026
…f8_int32

Signed-off-by: abdul rawoof <abdulr@bugqore.com>
@github-actions github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 13, 2026
@raulcd
raulcd merged commit e611f48 into apache:main Aug 14, 2026
56 of 57 checks passed
@raulcd raulcd removed the awaiting change review Awaiting change review label Aug 14, 2026
@github-actions github-actions Bot added the awaiting merge Awaiting merge label Aug 14, 2026
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit e611f48.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 11 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants