Skip to content

GH-45366: [C++][Parquet] Set is_compressed to false when data page v2 is not compressed#45367

Merged
mapleFU merged 9 commits into
apache:mainfrom
mapleFU:GH-45366
Mar 5, 2025
Merged

GH-45366: [C++][Parquet] Set is_compressed to false when data page v2 is not compressed#45367
mapleFU merged 9 commits into
apache:mainfrom
mapleFU:GH-45366

Conversation

@mapleFU

@mapleFU mapleFU commented Jan 28, 2025

Copy link
Copy Markdown
Member

Rationale for this change

Currently, if data page v2 is enabled, is_compressed is always set if Page has compression, however, this can be eliminated if:

  1. Page is empty
  2. Compression makes page event larger.

What changes are included in this PR?

Enhancement is_compressed setting in column_writer.cc

Are these changes tested?

Yes

Are there any user-facing changes?

No

@mapleFU mapleFU requested a review from wgtmac as a code owner January 28, 2025 15:00
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #45366 has been automatically assigned in GitHub to PR creator.

Comment thread cpp/src/parquet/column_writer.cc
@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Feb 7, 2025
@github-actions github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Feb 7, 2025

@raulcd raulcd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a neat improvement. Thanks @mapleFU ! The last change fixed the segmentation fault on Python. I am ok with this PR but will let other experts review / approve. I am just wondering if we don't add specific tests to validate this new behavior

@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Feb 7, 2025

@wgtmac wgtmac left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wgtmac wgtmac changed the title GH-45366: [C++][Parquet] Allowing is_compressed not set if DataPageV2 enabled GH-45366: [C++][Parquet] Set is_compressed to false when data page v2 is not compressed Feb 7, 2025

@pitrou pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on the principle, do you want to add a small test for it?

Comment thread cpp/src/parquet/column_writer.cc Outdated
@github-actions github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Mar 4, 2025
@mapleFU mapleFU requested a review from pitrou March 5, 2025 08:48

@pitrou pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just two minor suggestions

Comment thread cpp/src/parquet/column_writer_test.cc Outdated
Comment thread cpp/src/parquet/column_writer_test.cc Outdated
@mapleFU

mapleFU commented Mar 5, 2025

Copy link
Copy Markdown
Member Author

Thanks! Would merge after all ci passed

@mapleFU mapleFU merged commit c58ec9c into apache:main Mar 5, 2025
@mapleFU mapleFU removed the awaiting change review Awaiting change review label Mar 5, 2025
@mapleFU mapleFU deleted the GH-45366 branch March 5, 2025 10:12
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit c58ec9c.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants