Skip to content

[3.7] bpo-39389: gzip: fix compression level metadata (GH-18077)#18101

Merged
ned-deily merged 2 commits into
python:3.7from
miss-islington:backport-eab3b3f-3.7
Mar 4, 2020
Merged

[3.7] bpo-39389: gzip: fix compression level metadata (GH-18077)#18101
ned-deily merged 2 commits into
python:3.7from
miss-islington:backport-eab3b3f-3.7

Conversation

@miss-islington

@miss-islington miss-islington commented Jan 21, 2020

Copy link
Copy Markdown
Contributor

As described in RFC 1952, section 2.3.1, the XFL (eXtra FLags) byte of a
gzip member header should indicate whether the DEFLATE algorithm was
tuned for speed or compression ratio. Prior to this patch, archives
emitted by the gzip module always indicated maximum compression.
(cherry picked from commit eab3b3f)

Co-authored-by: William Chargin wchargin@gmail.com

https://bugs.python.org/issue39389

As described in RFC 1952, section 2.3.1, the XFL (eXtra FLags) byte of a
gzip member header should indicate whether the DEFLATE algorithm was
tuned for speed or compression ratio. Prior to this patch, archives
emitted by the `gzip` module always indicated maximum compression.
(cherry picked from commit eab3b3f)

Co-authored-by: William Chargin <wchargin@gmail.com>
@miss-islington

Copy link
Copy Markdown
Contributor Author

@wchargin and @serhiy-storchaka: Status check is done, and it's a failure ❌ .

1 similar comment
@miss-islington

Copy link
Copy Markdown
Contributor Author

@wchargin and @serhiy-storchaka: Status check is done, and it's a failure ❌ .

@ned-deily

Copy link
Copy Markdown
Member

@serhiy-storchaka The CI test run failed but, before restarting it by creating a new backport, should we really be backporting this to 3.7 at this stage? Is there a real problem that 3.7 users are encountering or do we risk introducing a needless change in behavior?

@wchargin

Copy link
Copy Markdown
Contributor

It’s a real problem that I’m encountering (hence the patch). I can work
around it by poking the appropriate bytes in the output file. It would
be nice not to have to do that, but it’s also not the end of the world.
Up to you.

@wchargin

Copy link
Copy Markdown
Contributor

(The CI failures look legit; the constants used were added in #9833,
which first landed in 3.8. We could either backport those constants or
just inline their values.)

@ned-deily

Copy link
Copy Markdown
Member

Ping

@wchargin

wchargin commented Mar 4, 2020

Copy link
Copy Markdown
Contributor

@ned-deily: We should apply the following patch to the backport:

From 2f951ad470973c7576dda0659d647cece4d77b50 Mon Sep 17 00:00:00 2001
From: William Chargin <wchargin@gmail.com>
Date: Tue, 3 Mar 2020 19:08:37 -0800
Subject: [PATCH] [bpo-39389](https://bugs.python.org/issue39389): add constants for 3.7 backport

Backport: <https://github.com/python/cpython/pull/18101>
---
 Lib/gzip.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Lib/gzip.py b/Lib/gzip.py
index 0b039d569d..e59b454814 100644
--- a/Lib/gzip.py
+++ b/Lib/gzip.py
@@ -17,6 +17,11 @@ FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT = 1, 2, 4, 8, 16
 
 READ, WRITE = 1, 2
 
+_COMPRESS_LEVEL_FAST = 1
+_COMPRESS_LEVEL_TRADEOFF = 6
+_COMPRESS_LEVEL_BEST = 9
+
+
 def open(filename, mode="rb", compresslevel=9,
          encoding=None, errors=None, newline=None):
     """Open a gzip-compressed file in binary or text mode.
-- 
2.23.0.rc1.1.g9eb78f1760

I can repro the test failure before this patch, and it passes after this
patch for me. (The patch cherry-picks part of #9833.)

But I don’t have write access to miss-islington. Do you need anything
from me to apply this patch?

@ned-deily ned-deily merged commit 12c45ef into python:3.7 Mar 4, 2020
@miss-islington miss-islington deleted the backport-eab3b3f-3.7 branch March 4, 2020 07:06
@wchargin

wchargin commented Mar 4, 2020

Copy link
Copy Markdown
Contributor

Awesome; thank you! :-)

webknjaz added a commit to cherrypy/cherrypy that referenced this pull request Nov 15, 2020
This change implements a fix to compression in gzip tool that CPython
applied per https://bugs.python.org/issue39389. It uses proper XFL
value in gzip header per https://tools.ietf.org/html/rfc1952, section
2.3.1.

Refs:
* python/cpython#18077 (CPython 3.9.0+)
* python/cpython#18100 (CPython 3.8.2+)
* python/cpython#18101 (CPython 3.7.7+)

Fixes #1849
webknjaz added a commit to cherrypy/cherrypy that referenced this pull request Nov 15, 2020
This change implements a fix to compression in gzip tool that CPython
applied per https://bugs.python.org/issue39389. It uses proper XFL
value in gzip header per https://tools.ietf.org/html/rfc1952, section
2.3.1.

Refs:
* python/cpython#18077 (CPython 3.9.0+)
* python/cpython#18100 (CPython 3.8.2+)
* python/cpython#18101 (CPython 3.7.7+)

Fixes #1849

(cherry picked from commit 3925ae9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants