test_strptime failure on OpenBSD

[serhiy-storchaka]

BPO	31635
Nosy	@malemburg, @abalkin, @serhiy-storchaka, @applio, @pganssle, @tirkarthi

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-09-29.09:32:17.044>
labels = ['type-bug', 'library']
title = 'test_strptime failure on OpenBSD'
updated_at = <Date 2018-09-18.14:37:36.638>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2018-09-18.14:37:36.638>
actor = 'p-ganssle'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-09-29.09:32:17.044>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 31635
keywords = []
message_count = 7.0
messages = ['303319', '325496', '325497', '325514', '325545', '325640', '325653']
nosy_count = 6.0
nosy_names = ['lemburg', 'belopolsky', 'serhiy.storchaka', 'davin', 'p-ganssle', 'xtreak']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31635'
versions = []

[serhiy-storchaka]

$ ./python -m test -vuall test_strptime
...

======================================================================
FAIL: test_week_of_year_and_day_of_week_calculation (test.test_strptime.CalculationTests) [Jan 1 on Sunday] (date=(1905, 1, 1), format='%G %V %w')
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/serhiy/py/cpython3.7/Lib/test/test_strptime.py", line 508, in test_helper
    self.assertEqual(strp_output[:3], ymd_tuple, msg)
AssertionError: Tuples differ: (1905, 1, 8) != (1905, 1, 1)

First differing element 2:
8
1

• (1905, 1, 8)
  ? ^

+ (1905, 1, 1)
? ^
: '1904 53 0': 374 != 1

======================================================================
...
FAILED (failures=16, skipped=1)

[pganssle]

I think this is the relevant test: https://github.com/python/cpython/blob/3.7/Lib/test/test_strptime.py#L536

The issue seems to be in strftime rather than in strptime. According to Serhiy's paste, it seems that datetime.datetime.strftime("%G %V %w") on BSD results in "1904 53 0", whereas on Linux it results in "1904 52 0".

I suspect the confusion *may* be related to the fact that ISO weekdays go Monday (1) to Sunday (7), where as %w goes Sunday (0) to Saturday (6).

[pganssle]

This appears to be a duplicate of issue bpo-13414, which was resolved as fixed in BSD. @serhiy Are you using a very old version of BSD?

[serhiy-storchaka]

I used OpenBSD 6.1. I'll check on more modern versions when have chance.

[pganssle]

@serhiy Looking more closely at bpo-13414, I think there's a good chance that we've just discovered a different bug in OpenBSD's implementation of wcsftime (or strftime). Looks like OpenBSD 6.1 was released in 2017, which should have been years after the fix from bpo-13414 was released.

It seems that OpenBSD does not have a bug tracker per se, and I can't see if this bug has been reported. I can try to install OpenBSD in a VM later and try to reproduce it, but if it's a bug in OpenBSD, I see some possible options:

1. Add an exception to the test suite on OpenBSD (to be removed or pinned when it's fixed in OpenBSD)

2. If the bug is just in wcsftime and not in strftime, it may be fixed by switching to using strftime as is done on Windows: https://github.com/python/cpython/blob/master/Modules/timemodule.c#L673

I have never tested it, but it's possible this will impact performance, and if it's a bad enough hit it may not be worth sacrificing general performance to avoid a bug in a corner case.

[pganssle]

I do think this is an active bug in OpenBSD's ISO calculations, here: https://github.com/openbsd/src/blob/b66614995ab119f75167daaa7755b34001836821/lib/libc/time/wcsftime.c#L326

I re-created this algorithm in Python and at the beginning of the year it gives the wrong answer at the beginning of the year in about 50% of years. I have spent some time trying to understand the way the algorithm is *supposed* to work, but I don't totally understand what it's doing. I'll report this to OpenBSD.

[pganssle]

Bug reported to BSD here: https://marc.info/?l=openbsd-bugs&m=153728102618747&w=2

[encukou]

Other the upstream report, we can also document that strptime shares quirks with the underlying C function. That's now tracked in #57095.

I didn't check if this is fixed. Closing with a canned reply:

---

OpenBSD is unsupported per PEP-11.

I'll close the bug and PR.
Like other issues for unsupported platforms, it'll be listed on the GitHub project for anyone who wants to support the platform unofficially or bring support back.
I encourage such people to add themselves to the Experts list so we know whom to ping.