gh-109638: Fix for significant backtracking in csv.Sniffer

[sg3-141-592]

#109638 it is possible to get significant backtracking in csv.Sniffer() inside the doublequote checking regex. This change introduces a zero-length lookahead assertion to reduce the amount of backtracking.

This yields a significant improvement in testing

import csv
import time

for NUM_ITERATIONS in range(10,70,10):

    test_str = '"",'*NUM_ITERATIONS + '"'*NUM_ITERATIONS + '0' + '"'*NUM_ITERATIONS + '0'

    t0 = time.time()

    dialect = csv.Sniffer().sniff(test_str)

    t1 = time.time()
    print(f"{NUM_ITERATIONS}, {t1-t0}")

NUM_ITERATIONS | Before           | After Regex Fix
10  | 0.002374649 | 0.000935555
20  | 0.051596165 | 0.003250599
30  | 0.371201515 | 0.017635345
40  | 1.477169752 | 0.054927588
50  | 4.845417738 | 0.120140076
60  | 11.52993703 | 0.252950668

[AA-Turner]

Please may you explain this comment?

[sg3-141-592]

Sure, this zero-width lookahead assertion change in the Regex can be done with an atomic group which is cleaner and more concise.

# Current change
(,|^)\W*"(?=(?P<zero>[^,|"\n]*))(?P=zero)"[^,|\n]*"\W*(,|$)
# Atomic Group
(,|^)\W*"(?>[^,|"\n]*)"[^,|\n]*"\W*(,|$)

But atomic groups are only supported in Python 3.11 onwards so I avoided using them here.

[AA-Turner]

Thanks Sean. This PR (if merged) would be part of Python 3.13, so let's use the better atomic group method.

A

[sg3-141-592]

Sure, I've switched us to the simpler atomic group setup. Performance is identical to the previous fix.

[aterrel]

This looks like a good quick fix for the problem.

Ultimately though, these regexs are hard to read and cause a few problems with lists and other items. I think we should be thinking about how to replace the sniffer to have a higher accuracy. (See https://github.com/ws-garcia/CSVsniffer which shows that only 67.54% accuracy). I've posted to dpo on this topic here: https://discuss.python.org/t/rewrite-csv-sniffer

[github-actions]

This PR is stale because it has been open for 30 days with no activity.