Feature request: sniff csv quoting and lineterminator

[Nikilauz]

If you read a csv file, alter the data and then write it back, it would be very nice to recreate also the quoting behaviour. So e.g. if the file is quoted in style csv.QUOTE_NONNUMERIC, it would be very useful to have the quoting property set correctly in the dialect object returned by csv.Sniffer().sniff.
Also, I just figured out that the line terminator is also hard-coded to "\r\n". Would also be nice to be adaptive...

Linked PRs

• gh-129374: adds quoting detection to csv sniffer #134416

[ericvsmith]

This probably belongs on discuss.python.org. But in any event, you'll need to specify more information. What specific changes would you like to see?

[Nikilauz]

In general, I would expect the Dialect object returned by csv.Sniffer.sniff to have its properties set in a way that a writer object initialized with the dialect would reproduce a similar csv string. This seems to be a clear correctness constraint for the dialect functionality for me, so is this really something to put in a discussion forum? I'm right aware that this may be an issue with a questionable benefit-cost-ratio, though...
Anyhow, I'm not familiar with the feedback/issue conventions of the python project and may have misunderstood the purpose of dialects, so feel free to correct me (:

[ericvsmith]

Could you provide sample code that shows what results you are getting, and explain how that differs from what expect to get?

[Nikilauz]

Sure:

import csv

s = '42,"hello","world",-1\n'    # e.g. via readline method from some csv file
d = csv.Sniffer().sniff(s)
d.quoting    # 0
d.lineterminator    # '\r\n'

x = list(csv.reader([s], d))    # [['42', 'hello', 'world', '-1']]

f = open("test.csv", "w", newline = "")
csv.writer(f, d).writerows(x)
f.close()

f = open("test.csv", "r+b")
f.read()    # b'42,hello,world,-1\r\n'
f.close()

So instead of b'42,hello,world,-1\r\n' I would expect to get b'42,"hello","world",-1\n'.
More precisely, I would expect d.quoting to be 2 (i.e. csv.QUOTE_NONNUMERIC) and d.lineterminator to be '\n'.

Thanks for your time!

[aterrel]

What would you expect if there are a mix of string quoted and not quoted?

[Nikilauz]

Well then it's none of the possible values of Dialect.quoting and may be some fallback default quoting style I guess, i.e. 0 or so...

[aterrel]

Okay here is a test I'll build towards. Let me know if that correct.

8a38c20#diff-dd5e5efc1566b1c1dc988769a04cc403d2416c8af5bf4785bec235d02a780813R1345

[Nikilauz]

Looks good, depending on the assertEquals and getvalue methods there could be some issue with comparing \n and \r\n. With actual files that was a problem, thats why I used bytestrings...

[aterrel]

@Nikilauz I've added the PR for the feature. Would be great if you could pull down the dev version of Python and test it out a bit. Unfortunately there are some gotchas but this seems to be the beginning of a solution.