Skip to content

In csv.py - guess_delimiter: add delimiters to list of eligible delimiters #111820

Description

@paulbakkerbloom

Feature or enhancement

Proposal:

In _guess_delimiter in csv.py append the delimiters argument to the list of eligible delimiters (now called ascii)

In csv.py - the _guess_delimiter method only looks for the first 128 ASCII characters as possible delimiter characters. However, the method also allows the caller to supply a delimiters argument - to say which delimiters take precedence if multiple candidates are found.

I would like to add this delimiters list to the list of eligible delimiters that the code considers when determining (guessing) the delimiter, on top of the standard 128 ASCII ones.

My particular use case is that I have multiple files I need to determine the delimiter from, and some have "¬" (yes, I know...) - which isn't included in the first 128 ASCII characters.

    def _guess_delimiter(self, data, delimiters):
    
    #....#

        eligible_delimiters = [chr(c) for c in range(127)] # 7-bit ASCII
        if delimiters:
            eligible_delimiters = list(set(eligible_delimiters + delimiters))

        # build frequency tables
        chunkLength = min(10, len(data))
        iteration = 0
        charFrequency = {}
        modes = {}
        delims = {}
        start, end = 0, chunkLength
        while start < len(data):
            iteration += 1
            for line in data[start:end]:
                for char in eligible_delimiters:

instead of

        ascii = [chr(c) for c in range(127)] # 7-bit ASCII

        # build frequency tables
        chunkLength = min(10, len(data))
        iteration = 0
        charFrequency = {}
        modes = {}
        delims = {}
        start, end = 0, chunkLength
        while start < len(data):
            iteration += 1
            for line in data[start:end]:
                for char in ascii:

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions