Add ANSI_QUOTES and SQL mode validation#452
Open
JanJakes wants to merge 5 commits into
Open
Conversation
Contributor
🤖 Lexer benchmarkChanges to lexer-related files were detected and triggered a benchmark:
Note: Hosted runners are noisy, and absolute numbers vary. Treat the results with caution and verify them locally. To reproduce locally: |
JanJakes
changed the title
JanJakes
force-pushed
the
ansi-quotes
branch
11 times, most recently
from
July 17, 2026 07:50
b8d6367 to
3f11622
Compare
Map supported SQL mode names to their native MySQL bit values and use the bitmap as the driver's internal representation. Serialize active modes by bit position so duplicates collapse and @@sql_mode follows MySQL's canonical order.
Validate SQL mode names and numeric masks before changing the session state. Report invalid values with MySQL's SQLSTATE 42000 and error 1231, while retaining MySQL's handling for empty list components and incorrect value types.
When ANSI_QUOTES is active, MySQL treats a double-quoted sequence as a quoted identifier rather than a string literal. Emulate this in the lexer by emitting a backtick-quoted identifier token for double-quoted text when the mode is set, mirroring how NO_BACKSLASH_ESCAPES already alters tokenization. The driver already forwards its active SQL modes to the lexer, so no further wiring is needed. See: https://dev.mysql.com/doc/refman/8.4/en/sql-mode.html#sqlmode_ansi_quotes
MySQL's composite ANSI mode is shorthand for a set of component modes. Expand it when sql_mode is set and store the resulting list, so that "@@sql_mode" and individual mode checks reflect the components: REAL_AS_FLOAT, PIPES_AS_CONCAT, ANSI_QUOTES, IGNORE_SPACE, ONLY_FULL_GROUP_BY REAL_AS_FLOAT and ONLY_FULL_GROUP_BY are stored but not yet respected by the emulation. The driver is the source of truth for SQL modes, but the lexer also recognizes the composite ANSI mode directly so it stays correct when used standalone, applying the components that affect tokenization: PIPES_AS_CONCAT, IGNORE_SPACE, and ANSI_QUOTES. See: https://dev.mysql.com/doc/refman/8.4/en/sql-mode.html#sqlmode_ansi
MySQL does not process backslash escape sequences inside quoted identifiers; the bounding quote is escaped only by doubling it. The lexer was applying string-literal backslash escaping to backtick identifiers (and, with the new ANSI_QUOTES support, to double-quoted identifiers), both when scanning for the closing quote and when unquoting the value. As a result, an identifier like `a\nb` resolved to "a<newline>b", and `a\` (a trailing backslash) failed to tokenize because the backslash was treated as escaping the closing quote. Restrict backslash escaping to string literals so identifiers preserve backslashes verbatim. See: https://dev.mysql.com/doc/refman/8.4/en/identifiers.html
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves SQL mode handling and adds
ANSI_QUOTESandANSIIsupport. This includes:ANSI_QUOTESis supported by the driver, standalone parser, and native lexer.ANSISQL mode is expanded into its component modes.Why
The
ANSI_QUOTESSQL mode support keeps reappearing in our recent performance optimization and parser experiments. This PR extracts and completes that functionality.Additionally, the driver stored arbitrary mode names, accepted invalid assignments, always treated double quotes as string delimiters, did not expand did not expand composite SQL modes, and applied string escaping to quoted identifiers. The standalone parser and native lexer lacked equivalent quoted-identifier handling.