Skip to content

[BUGFIX] Reject data sets listing a column twice - #755

Merged
sbuerk merged 1 commit into
9from
dataset-import-duplicate-columns-9
Sep 16, 2026
Merged

sbuerk merged 1 commit into
9from
dataset-import-duplicate-columns-9

Conversation

@sbuerk

@sbuerk sbuerk commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Backport of #754 to branch 9.

Clean cherry-pick of the main commit, no adaptations needed. The check lives
in DataSet::parseData() and does not depend on the bulk insert of #752, so
this branch is based on 9 directly and not stacked anymore. It is also
cherry-picked onto the combined verification branch (#753).

Note for branch 9: third-party test suites with such a broken fixture fail
with the exception after updating. Previously they imported silently, but with
#752 they fail anyway on MySQL, MariaDB and PostgreSQL, just with a less helpful
SQL error.

Quality gates

Run locally with Build/Scripts/runTests.sh, PHP 8.2 (composer resolved
typo3/cms-core 14.3.x-dev):

  • -s composerUpdate: passed
  • -s cgl -n: passed
  • -s lint: passed
  • -s phpstan: passed
  • -s unit: passed

Original description

A CSV data set listing a column twice in the header row of a table section has
been accepted silently so far: header and values are combined into one element
per row, so the duplicate collapses into a single key with the last value
winning, hiding the mistake in the fixture.

Since #744 data sets are imported with a single bulk insert per table and the
column list is taken from the header row directly, so the duplicate ends up
twice in the INSERT statement. MySQL, MariaDB and PostgreSQL reject that with
an error pointing neither to the data set nor to the column (SQLite accepts it):

SQLSTATE[42000]: Syntax error or access violation: 1110 Column configuration specified twice
SQLSTATE[42701]: Duplicate column: 7 ERROR:  column "configuration" specified more than once

Change

DataSet::read() now throws a dedicated DuplicateDataSetColumnException
naming the file, the table and the duplicated column, for imports and
assertions alike:

DataSet "…/sys_file_storage.csv" lists column "configuration" more than once in
the header row of table "sys_file_storage". Remove the duplicate column, only
one value per column can be imported or asserted.

Silently deduplicating the column list was considered and dropped: it would keep
broken fixtures working and import whichever value comes last, without anyone
noticing.

Found by / impact on TYPO3 Core

The TYPO3 v14.3 Core CI run against the combined backport branch (#753), see
https://review.typo3.org/c/Packages/TYPO3.CMS/+/95869 and
pipeline 108665. Two
sys_file_storage.csv fixtures of the v14.3 form extension list configuration
twice. They are fixed with https://review.typo3.org/c/Packages/TYPO3.CMS/+/95875
(forge #110722), which needs to be
merged before a release containing this change is required by Core 14.3
.

All CSV files of Core main, 14.3 and 13.4 were scanned with the header
parsing rules of DataSet::parseData(): those two fixtures are the only ones
listing a column twice.

Quality gates

Run locally with Build/Scripts/runTests.sh, PHP 8.2:

  • -s cgl -n: passed
  • -s lint: passed
  • -s phpstan: passed
  • -s unit: passed, the new test failed before the change

A CSV data set listing a column twice in the header row of a table
section has been accepted silently so far. Both the header and the
values are combined into one element per row, so the duplicate column
collapses into a single key with the last value winning, hiding the
mistake in the fixture.

Since data sets are imported with a single bulk insert per table, the
column list is taken from the header row directly and the duplicate
ends up twice in the INSERT statement. MySQL, MariaDB and PostgreSQL
reject that with an error that points neither to the data set nor to
the column:

  SQLSTATE[42000]: Column 'configuration' specified twice

Reading a data set now throws a dedicated exception naming the file,
the table and the duplicated column, for imports and assertions alike,
so the fixture can be fixed right away instead of silently importing
one of the values.

Releases: main, 9
@sbuerk
sbuerk force-pushed the dataset-import-duplicate-columns-9 branch from e8a934f to 03a075d Compare September 16, 2026 13:15
@sbuerk sbuerk changed the title [BUGFIX] Pass duplicate data set columns only once [BUGFIX] Reject data sets listing a column twice Sep 16, 2026
@sbuerk
sbuerk merged commit 79def74 into 9 Sep 16, 2026
8 checks passed
@sbuerk
sbuerk deleted the dataset-import-duplicate-columns-9 branch September 16, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant