Skip to content

fix(SelectQuery): process the last partial chunk in runChunks()#255

Merged
roxblnfk merged 4 commits into
2.xfrom
bugfix/runchunks-last-partial-chunk
Jun 11, 2026
Merged

fix(SelectQuery): process the last partial chunk in runChunks()#255
roxblnfk merged 4 commits into
2.xfrom
bugfix/runchunks-last-partial-chunk

Conversation

@roxblnfk

Copy link
Copy Markdown
Member

🔍 What was changed

SelectQuery::runChunks() no longer skips the last partial chunk.

The loop guard was $offset + $limit <= $count, which required a full remaining chunk and stopped as soon as fewer than $limit rows were left. The trailing partial page ($count % $limit rows) was silently dropped — no error, no warning. The guard is changed to $offset < $count, so the final iteration issues LIMIT N OFFSET k and naturally returns the remaining < $limit rows.

Note

The same guard also broke the limit > count case entirely: a table with fewer rows than the chunk size processed zero rows. This is fixed by the same change and covered by a dedicated test.

🤔 Why?

Silent data loss in any batch/export/migration built on runChunks(). Full coverage previously happened only when $count was an exact multiple of $limit.

Example — 2500 rows, runChunks(1000, …):

offset=0     0+1000 <= 2500  ✓  → rows 0…999
offset=1000  1000+1000<=2500 ✓  → rows 1000…1999
offset=2000  2000+1000<=2500 ✗  → STOP
                                  rows 2000…2499  NEVER processed

📝 Checklist

  • Closes [Bug]: SelectQuery::runChunks() silently skips the last partial chunk (data loss) #254

  • How was this tested:

    • Tested manually
    • Unit tests added

    Added regression and edge-case tests to StatementTest (common across all drivers):

    • testChunksProcessLastPartialChunk — trailing partial page is visited (count % limit != 0)
    • testChunksWithLimitGreaterThanCount — chunk size larger than the row count still yields every row
    • testChunksPassOffsetAndCountToCallback$offset/$count callback arguments
    • testChunksOnEmptyResultNeverInvokesCallback — empty result set never invokes the callback

📃 Documentation

Not needed — behavioral bugfix, public API unchanged.

roxblnfk and others added 3 commits June 11, 2026 13:34
Adds a regression test for #254: runChunks() silently drops the trailing partial page when the total row count is not a multiple of the chunk size (e.g. 10 rows with a chunk size of 3 leaves row #10 unvisited).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The loop guard `$offset + $limit <= $count` stopped as soon as fewer than $limit rows remained, so the trailing partial page (count % limit rows) was never passed to the callback — silent data loss in any batch/export/migration built on runChunks(). Changing the guard to `$offset < $count` lets the final iteration return the remaining rows via LIMIT/OFFSET.

Fixes #254

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Covers cases left untested around #254: chunk size larger than the total row count (previously processed zero rows under the old guard), an empty result set, and the $offset/$count arguments passed to the callback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@roxblnfk roxblnfk requested review from a team as code owners June 11, 2026 09:38
@github-actions github-actions Bot added the type: test Test label Jun 11, 2026
@roxblnfk roxblnfk requested a review from Copilot June 11, 2026 09:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a data-loss bug in SelectQuery::runChunks() where the final partial chunk (and the limit > count case) could be skipped, and adds regression tests to ensure all rows are processed across chunk boundaries.

Changes:

  • Update SelectQuery::runChunks() loop guard to process the trailing partial chunk ($offset < $count).
  • Add functional regression tests covering partial tail chunks, limit > count, empty results, and callback offset/count arguments.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/Query/SelectQuery.php Adjusts chunk iteration condition so the final partial page is not skipped.
tests/Database/Functional/Driver/Common/Driver/StatementTest.php Adds new test cases validating chunk traversal behavior and callback arguments.
Comments suppressed due to low confidence (1)

src/Query/SelectQuery.php:312

  • runChunks() can enter an infinite loop when $limit <= 0 (because $offset never increases meaningfully, and the loop guard only checks $offset < $count). Since runChunks() accepts any int, it should validate that the chunk size is a positive integer and fail fast with an exception to avoid hanging processes.
        while ($offset < $count) {
            $result = $callback(
                $select->offset($offset)->getIterator(),
                $offset,
                $count,

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/Database/Functional/Driver/Common/Driver/StatementTest.php
MSSQL returns the paginated id column as a string and its OFFSET/FETCH ordering is not guaranteed without an explicit ORDER BY. Add orderBy('id') for deterministic iteration and use assertEquals (matching the existing testChunks convention) so the row-value assertions don't depend on the driver's scalar type.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.68%. Comparing base (c4d40ce) to head (b4bf8a9).
⚠️ Report is 1 commits behind head on 2.x.

Additional details and impacted files
@@            Coverage Diff            @@
##                2.x     #255   +/-   ##
=========================================
  Coverage     95.68%   95.68%           
  Complexity     2038     2038           
=========================================
  Files           137      137           
  Lines          5775     5775           
=========================================
  Hits           5526     5526           
  Misses          249      249           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@roxblnfk roxblnfk merged commit 38c3403 into 2.x Jun 11, 2026
30 of 31 checks passed
@roxblnfk roxblnfk deleted the bugfix/runchunks-last-partial-chunk branch June 11, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: SelectQuery::runChunks() silently skips the last partial chunk (data loss)

2 participants