Add index schema 2.1 with delta support

[JohnMcPMS]

📖 Description

Adds a new schema version with support for generating and consuming delta databases. When using the new schema, the general flow is:

1. Update the index as normal; when appropriate, mark a baseline.
2. When making future updates, provide the baseline and delta output locations before packaging prep.
3. This will produce the normal package ready index and a delta from the baseline.

This allows the existing flow to be maintained while also producing deltas. A future change will introduce the public surface to do those interactions.

The spec (#6500) has more details, but the basic concept is that we record the difference between the baseline and the new index, keeping package rows fixed so that we can merge them easily at runtime. The delta uses new table names and creates views that merge the tables from the delta and baseline. This means that the code to read the index doesn't need to change.

🔗 References

Spec: #6500

🔍 Validation

Added significant new test coverage for delta scenarios.

Microsoft Reviewers: Open in CodeFlow

[ranm-msft]

This had no reviews yet, so flagging that I have read it - focused on the merge/view layer (DeltaViews.cpp, DeltaGeneration.cpp) and the builder/wrapper changes rather than all 36 files.

Three things I went looking for and was glad not to find problems with:

• ATTACH binds the path as a parameter rather than interpolating it, so a baseline filename is not an injection surface.
• DatabaseSpecifier is a nice solve for the disposition problem. Having one type render the URI means the ATTACH target and the sqlite3_open_v2 target cannot drift apart, which is the bug I went looking for first.
• Association suppression is row-level rather than package-level, so a delta that changes one mapping for a package does not silently drop that package's other baseline associations. That one is easy to get wrong.

I also convinced myself the writtenRowIds collision check is not dead code - it is what lets a removed-then-re-added package reclaim its rowid - and that validating baseline affinity before attaching is deliberate, so a failed validation does not leave the caller's connection carrying a half-configured schema.

The only thing I would reconcile is on the spec side rather than here: the view SQL in #6500 no longer matches these definitions. I left the detail there.

One question: is the expectation always baseline plus a single delta, or could deltas chain? The rowid allocation reads like it assumes one delta over one baseline, and I could not tell from here whether chaining is out of scope by design.

[JohnMcPMS]

One question: is the expectation always baseline plus a single delta, or could deltas chain? The rowid allocation reads like it assumes one delta over one baseline, and I could not tell from here whether chaining is out of scope by design.

The current design is only to have one baseline and one delta at this time.

[Copilot]

🟡 Changes recommended

Direct 1.7-to-2.1 migration is broken, and delta affinity and metadata handling have unresolved integrity gaps.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds schema 2.1 support for generating delta indexes and transparently merging them with baseline indexes.

Changes:

• Adds delta generation, tables, merged views, and baseline affinity handling.
• Extends update tracking with removals and change sequences.
• Adds comprehensive delta and SQLite builder tests.

File summaries

File	Description
src/AppInstallerSharedLib/SQLiteWrapper.cpp	Adds database specifier support.
src/AppInstallerSharedLib/SQLiteStorageBase.cpp	Opens storage through database specifiers.
src/AppInstallerSharedLib/SQLiteStatementBuilder.cpp	Adds SQL constructs needed by deltas.
src/AppInstallerSharedLib/Public/winget/SQLiteWrapper.h	Declares database dispositions and specifiers.
src/AppInstallerSharedLib/Public/winget/SQLiteStorageBase.h	Exposes specifier-based storage construction.
src/AppInstallerSharedLib/Public/winget/SQLiteStatementBuilder.h	Declares new statement-builder operations.
src/AppInstallerRepositoryCore/Microsoft/SQLiteIndex.h	Adds delta properties and baseline APIs.
src/AppInstallerRepositoryCore/Microsoft/SQLiteIndex.cpp	Implements delta opening, generation configuration, and baseline marking.
src/AppInstallerRepositoryCore/Microsoft/Schema/SQLiteIndexContextData.h	Adds delta path context properties.
src/AppInstallerRepositoryCore/Microsoft/Schema/ISQLiteIndex.h	Adds schema-level delta operations.
src/AppInstallerRepositoryCore/Microsoft/Schema/ISQLiteIndex.cpp	Registers schema 2.1.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/Interface.h	Defines the 2.1 interface.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/Interface_2_1.cpp	Implements migration and delta lifecycle behavior.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/DeltaViews.h	Declares merged delta views.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/DeltaViews.cpp	Builds baseline/delta merge views.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/DeltaTables.h	Declares delta table helpers.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/DeltaTables.cpp	Creates and packages delta tables.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/DeltaGeneration.h	Declares delta generation.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/DeltaGeneration.cpp	Computes and writes delta contents.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_0/PackageUpdateTrackingTable.h	Extends tracking contracts for removals and sequences.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_0/PackageUpdateTrackingTable.cpp	Records and queries delta-relevant changes.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_0/PackagesTable.h	Supports explicit package row IDs.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_0/PackagesTable.cpp	Preserves row IDs during packaging.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_0/OneToManyTableWithMap.cpp	Exposes map-table naming helpers.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_0/Interface.h	Adds packaging extension state and hooks.
src/AppInstallerRepositoryCore/Microsoft/Schema/2_0/Interface_2_0.cpp	Integrates stable IDs and delta packaging hooks.
src/AppInstallerRepositoryCore/AppInstallerRepositoryCore.vcxproj.filters	Groups schema 2.1 files.
src/AppInstallerRepositoryCore/AppInstallerRepositoryCore.vcxproj	Builds schema 2.1 sources.
src/AppInstallerCLITests/SQLiteWrapper.cpp	Tests new SQLite wrapper and builder behavior.
src/AppInstallerCLITests/SQLiteIndexTestCommon.h	Declares shared index test fixtures.
src/AppInstallerCLITests/SQLiteIndexTestCommon.cpp	Implements shared index test fixtures.
src/AppInstallerCLITests/SQLiteIndexDelta.cpp	Adds extensive delta behavior tests.
src/AppInstallerCLITests/SQLiteIndex.cpp	Uses shared fixtures and updates tracking tests.
src/AppInstallerCLITests/main.cpp	Adds optional failed-test summaries.
src/AppInstallerCLITests/AppInstallerCLITests.vcxproj.filters	Groups new test files.
src/AppInstallerCLITests/AppInstallerCLITests.vcxproj	Builds new delta tests and helpers.

Review details

• Files reviewed: 36/36 changed files
• Comments generated: 3
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

Migration paths, reusable delta output generation, and read-only baseline attachment currently contain correctness issues.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

src/AppInstallerRepositoryCore/Microsoft/Schema/2_1/Interface_2_1.cpp:40

• A direct 1.7 → 2.1 migration fails here. Because this object already has m_trackingRemovalBehavior == Record, the preceding V2.0 migration creates update_tracking with the removal columns; this call then attempts to add is_removed again and SQLite reports a duplicate column. Commit immediately when the V2.0 migration succeeded, and only alter an existing table for an actual 2.0 → 2.1 migration.

        bool v2result = V2_0::Interface::MigrateFrom(connection, current);

        // Migration from 2.0 → 2.1
        if (v2result || (currentVersion.MajorVersion == 2 && currentVersion.MinorVersion == 0))
        {
            V2_0::PackageUpdateTrackingTable::AddRemovalTrackingColumns(connection);

• Files reviewed: 36/36 changed files
• Comments generated: 6
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

URI path encoding can target wrong files, and failed packaging can leave a stale delta that blocks retries.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 40/41 changed files
• Comments generated: 2
• Review effort level: Balanced