RFC / proposed fix for #530: --compare-dest leaves behind directories with no changes

[dr-who]

This is a proposed solution put up for discussion, not a finished change. I'd like a steer on the behaviour question at the bottom before anyone considers merging it — I'm happy to respin it differently.

The problem

--compare-dest is documented to populate the destination with only what differs from the compare hierarchy, but rsync creates every source directory there, including ones whose contents already match. The reporter also noticed those directories aren't mentioned by -v.

Both symptoms come from one spot in recv_generator(). A directory that fully matches makes try_dests_non() return -2, and the caller does:

if (j == -2) { itemizing = 0; code = FNONE; statret = 1; }   /* silences -v ... */
...
if (real_ret != 0 && do_mkdir_at(fname, ...) < 0 ...)         /* ...but still mkdirs */

real_ret is the stat result from before the lookup, so the mkdir happens regardless of the match, and code = FNONE is why -v stays quiet.

How much work that costs

A tree of 200 directories in which only 3 had changed:

	before	after
files created	3	3
directories created	400	6
empty directories	197	0

The reporter saw 129k directories created for 23k that actually changed.

Proposed approach

rsync can't know at mkdir time whether a descendant will differ, so a matching directory is still created — which leaves its attribute handling exactly as it is today and gives any changed descendant a home — and is removed again once the transfer finishes if nothing was placed in it. Candidates are sorted descending so a child is tried before its parent (nested runs collapse), and rmdir() failing on a directory that did receive content is precisely the wanted behaviour. Removing an entry bumps the containing directory's mtime, so that is saved and restored around the rmdir.

I first tried the more obvious thing — skip the mkdir and create parents on demand via make_path() — and abandoned it: directories materialised that way lost their metadata (mode 755 instead of 751, and the current time instead of the source mtime). That's why this takes the create-then-prune route instead.

--link-dest and --copy-dest are deliberately untouched; those build a complete tree by design (verified in testing).

Tests

• testsuite/alt-dest_test.py updated. It builds its check tree as "a full copy minus the two files the alt dirs supply", which retains directories left with nothing to transfer. Its own comment already describes the intent as "dest grows just the deltas", so it now also passes -m (--prune-empty-dirs) to express that.
• testsuite/compare-dest-empty-dirs_test.py added, covering the whole contract: unchanged subtrees leave nothing behind, changed files still arrive (including several levels down), and kept directories keep their permissions and mtimes — including a parent whose child was pruned out of it. It fails on master ("left 39 empty directories behind") and passes with this change.

Full suite: 108 passed, 0 failed.

The question I'd like decided

This changes long-standing user-visible behaviour of --compare-dest. The reporter and the option's documented purpose both say the current behaviour is wrong, but someone may well be relying on the full directory tree appearing.

If compatibility is the greater concern, this can sit behind an opt-in option instead (or be folded into --prune-empty-dirs semantics) — the mechanism is unchanged either way, it's just where it's gated. Tell me which you'd prefer and I'll rework it. Equally happy to drop it if you think the current behaviour is intended.