Skip to content

Local rsync --sparse extremely slow with regular file #773

Description

@q3cpma

Hello, I got this strange behavior while filling my new RAID 6 from a backup (3.5" hard drive in USB 3 dock):

Versions and topology
$ command rsync --version
rsync  version 3.4.1  protocol version 32
Copyright (C) 1996-2025 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, symlinks, symtimes, hardlinks, hardlink-specials,
    hardlink-symlinks, IPv6, atimes, batchfiles, inplace, append, ACLs,
    xattrs, optional secluded-args, iconv, prealloc, stop-at, no crtimes
Optimizations:
    SIMD-roll, no asm-roll, openssl-crypto, no asm-MD5
Checksum list:
    md5 md4 sha1 none
Compress list:
    zlibx zlib none
Daemon auth list:
    sha512 sha256 sha1 md5 md4

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.
$ uname -a
Linux gentoo-zen2600 6.12.16-gentoo #3 SMP PREEMPT_DYNAMIC Sun May 25 13:18:45 CEST 2025 x86_64 AMD Ryzen 5 2600 Six-Core Processor AuthenticAMD GNU/Linux
$ lsblk
NAME                  MODEL                PARTLABEL FSTYPE              SIZE MOUNTPOINT
sda                   ST10000NT001-3LY101                                9.1T
└─sda1                                               DM_integrity        9.1T
  └─dm-integrity-sda1                                linux_raid_member   9.1T
    └─md0                                            xfs                18.2T /home/user/data
sdb                   ST10000NT001-3LY101                                9.1T
└─sdb1                                               DM_integrity        9.1T
  └─dm-integrity-sdb1                                linux_raid_member   9.1T
    └─md0                                            xfs                18.2T /home/user/data
sdc                   ST10000NT001-3LY101                                9.1T
└─sdc1                                               DM_integrity        9.1T
  └─dm-integrity-sdc1                                linux_raid_member   9.1T
    └─md0                                            xfs                18.2T /home/user/data
sdd                   ST10000NT001-3LY101                                9.1T
└─sdd1                                               DM_integrity        9.1T
  └─dm-integrity-sdd1                                linux_raid_member   9.1T
    └─md0                                            xfs                18.2T /home/user/data
sde                   LDLC                                             111.8G
├─sde1                                                                     2M
├─sde2                                               ext2                128M /boot
├─sde3                                               swap                  8G [SWAP]
└─sde4                                               xfs               103.7G /
sdf                   HGST HUH728080ALE604                               7.3T
└─sdf1                                               xfs                 7.3T /home/user/mnt.sdf1
$ dd if=/dev/urandom of=mnt.sdf1/big bs=1M count=1k
$ command rsync --progress --sparse mnt.sdf1/big data/
big
      9,502,720   0%    1.36MB/s    0:12:41  ^C
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at io.c(519) [generator=3.4.1]
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(716) [sender=3.4.1]
$ command rsync --progress mnt.sdf1/big data/
big
  1,073,741,824 100%  391.58MB/s    0:00:02 (xfr#1, to-chk=0/1)

I had always enabled --sparse before that, to handle the occasional sparse file without thinking much of it. Any idea why? I can run further debug commands if needed.

EDIT: wait, is this related to #319 ? --sparse may be very different from what I thought... if this is about finding holes by brute force instead of skipping holes known by the filesystem in sparse files, I suggest something to mention it in the manual and discourage it for most cases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions