User Tools

Site Tools


linux:sync-non-blocking

Safe Data Migration under High I/O Load

Use this method when you need to transfer data from a storage device that is already under heavy load (e.g., RAID resync, slow USB, high latency) to prevent system lockups (high I/O Wait).

The Command

ionice -c 3 rsync -avxW --progress /source/path/ /destination/path/

Parameter Breakdown

  • ionice -c 3 (Idle Class)
    • Function: Sets the I/O scheduling class to “Idle”.
    • Effect: The process will only read/write to the disk when no other program needs access to it.
    • Benefit: Prevents the system from freezing or becoming unresponsive (High Load/IO Wait) during the transfer.
  • rsync -W (–whole-file)
    • Function: Copies the whole file without calculating delta checksums.
    • Effect: Reduces CPU usage and disk seek operations.
    • Benefit: Significantly faster for directories with thousands of small files (e.g., `node_modules`, Docker overlays) where calculating diffs is more expensive than the transfer itself.
  • -avx
    • -a: Archive mode (preserves permissions, timestamps, ownership).
    • -v: Verbose (shows progress).
    • -x: One File System (prevents rsync from crossing partition boundaries).

Typical Use Case

Migrating Docker data or large file structures while a RAID array is resyncing or scrubbing:

# Example: Moving Docker data to a new partition
ionice -c 3 rsync -avxW --progress /var/lib/docker/ /mnt/new_drive/docker/
linux/sync-non-blocking.txt · Last modified: by odefta