majorinfra-failureverifiedfirsthand
rsync kept dying on iCloud's evicted files with 'Resource deadlock avoided'
An hourly job that mirrors an Obsidian vault on iCloud into a git repository kept intermittently crashing inside rsync with `Resource deadlock avoided`. Some runs failed, some passed, and it was hard to reproduce.
Cause
iCloud's storage optimization had evicted the contents of some files, leaving 'dataless' placeholders. rsync tried to read them with mmap and hit EDEADLK. The evicted files carry no .icloud extension, so they slipped past the script's exclude pattern too.
Consequence
The mirror failed only on runs that touched an evicted file. The first fix swapped macOS's bundled rsync for GNU rsync, but GNU rsync also reads through mmap, and the same deadlock came back three days later.
Fix
Before running rsync, find evicted files with find -flags +dataless, then force them to materialize with brctl download plus head -c 1 before syncing.
What happened
An Obsidian vault lives on iCloud. Every hour, a launchd job rsyncs it into a
git mirror and pushes to a private repository. One day that job sent a
notification: “rsync failed (Code 20).” The log said:
error: <some file>.md: mmap: Resource deadlock avoidedThe chaos on the ground
macOS’s /usr/bin/rsync is called rsync, but it’s actually openrsync, a
separate implementation that describes itself as “rsync 2.6.9 compatible.” It
was trying to mmap iCloud “dataless” files — files whose contents have been
evicted to the cloud by storage optimization, so their size is non-zero but
they occupy zero blocks on disk — and crashing with EDEADLK.
Evicted files don’t get a .icloud extension, so they walked right past the
script’s *.icloud exclusion. On runs where nothing was evicted, the sync
succeeded. So it failed sometimes and passed other times.
The script also carried --inplace, which was itself a workaround for a
different openrsync bug: filenames with emoji or long Japanese names failed with
mkstempat: Illegal byte sequence. The job was caught between two openrsync bugs.
The first fix came out of a session with an AI coding agent. Install GNU rsync
3.4.4 via Homebrew, select it explicitly in the script, and stop with an error
instead of silently falling back to openrsync if it’s missing. Drop --inplace.
The working notes stated that GNU rsync reads with read(), so the mmap
deadlock could not happen. A full sync of 2,980 files succeeded.
Three days later, the same symptom was back under a different exit code:
read errors mapping ...: Resource deadlock avoided (11)GNU rsync uses mmap for reading too. The premise in the notes was wrong.
Root cause
The real problem wasn’t the rsync implementation. It was the evicted files themselves. mmap on a file whose contents aren’t on local disk deadlocks no matter which rsync does it. The first fix leaned on an “openrsync is the culprit” diagnosis and recorded the issue as fully resolved without checking how GNU rsync actually reads files.
The fix
A step now runs before rsync to deal with evicted files first:
find "$VAULT_SRC" -type f -flags +datalesslocates them. It turned out to be the shortest way to detect evicted files.- Each one gets
brctl downloadto request the download, thenhead -c 1to actually read it and confirm it has materialized.
The switch to GNU rsync and the removal of --inplace stayed in place. Forcing
evicted files to download can work against iCloud’s storage optimization;
that caveat remains on the books.
Sources
- Firsthand account from the site operator. There is no public write-up to link to.