Borg 1.4.0 peeks into excluded directories

[dukelsky]

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

ISSUE

System information. For client/server mode post info for both machines.

borg 1.4.0 at both the server and the client. I tried both self built rpm package and borg-linux-glibc231 from Releases on the client machine.

Fedora Linux KDE 40 (client), AlmaLinux 9.4 (server)

Client: ASUS PRIME B550-PLUS motherboard, AMD Ryzen 7 5700G, 64 GB RAM.

How much data is handled by borg?

~600 GB

Full borg commandline that lead to the problem

/usr/bin/borg --iec --show-rc create --list --filter=- --dry-run --compression zstd --patterns-from=$PATTERN_FILE borg@nas.example.org:/path/to/repo::rootfs-{now:%Y-%m-%d} / >> $LOG 2>&1

cat $PATTERN_FILE
-/dev/
-/home/*/.cache/
-/media/
-/mnt/
-/proc/
-/root/.cache/
-/run/
-/sys/
-/tmp/
-/var/cache/
-/var/crash/
-/var/lib/libvirt/images/
-/var/lib/mock/
-/var/lib/sddm/.cache/
-/var/log/
-/var/spool/
-/**/tmp/

Describe the problem you're observing.

Most excluded directories are not backed up, but I find the following in the log:

...
/proc/6915/task/6915/fd/10: stat: [Errno 2] No such file or directory: '10'
/proc/6915/task/6915/fdinfo/10: stat: [Errno 2] No such file or directory: '10'
/proc/6915/fd/7: stat: [Errno 2] No such file or directory: '7'
/proc/6915/fdinfo/7: stat: [Errno 2] No such file or directory: '7'
- /proc
- /var/log
- /var/lock
...
- /media
/run/user/1000/doc: stat: [Errno 13] Permission denied: 'doc'
- /run

Can you reproduce the problem?

The problem is reproducible, and it does not matter whether --dry-run is used or not. I did not try the previous versions, since this is the first time I used borgbackup.

[ThomasWaldmann]

I suspect your patterns don't match like you think they do. See the patterns help page and use these commands to see what's excluded and what not:

• borg list REPO::archive <- check how the paths look like: no leading slash!
• borg create --dry-run --list ... <- use this to test your adapted patterns

[dukelsky]

I removed the leading slash in all patterns, but the result is the same: borg peeks into /proc and /run.

[dukelsky]

I removed --filter=- from the command line, and now I see that borg lists everything in all "excluded" directories including /sys, /proc, /run etc.

x /proc/1872/task/1939/fdinfo/251
x /proc/1872/task/1939/fdinfo/253
x /proc/1872/task/1939/fdinfo/254
x /proc/1872/task/1939/fdinfo/256
x /proc/1872/task/1939/fdinfo/258
x /proc/1872/task/1939/fdinfo/284
x /proc/1872/task/1939/fdinfo/255
/proc/1872/task/1939/fdinfo/255: stat: [Errno 2] No such file or directory: '255'

But why?

[Aztorius]

I removed --filter=- from the command line, and now I see that borg lists everything in all "excluded" directories including /sys, /proc, /run etc.

x /proc/1872/task/1939/fdinfo/251
x /proc/1872/task/1939/fdinfo/253
x /proc/1872/task/1939/fdinfo/254
x /proc/1872/task/1939/fdinfo/256
x /proc/1872/task/1939/fdinfo/258
x /proc/1872/task/1939/fdinfo/284
x /proc/1872/task/1939/fdinfo/255
/proc/1872/task/1939/fdinfo/255: stat: [Errno 2] No such file or directory: '255'

But why?

Borg lists everything because an include rule may match on a subdirectory or file.

I believe what you are looking for is the exclude no-recurse pattern prefix '!' instead of the basic exclude '-'. See the doc : https://borgbackup.readthedocs.io/en/stable/usage/help.html

From the doc :

Use the prefix !, followed by a pattern, to define an exclusion that does not recurse into subdirectories. This saves time, but prevents include patterns to match any files in subdirectories.

[dukelsky]

Thanks for the tip.

[ThomasWaldmann]

Just wanted to note that using ! instead of - is:

• advisable for dirs like /proc and /sys on Linux where no includes below the excluded dir are expected
• but it is not the root cause why your patterns file did not work. the root cause is likely that you did not match the paths of all files that you wanted to exclude.

[dukelsky]

I want to include the special directories /dev, /proc, /run, /sys, but not their contents. I tried this:

+ re:^(dev|proc|run|sys|tmp)$
! re:^(dev|proc|run|sys|tmp)

But despite my expectations, there were directories one level lower in the list.

x /sys/fs
x /sys/devices
x /sys/dev
x /sys/bus
x /sys/class
x /sys/firmware
x /sys/hypervisor
x /sys/kernel
x /sys/power
x /sys/module
x /sys/block
- /sys

[Aztorius]

If you want to include /sys but not its content, I think you have done the right thing. Borg needs to list the content to exclude it with the rules.

[ThomasWaldmann]

IIRC:

! dev/
! proc/
! sys/

Important:

• no leading slash, see borg list output
• if you want to back up the directories, but not their contents, have the trailing slash. having that will not exclude the directory itself (because it does not yet match the trailing slash in the full path).
• it WILL "peek into" these directories, listing and trying to match the files in there. but THEN, all candidate paths will match the pattern and all will be excluded (and not opened / read).
• because you used !, it will also not recurse any further.

[ThomasWaldmann]

Duplicate of #7220.