Finding, Archiving and Moving Files: find, xargs, tar, gzip/zstd, zip, rsync, diff and Checksums

Version note: The examples in this post were run on a test server running Ubuntu Server 24.04.5 LTS (GNU findutils 4.9.0, GNU tar 1.35, zstd 1.5.5, gzip 1.12, xz 5.4.5, rsync 3.2.7, OpenSSH 9.6p1, coreutils 9.4) and on a second Ubuntu 24.04 environment used as the client. The Ubuntu 26.04.1 LTS container image ships findutils 4.10.0, rsync 3.4.1, zstd 1.5.7 and the uutils sha256sum; the tar --zstd, find -print0 | xargs -0 and sha256sum -c I tried there gave the same output. The Windows tar.exe was not run.

Guess First

The database backup of Bookstore, the online bookstore used as the shared example across this site's series, is written to /var/lib/bookstore/backup every night at 02:30. The disk is filling up and you want to delete the backups older than 7 days. It is 07:20 and there are 11 backups in the directory:

$ sudo ls -l --time-style=+%m-%d\ %H:%M /var/lib/bookstore/backup
total 21516
-rw-r--r-- 1 bookstore bookstore 2000000 11-05 02:30 BookstoreDb-2026-11-05.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-06 02:30 BookstoreDb-2026-11-06.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-07 02:30 BookstoreDb-2026-11-07.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-08 02:30 BookstoreDb-2026-11-08.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-09 02:30 BookstoreDb-2026-11-09.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-10 02:30 BookstoreDb-2026-11-10.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-11 02:30 BookstoreDb-2026-11-11.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-12 02:30 BookstoreDb-2026-11-12.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-13 02:30 BookstoreDb-2026-11-13.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-14 02:30 BookstoreDb-2026-11-14.bak.xz
-rw-r--r-- 1 bookstore bookstore 2000000 11-15 02:30 BookstoreDb-2026-11-15.bak.xz

Today is 15 November. The backup from 8 November at 02:30 was taken 7 days and 5 hours ago, so it is older than 7 days. How many files does find /var/lib/bookstore/backup -name '*.bak.xz' -mtime +7 find: four or three? The answer is in the next section.

This post is about finding, packaging and moving files on a server: find and xargs, tar with gzip, xz and zstd, zip, rsync, and the checksum that shows the package was not damaged on the way. At the end we build Bookstore's release package on one machine, move it to the server and verify it there.

find: Tests and Actions

$ sudo find /var/lib/bookstore/backup -name '*.bak.xz' -mtime +7 | sort
/var/lib/bookstore/backup/BookstoreDb-2026-11-05.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-06.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-07.bak.xz
$ sudo find /var/lib/bookstore/backup -name '*.bak.xz' -mmin +$((7 * 24 * 60)) | sort
/var/lib/bookstore/backup/BookstoreDb-2026-11-05.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-06.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-07.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-08.bak.xz

Three. -mtime drops the fractional part when it calculates a file's age in days: 7 days and 5 hours counts as "7 days", and since +7 means "greater than 7", 8 November is left out. In practice -mtime +7 means "at least 8 days old". -mmin, which counts minutes, uses a real 7 days (10080 minutes) and finds four files. If the retention period is a rule (say, "backups are kept for 7 days"), know which meaning you intend and pick the command accordingly.

A find expression is made of two kinds of parts. Tests select the file: -name (name as a glob, in quotes), -type f / -type d (file or directory), -mtime / -mmin (modification time), -size +1M (size), -newer file or -newermt "2 days ago" (newer than a file or a date), -maxdepth / -mindepth (how deep to go). Actions say what to do with the selected file: -print (the default), -delete, -exec.

$ sudo find /var/lib/bookstore -type f -size +1M -newermt "2 days ago" | sort
/var/lib/bookstore/backup/BookstoreDb-2026-11-14.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-15.bak.xz
$ sudo find /opt/bookstore/releases -mindepth 1 -maxdepth 1 -type d | sort
/opt/bookstore/releases/2026-11-14-6
/opt/bookstore/releases/2026-11-15-1

-exec has two forms, and the difference is how many times the command runs:

$ sudo find /var/lib/bookstore/backup -name '*.bak.xz' -mtime +7 -exec echo rm {} \;
rm /var/lib/bookstore/backup/BookstoreDb-2026-11-07.bak.xz
rm /var/lib/bookstore/backup/BookstoreDb-2026-11-06.bak.xz
rm /var/lib/bookstore/backup/BookstoreDb-2026-11-05.bak.xz
$ sudo find /var/lib/bookstore/backup -name '*.bak.xz' -mtime +7 -exec echo rm {} +
rm /var/lib/bookstore/backup/BookstoreDb-2026-11-07.bak.xz /var/lib/bookstore/backup/BookstoreDb-2026-11-06.bak.xz /var/lib/bookstore/backup/BookstoreDb-2026-11-05.bak.xz

\; runs the command once per file; + collects the files and runs the command as few times as possible. With thousands of files the difference is real, because every run is a new process. The echo I put in front of the command is deliberate too: see what will be deleted before deleting it. Once you trust the list, leave the deleting to find:

$ sudo find /var/lib/bookstore/backup -name '*.bak.xz' -mtime +7 -print -delete
/var/lib/bookstore/backup/BookstoreDb-2026-11-07.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-06.bak.xz
/var/lib/bookstore/backup/BookstoreDb-2026-11-05.bak.xz
$ sudo find /var/lib/bookstore/backup -name '*.bak.xz' | wc -l
8

-print -delete prints every file it deletes; when it runs from a timer, these lines leave a trace in the journal (fifth and sixth posts). find's output comes in directory order, not alphabetically; I added sort to make it readable.

Order matters in find, because the expression is evaluated left to right and -delete is an action, not a test:

$ find /tmp/demo -delete -name "*.log"
$ ls -a /tmp/demo
ls: cannot access '/tmp/demo': No such file or directory

The directory held two .log files and a keep.txt. Because -delete came before -name, it was applied to every file, and at the very end to the directory itself. find gave no warning at all.

xargs and File Names with Spaces

xargs turns the lines it reads from stdin into arguments for a command. It is often used together with the pipes from the seventh post, but it has a trap with file names. If report files that came from Windows have spaces in their names:

$ ls /tmp/exports
sales report 2026-10-01.csv
sales report 2026-10-02.csv
sales report 2026-10-03.csv
sales-2026-11-14.csv
$ find /tmp/exports -name '*.csv' -mtime +30 | xargs rm
rm: cannot remove '/tmp/exports/sales': No such file or directory
rm: cannot remove 'report': No such file or directory
rm: cannot remove '2026-10-03.csv': No such file or directory
...
$ find /tmp/exports -name '*.csv' -mtime +30 -print0 | xargs -0 rm -v
removed '/tmp/exports/sales report 2026-10-03.csv'
removed '/tmp/exports/sales report 2026-10-02.csv'
removed '/tmp/exports/sales report 2026-10-01.csv'

By default xargs splits on whitespace, and one file name becomes three arguments. Here we only got errors; but if the part after a space had matched the name of another file, rm would have deleted that file. -print0 separates file names with the NUL character, the only character that cannot appear in a file name, instead of whitespace, and xargs -0 reads them that way. Write every pipe from find to xargs with this pair. -exec rm {} + does the same job safely too; where xargs really pays off is running work in parallel with -P 4 or passing one argument at a time with -n 1. Using xargs in scripts is covered in the data flow post of the Bash Scripting series.

tar: Packaging, Looking Inside, Extracting

You build the release package on your own machine. If you work on Windows, do this step in Ubuntu on WSL or on a Linux runner in CI; the commands below are the same there. First publish with the command from the first post (dotnet publish src/Bookstore.Api -c Release -r linux-x64 --self-contained false -o out/publish), then package:

$ tar --zstd -cf bookstore-api-2026-11-15-1.tar.zst -C out/publish .
$ tar -tvf bookstore-api-2026-11-15-1.tar.zst
drwxr-xr-x ada/ada           0 2026-11-15 07:03 ./
-rw-r--r-- ada/ada       22244 2026-11-15 07:03 ./Bookstore.Api.pdb
-rw-r--r-- ada/ada        1198 2026-11-15 07:03 ./Bookstore.Api.deps.json
-rw-r--r-- ada/ada         472 2026-11-15 07:03 ./Bookstore.Api.runtimeconfig.json
-rw-r--r-- ada/ada          53 2026-11-15 07:03 ./Bookstore.Api.staticwebassets.endpoints.json
-rw-r--r-- ada/ada       15360 2026-11-15 07:03 ./Bookstore.Api.dll
-rwxr--r-- ada/ada       24360 2026-10-29 20:16 ./Microsoft.Extensions.Hosting.Systemd.dll
-rw-r--r-- ada/ada         119 2026-11-14 15:51 ./appsettings.json
-rwxr-xr-x ada/ada       78192 2026-11-15 07:03 ./Bookstore.Api

tar's letters: c create, t list, x extract; v verbose, f the archive file name (which comes right after it). Compression is chosen with -z (gzip), -J (xz) and --zstd; when extracting, GNU tar detects the compression itself, so -tf and -xf are enough. -C out/publish . means "first change into this directory, then add what is inside", and it determines the paths in the archive. When you forget -C, this is the difference:

$ tar -czf without-c.tar.gz out/publish && tar -tzf without-c.tar.gz | head -n 3
out/publish/
out/publish/Bookstore.Api.pdb
out/publish/Bookstore.Api.deps.json
$ tar -czf nginx-conf.tar.gz /etc/nginx/sites-available
tar: Removing leading `/' from member names
$ tar -tzf nginx-conf.tar.gz
etc/nginx/sites-available/
etc/nginx/sites-available/default

The first archive creates an out/publish/ directory wherever it is extracted; when you extract it into a release directory, the files end up in a subdirectory and the service cannot find Bookstore.Api.dll. When adding with an absolute path, GNU tar strips the leading / itself and warns; that way extracting the archive does not overwrite /etc but creates etc/ where you are. When you see this warning, rewrite the command as -C /etc/nginx sites-available. To leave things out, use --exclude: with --exclude="*.pdb" the debug symbols stay out of the package. I kept them, because I want to see line numbers in the stack trace of an exception in production.

gzip, xz, zstd and zip

Bookstore's framework-dependent package is 70 KB; the compression algorithm does not matter here. To see a difference, I published the same application with the runtime included (self-contained), which made the directory 106 MB. I packaged and extracted it with three algorithms:

The times are from this lab machine and a single run; your absolute numbers will differ, the ratios will stay similar. At its default level (3), zstd produced the same size as gzip ten times faster. xz made the package a quarter smaller, but packaging was about 80 times slower. zstd makes sense for a release package built on every deploy and kept for a few days, xz for backups that are stored for a long time and extracted once; the backup in the twelfth post was compressed with xz too. Because gzip can be opened everywhere, it is still the safest choice for a file you send to someone else.

If you send the package to a teammate who uses Windows, zip is more practical; Windows Explorer opens zip files directly:

$ zip -qr bookstore-api-2026-11-15-1.zip out/publish && unzip -l bookstore-api-2026-11-15-1.zip | tail -n 3
    78192  2026-11-15 07:03   out/publish/Bookstore.Api
---------                     -------
   141998                     9 files

zip does not carry the Linux file owner and group; use tar for packages that go to a server. To look inside a single compressed file without extracting it there are zcat, xzcat, zstdcat and the zgrep from the seventh post.

rsync: The Trailing / and --delete

rsync compares two directories and sends only the difference. The most used flags are -a (keep permissions, times and symlinks), -v (list what is sent), -z (compress on the way) and --dry-run (-n; show what it would do without doing anything). The meaning of the trailing / on the source:

$ rsync -av --dry-run out/publish bookstore-dev:/tmp/stage/
sending incremental file list
publish/
publish/Bookstore.Api
publish/Bookstore.Api.deps.json
...
$ rsync -av --dry-run out/publish/ bookstore-dev:/tmp/stage/
sending incremental file list
./
Bookstore.Api
Bookstore.Api.deps.json
...

out/publish sends the directory itself, out/publish/ sends its contents. A trailing / on the destination makes no difference. --delete makes this distinction dangerous, because it deletes everything on the destination that is not in the source. When I wrote the destination one level too high:

$ rsync -av --delete --dry-run out/publish/ bookstore-dev:/opt/bookstore/
sending incremental file list
deleting releases/2026-11-14-6/web.config
deleting releases/2026-11-14-6/appsettings.json
...
deleting releases/2026-11-14-6/Bookstore.Api
deleting releases/2026-11-14-6/
deleting releases/
deleting current
./
Bookstore.Api
...

Without --dry-run, all releases and the current link would have been gone. Always run --delete with --dry-run first and read the "deleting" lines. In Bookstore we remove this risk completely by extracting every release into a new, empty directory; then --delete is not needed at all.

rsync, scp and sftp all work over SSH and use the ~/.ssh/config alias from the first post. The choice depends on the job: rsync sends only the difference on repeated copies and can resume an interrupted transfer with --partial; scp is the shortest way for a single file; sftp opens an interactive session for browsing the server and fetching files (it also runs from a script with -b). Since OpenSSH 9.0, scp uses the SFTP protocol under the hood:

$ scp -v bookstore-api-2026-11-15-1.zip bookstore-dev:/tmp/stage/ 2>&1 | grep -i subsystem
debug1: Sending subsystem: sftp

So the file name interpretation problems of scp's old protocol are gone; if you need the old protocol, choose it with scp -O.

Bookstore: Moving and Verifying the Package

Produce a checksum file along with the package and send the two together:

$ sha256sum bookstore-api-2026-11-15-1.tar.zst > bookstore-api-2026-11-15-1.tar.zst.sha256
$ cat bookstore-api-2026-11-15-1.tar.zst.sha256
5f34d38564f81c58bcc4181f53adb823605d06bf59cce89885c5bc38b94d756e  bookstore-api-2026-11-15-1.tar.zst
$ rsync -av bookstore-api-2026-11-15-1.tar.zst bookstore-api-2026-11-15-1.tar.zst.sha256 bookstore-dev:/tmp/stage/
sending incremental file list
bookstore-api-2026-11-15-1.tar.zst
bookstore-api-2026-11-15-1.tar.zst.sha256

sent 69,543 bytes  received 54 bytes  139,194.00 bytes/sec
total size is 69,335  speedup is 1.00
$ rsync -av bookstore-api-2026-11-15-1.tar.zst bookstore-api-2026-11-15-1.tar.zst.sha256 bookstore-dev:/tmp/stage/
sending incremental file list

sent 106 bytes  received 12 bytes  236.00 bytes/sec
total size is 69,335  speedup is 587.58

On the second run rsync sent nothing: the files are the same. I did not add -z, because the zstd package is already compressed. On the server, verify first, then extract as the deploy user into a new, empty release directory:

$ cd /tmp/stage
$ sha256sum -c bookstore-api-2026-11-15-1.tar.zst.sha256
bookstore-api-2026-11-15-1.tar.zst: OK
$ sudo -u deploy mkdir /opt/bookstore/releases/2026-11-15-1
$ sudo -u deploy tar -xf bookstore-api-2026-11-15-1.tar.zst -C /opt/bookstore/releases/2026-11-15-1
$ ls -l /opt/bookstore/releases/2026-11-15-1
total 160
-rwxr-xr-x 1 deploy bookstore 78192 Nov 15 07:03 Bookstore.Api
-rw-r--r-- 1 deploy bookstore  1198 Nov 15 07:03 Bookstore.Api.deps.json
-rw-r--r-- 1 deploy bookstore 15360 Nov 15 07:03 Bookstore.Api.dll
-rw-r--r-- 1 deploy bookstore 22244 Nov 15 07:03 Bookstore.Api.pdb
-rw-r--r-- 1 deploy bookstore   472 Nov 15 07:03 Bookstore.Api.runtimeconfig.json
-rw-r--r-- 1 deploy bookstore    53 Nov 15 07:03 Bookstore.Api.staticwebassets.endpoints.json
-rwxr--r-- 1 deploy bookstore 24360 Oct 29 20:16 Microsoft.Extensions.Hosting.Systemd.dll
-rw-r--r-- 1 deploy bookstore   119 Nov 14 15:51 appsettings.json

The files belong to deploy and their group is bookstore: tar run by a user other than root does not take the owner from the archive, and the group comes from the setgid release directory from the third post. To see what the checksum catches, I tried a copy of the package cut off halfway:

$ head -c 30000 bookstore-api-2026-11-15-1.tar.zst > /tmp/broken.tar.zst
$ sha256sum /tmp/broken.tar.zst bookstore-api-2026-11-15-1.tar.zst | cut -c1-16
dddd92d2d679eeef
5f34d38564f81c58
$ tar -tf /tmp/broken.tar.zst
/*stdin*\ : Read error (39) : premature end
tar: Child returned status 1
tar: Error is not recoverable: exiting now
$ cmp /tmp/broken.tar.zst bookstore-api-2026-11-15-1.tar.zst
cmp: EOF on /tmp/broken.tar.zst after byte 30000, in line 133

This time tar caught the error too, because zstd could not find the end of the archive. But tar does not always fail when it extracts a file with one changed byte; a checksum catches even a single bit of difference. cmp compares two files byte by byte and tells you where the first difference is; when you have both copies, it makes a checksum unnecessary. A checksum has a limit too: a .sha256 file that travels the same way as the package catches transfer errors, but someone who changes the package on purpose can change the checksum as well. Against that the package has to be signed; that is not the topic of this post.

One more check before moving the release to current: has anything been changed by hand in the current release on the server?

$ diff -u /opt/bookstore/current/appsettings.json /opt/bookstore/releases/2026-11-15-1/appsettings.json
--- /opt/bookstore/current/appsettings.json	2026-11-14 22:41:00.000000000 +0000
+++ /opt/bookstore/releases/2026-11-15-1/appsettings.json	2026-11-14 15:51:10.000000000 +0000
@@ -1,7 +1,7 @@
 {
   "Logging": {
     "LogLevel": {
-      "Default": "Debug",
+      "Default": "Information",
       "Microsoft.AspNetCore": "Warning"
     }
   }

Someone set the log level to Debug on the server at 22:41 last night to debug a problem (in the lab I made this change on purpose). The new release will undo it; that is right, but you need to know it before the deploy and ask that person. diff -rq old new lists the files that differ between two directories. From here on it is the part of the series you know: switching current to the new release with ln -sfn (second post) and restarting the service (fifth post).

Production Notes

Run every find command that deletes with -print (or -exec echo) first and read the output; then replace only -print with -delete. If it is going to run from a timer, write the path as an absolute path and add -maxdepth; these two limit the damage when a variable comes up empty and find starts from /.

Check the retention rule not only by file age but also by how many backups remain. If the backup job does not run for a few days, "delete everything older than 7 days" deletes all backups; checking that at least a few backups remain before deleting is a job for a script and the topic of the backup script post of the Bash Scripting series.

Build the package once in CI, take its checksum in CI and move the same package to every environment; it is the rule from the "Build Once, Deploy Many" post of the CI/CD Fundamentals series. Rebuilding on the server or recreating the package on different machines means that what you tested and what you deployed are not the same thing.

Common Pitfalls

Running rsync --delete without --dry-run, especially without checking the trailing / on the source or the right level of the destination.

Writing -delete before the tests in find. Order defines the expression; -delete has to come last.

Using find | xargs rm with file names that contain spaces. Use -print0 | xargs -0 or -exec … {} +.

Packaging with an absolute path or without -C. When extracted, the files end up in an unexpected subdirectory.

Reading -mtime +7 as "older than 7 days". For a real 7 days use -mmin +10080 or ! -newermt "7 days ago".

A Small Experiment

★☆☆ In a test directory, create ten files of different ages with touch -d and write down in a table which files -mtime +3, -mtime 3 and -mtime -3 find.

★★☆ Publish Bookstore self-contained and compare tar --zstd with the same command given ZSTD_CLEVEL=19: how much smaller did the package get, and how much longer did it take?

★★★ While sending the package to the server with rsync, interrupt the connection halfway (Ctrl-C), then start again with and without --partial. How much data was sent again in each case? Verify with sha256sum -c at the end.

While moving files to the server we always used SSH through the single alias from the first post. In the next post, the last one of the series, we go deeper into SSH: the details of ~/.ssh/config, connecting to private servers through a bastion, building a tunnel to a remote database with port forwarding, and the risk of agent forwarding.