The pending updates are a list of thy sins, in reverse chronological order.
Staged dnf patch management across a RHEL 9 fleet.
Collect what is pending, tag the security advisories, and apply with live re-validation and
opt-in serialized reboots.
“And forty of them were severe, and it was the host no one was willing to reboot.” Findings 9:12 — every fleet has exactly one of these
dnf check-update is an examination of conscience. It is honest, it
is complete, and it is entirely useless on eighty hosts one at a time. This runs it
everywhere, tags which updates carry an advisory and at what severity, and then does nothing
at all until thou sayest so.
the three stages — click to advance
Read across the whole fleet, tagged by how much thou shouldst care
Discovery is read-only and genuinely read-only. It enumerates pending updates from
dnf check-update, tags which carry a security advisory — with
severity — from dnf updateinfo list security, and reads
reboot-required state from needs-restarting -r. It writes
patch_plan.json, a workbook, and nothing at all to any target.
The severity comes from the vendor's advisory data rather than from the tool's opinion. A CVE that Red Hat rates Important is rated Important in the sheet, and thou canst point at the advisory id in the meeting where somebody asks why this needs a window.
--security restricts the work to advisories with a security
classification. It is the mode most fleets should live in most of the time: it turns a
six-thousand-package argument into a defensible list, and it is far easier to get a change
window for “the CVEs” than for “everything”.
needs-restarting comes from
dnf-utils / yum-utils. On a host without it,
reboot state is reported as unknown — which is honest, and which is not the
same as no.
The plan is a candidate list, not an instruction
Between the discovery on Tuesday and the window on Saturday, a mirror moved, a package
was superseded, somebody patched one host by hand. So apply re-runs
check-update on the live host and installs only the
intersection of the plan and what is still pending. Nothing else.
Then it asks, per host, with the counts in front of thee:
y patches the host. N, the default, spares
it. q ends the run and leaves the remaining hosts entirely untouched.
And it can be undone. apply writes
patch_results.json recording what was installed and the
transaction id per host. Feed that to rollback and it runs
dnf history undo against a single named host — the package
manager's own undo, not a reimplementation of it.
Report-only until thou optest in, and then one host at a time
A patched kernel that has not been booted is not a patched host. A rebooted database that nobody expected to reboot is an incident. Both are true at once, and no automation can resolve the tension for thee — so the default is to tell thee and stop.
Report-only by default. The report and the plan flag every host whose kernel or core libraries need a restart. Nothing reboots.
--reboot opts in, on the apply run. And then the
discipline is strict: apply installs on a host, schedules its reboot, and
waits — up to --reboot-timeout, default 600
seconds — for that host to answer SSH again before it touches the next one.
One at a time, not in batches. Serialized is the whole point. A bad kernel takes down one host and stops the run; a batch of four takes down four before anyone notices.
Stopping is the feature. The alternative — carrying on through the remaining nineteen while one host is already down — is how a patch window becomes an outage report.
Confess, absolve, and only then raise the body
Auth flags match every other book: keys or agent by default,
--ask-ssh-pass or --ssh-pass-env VAR for
password login, --sudo-pass-same-as-ssh for sudo. Passwords travel by
stdin or the SSHPASS environment variable, never argv.
Breaking the fourth wall, briefly
A real tool, written by a real sysadmin who has sat through the meeting where somebody asks how many hosts are missing the kernel CVE and the honest answer is that nobody has counted. Agentless, MIT licensed.
It shells out to dnf and reads its machine-readable output rather
than reimplementing dependency resolution, because a patching tool that thinks it knows
better than the package manager will eventually be wrong about a transaction on eighty hosts
at once. Rollback is dnf history undo for the same reason.
The remote scripts run under sudo, and package names taken from the plan are shell-quoted before they reach the install command. Untrusted strings written to the report are neutralised, so a hostile package name cannot become a spreadsheet formula when thou openest it. That part is not a joke.
Its plans can also be handed to linux-apply, whose patch adapter re-checks the same packages live before it does anything — though the serialized reboot lives here, and only here.
Built by vK, who has been paged at 03:00 and did not enjoy it. Companion to linux-audit, which examines, and linux-harden, which absolves.
Kneel also at the parish and read at the scriptorium. The Cluster abides. The Loop reconciles. The pending updates accumulate.
Nothing here works alone, and neither shouldst thou
The fleet is one testament in several books. Each does one job, writes a report thou canst hand to anyone, and refuses to be clever with thy servers.