Pavois
FR

From first public release to an evidence-backed delivery chain

Project news · September 18, 2026 · 8 min read · Stéphane Robert

Pavois has been public since 16 September 2026. The first releases exposed a gap between tests run from a checkout and the standalone artefact delivered to a clean machine. Here is what that gap changed in the delivery chain, and what is now mechanically verified.

On 16 September 2026, Pavois went from a private repository to a public project. It is a Linux compliance scanner that audits a machine’s effective configuration rather than its configuration files, grades it A to E per standard (ANSSI BP-028, CIS, PCI DSS, NIST, STIG), and can apply and then undo its own hardening. This post covers why it stayed private for so long, what the first 48 hours broke, and where I want to take it.

Why it stayed private

The code was ready well before. What was not ready was me. I pushed the date back more times than I counted, for one reason: fear of looking ridiculous. Publishing a compliance tool means exposing yourself to someone running it on their machine and finding out the verdict is wrong. In this field, a tool that is wrong is worse than useless: it delivers false confidence, and false confidence on a production server has a real cost.

That reasoning has a flaw, and it cost me months: a tool that stays private never gets it wrong in public, but it never gets better either. No amount of tests written by the person who wrote the code replaces one user who does not know what they are supposed to do.

What the first 48 hours proved

Five releases in three days. Those five releases are not five reworks of the product: they mostly trace the stabilisation of its delivery chain. What was actually wrong fits in one sentence.

v0.1.0 and v0.1.1 shipped a binary incapable of doing its job. Both times for the same reason: every test I ran lived inside the repository, where the rule profiles and the reference sit on disk. The published binary arrives alone on a fresh machine. Both times a user on a clean VM found it, not me. v0.1.2 did worse in another register: three empty embedded directories, which is not a build error. It compiles, it publishes, and the only symptom is a sentence the user meets on their first command: this binary embeds none and none is on disk.

The fix was not a patch, it was a ladder where each rung catches a class the previous one cannot see:

rung what it catches
prepush everything a pull request would refuse, offline
release:standalone a path resolved against the current directory
release:ci-build what the release builds, not what I build
release:preflight the package, extracted and executed
release:scenario the whole scenario on a clean VM
release:published the artefact GitHub actually published

The scenario rung carries one assertion per closed first-contact issue. An issue closed only in a changelog comes back; an issue closed with a line in a harness does not.

In three days, strangers found what months of anxiety had not fixed. That is the strongest argument I know against keeping a project private.

What Pavois does, in one idea

It audits resolved state, not files. A scanner reading /etc/ssh/sshd_config misses Include directives, drop-ins, and everything a distribution package layers on top. Pavois asks the daemon what it actually applies:

# what most scanners do
grep PermitRootLogin /etc/ssh/sshd_config

# what Pavois does
sshd -T | grep permitrootlogin

Same principle throughout: sysctl -a for the kernel, systemctl show for units, nginx -T and apachectl -S for web servers. It costs more (it needs root, hence --sudo) and it is the only reading that describes the machine as it runs.

pavois scan local --sudo          # A to E grade + a self-contained HTML report
pavois harden plan user@host --sudo --key ~/.ssh/id_ed25519
pavois harden apply plan.yml --reboot --scan   # a PASS after reboot is proven

“I already have Ansible and a scanner. Why add Pavois?”

It is the question I have been asked most, and the honest answer starts with what Pavois does not ask for.

Your configuration management stays the source of desired state. Ansible, Puppet or Packer describe what the machine should be; Pavois reads back what it actually resolved, which is not the same question and not the same answer. A playbook that converged without an error does not tell you that a drop-in added afterwards has not undone its work.

Your compliance scanner can stay where it is. The useful first step is not a replacement, it is a comparison: one image, both tools side by side, and you look at what each of them measured and with what evidence. It costs little, it is reversible, and it decides on facts rather than on a promise.

What Pavois adds is three things: it reads the effective state rather than the files, it says whether a PASS survives a reboot, and it produces evidence you can export as OSCAL, SARIF or JSON. If you do not miss those three, keep what you have.

“Do I need to adopt several standards?”

No. The site encouraged that confusion, and it deserved fixing.

The multiple mappings belong to Pavois’s corpus: one neutral control carries all of its CIS, ANSSI BP-028, NIST, PCI DSS and STIG references, which is how the rule base avoids writing the same check five times. That is a property of the corpus, not an obligation for you.

If your organisation only answers to ANSSI, audit only ANSSI:

pavois scan local --sudo --standard bp28

--standard filters what the engine runs, not what the report displays. And pavois harden apply takes it too, for the remediation values that differ between standards.

One last distinction that matters: a mapping is not a certification. A control carrying the reference CIS 5.2.1 means it measures what that paragraph asks for, not that your machine is CIS certified. Pavois produces the evidence; the auditor produces the verdict.

Where I want to take it

The goal is not to pile up controls. It is to make Pavois a methodology you can argue with: a tool whose verdicts can be challenged because they are motivated, sourced and reproducible.

A qualified verdict rather than a green light. A PASS has to say two things: the setting is live now, and it survives a reboot. Those are two independent axes, and the grade accounts for both. It is already in place and explained here.

Three states, not two. ALLOWED, BLOCKED and UNKNOWN. Absence of evidence is never evidence of absence. A control Pavois cannot measure has to say it cannot measure it, and that verdict must not count as a pass. UNKNOWN beats false confidence, and that is the work in progress.

Say what it cannot fix. A separate filesystem is decided at install time, a missing KSPP option needs a rebuilt kernel, a GRUB password can lock you out. Those three classes are not something an apply can close, so the report prints two grades: the raw grade, and the remediable posture, meaning what Pavois can reach on its own.

Know how to go back. Among the tools that remediate (OpenSCAP, Ubuntu’s USG, the CIS Build Kits, ansible-lockdown), none ships the inverse of its own remediation. Pavois does, at 96% measured on a fresh Debian 12, and it publishes the 4% that do not come back, one by one. A rollback is not a null operation, and the manifest says so before you confirm.

Output other tools can read. The rule base exports as OSCAL, results as SARIF, JUnit, JSON and CSV. Evidence that cannot leave its own tool is not evidence, it is a screenshot.

What is not true today

Pavois covers nine Linux systems. Two have been through the full campaign on a clean VM: Debian 12 and Debian 13. The other seven are curated and statically validated, but no campaign has run on them: treat them as experimental. “9 systems supported” and “9 systems proven” are not the same sentence, and only one of them is true.

There is no fleet aggregation either. One run audits one target. A loop and a report directory are enough, because every scan emits a complete JSON, but that is a stated gap, not a hidden feature.

Key takeaways

  • Pavois has been public since 16 September 2026, under Apache-2.0.
  • It audits effective configuration (sshd -T, sysctl, systemctl show), not files.
  • The gap that mattered was not in the controls but in the packaging: tests run from a checkout cannot see what a standalone artefact does on a clean machine. Five releases in three days closed that hole, and each rung of the ladder is the trace of it.
  • The direction: a qualified verdict, three states including UNKNOWN, two grades including the remediable posture, and a rollback that publishes its own limits.

Next steps

← All posts