Safety rails, not safety warnings
Most automated security tooling ships its safety story in the README. "Only run this against systems you're authorized to test." "Be careful with the rate limits." "Don't point it at production." These are warnings — and warnings are the weakest possible control, because they depend entirely on a tired human reading them and staying disciplined at 2 a.m.
As we hand more of the discovery loop to automation and AI agents, that model breaks. A system that can decide what to probe next, and act on it in milliseconds, cannot have its safety rest on operator attentiveness. The constraints have to live in the architecture.
Capability is outrunning control
The research energy in automated vulnerability discovery goes almost entirely into capability: find more classes of bug, find them faster, chain them further. Authorization, scope, and blast-radius get treated as operational concerns — something the operator wraps around the tool, not something the tool guarantees about itself.
That's backwards for anything autonomous. If the system can take actions on its own, then the question that matters is not "how capable is it?" but "what is it structurally unable to do?"
A safe automated system isn't one that's been told the rules. It's one that cannot break them even when the discovery logic "wants" to.
What architectural safety looks like
In the research system behind my thesis, the discovery engine is deliberately not in charge. It's subordinate to an enforcement layer that sits between it and the world, and every outbound action passes through gates it cannot route around:
- Authorization gating. No action executes without an explicit, checkable authorization for that target. Absent or expired authorization isn't a warning — it's a hard stop.
- Scope enforcement. Declared scope is a boundary the engine physically cannot cross. "Out of scope" isn't a policy the operator remembers; it's a wall.
- Blast-radius and rate control. Impact is bounded and observable by construction, not by hoping the defaults are sane.
- Verification before report. A finding has to be validated before it counts, which also curbs the automation's incentive to be reckless for signal.
- Kill-switch. One reliable stop that actually stops everything.
- Tamper-evident audit. Every action is accountable after the fact, so the system's behaviour is reviewable rather than taken on trust.
Why this is the interesting problem
It would be easy to treat all of this as compliance overhead — boxes to tick so nobody sues you. I think that misses the point. Designing a system whose restraint is provable is a genuinely hard engineering problem, and it's the one that decides whether automated discovery is a net good or a liability.
It's also where the research contribution is. Anyone can bolt on more capability. Making the safety non-bypassable — and measuring what that costs you in coverage — is the part worth a thesis.
Where this goes next
The obvious follow-up question: if you enforce all of this, how much discovery power do you give up? That trade-off — the cost of being safe — is the subject of the next post.
This is part of an open build log for my Master's research on safe automated vulnerability discovery. All experimentation runs against self-owned, authorized lab targets only. More about the research →