Extra accreditation and compliance necessities have been added in response to cyber incidents. Whereas these frameworks play an necessary position in establishing safety baselines, true safety is extra than simply attaining an ideal compliance rating. As I usually say, “policies and procedures won’t stop an attacker, they’ll just have more documents to exfiltrate when they breach us.”
Testing how our environments stand up to a decided risk actor is the true validation of safety posture. That’s the place the annual guide penetration take a look at is available in, with boards now demanding to see constructive outcomes.
There are, nonetheless, important points with guide penetration testing I’ve skilled, notably when carried out solely yearly.
Pace, scope, and the human bottleneck
The constraints of guide testing grew to become more and more obvious as the environment grew extra complicated. Each engagement was certain by time and funds, forcing troublesome trade-offs about what to check and the way deeply. The standard and comprehensiveness of outcomes different considerably relying on which guide we engaged, their particular person experience, their familiarity with rising methods, and the way a lot they might accomplish throughout the contracted hours.
Conventional penetration testing delivered what I got here to see as a basically flawed worth proposition. We’d make investments important funds to obtain a snapshot of our safety posture weeks after the take a look at concluded and from that second it started getting older like milk. There was no ongoing suggestions loop, no steady validation of our safety controls. We have been basically flying blind between annual checks, hoping our defenses remained efficient even because the risk panorama advanced day by day round us.
The remediation black gap
Maybe most irritating was what occurred after we acquired findings. Our groups would work diligently to implement fixes, however we hardly ever had the funds or alternative to deliver testers again to validate remediation. We have been left with uncertainty. This hole between identification and verification created a harmful blind spot in our safety program.
Conventional vulnerability assessments leaned closely on CVSS severity scores that didn’t inform us how exploitable a vulnerability was in our particular setting or the place it sat inside a sensible assault path. We wanted to know what an attacker may truly accomplish by chaining vulnerabilities collectively.
A greater method ahead
Annoyed with these limitations, I explored automated penetration testing, a class that features breach and assault simulation (BAS) and steady automated purple teaming (CART). Platforms like Pentera and Horizon3.ai’s NodeZero conduct steady, on-demand simulations utilizing real-world attacker ways, methods, and procedures.
They provide black field testing (simulating exterior attackers), gray field testing (simulating insider threats), and customized eventualities concentrating on particular dangers like ransomware or zero-day exploits.
Most significantly, they ship outcomes immediately, no ready weeks for experiences, and allow instant retesting to validate fixes.
The implementation and funding
We moved from $35,000 for an annual guide take a look at to $90,000 yearly for an automatic platform, delivering over $1.3 million price of equal testing. Our cadence jumped from one take a look at per 12 months to a minimal of 38, with limitless flexibility for added simulations.
We established a fortnightly rhythm of black field and gray field checks, supplemented by month-to-month customized eventualities concentrating on particular issues like ransomware assaults. This gave our crew two weeks to remediate earlier than retesting confirmed fixes labored. These instruments take a look at extra in a day than human testers accomplish in per week, quickly adjusting to findings and leveraging gaps to probe deeper.
Surprising classes and crew transformation
The platform delivered insights that basically modified our understanding. Take password safety: we’d adopted longer passphrases, assured that fourteen-character phrases would improve breach time from eight months to 12 billion years. The software shattered that confidence, cracking a 23-character passphrase containing upper- and lower-case letters, numbers, and particular characters in below half an hour. The lesson was humbling, people are predictable. Attackers keep wordlists and precomputed hash lists in rainbow tables particularly concentrating on widespread phrases. Passphrase size issues, however high quality issues extra.
The retesting capabilities proved recreation altering. Safety groups may establish issues, remediate them, and instantly retest to confirm fixes have been efficient. The platform generated each executive-level experiences for board displays and detailed technical experiences for safety groups to motion immediately, not weeks later.
Maybe most significantly, the platform elevated our crew’s functionality. Till your crew experiences an automatic penetration testing software exploiting their setting, they gained’t totally comprehend the way to apply defensive ideas to their particular techniques. Every simulated assault was totally documented, offering real-time studying alternatives. The groups started treating the platform as a recreation they have been decided to win.
Rethinking prioritization: assault paths over severity scores
One of the vital important revelations was how automated penetration testing reworked our vulnerability administration. We found that the critical-rated vulnerability receiving instant consideration is likely to be buried 5 layers deep in an assault path, whereas a low-rated vulnerability we’d deprioritized may very well be the preliminary entry level attackers would exploit. Extra revealing nonetheless, the platform confirmed how seemingly low-risk vulnerabilities may very well be chained collectively to entry important techniques.
This modified our patching technique. As a substitute of reflexively addressing vulnerabilities by CVSS severity scores, we targeted on what attackers may truly use to determine a foothold. Given the overwhelming variety of vulnerabilities requiring fixed consideration, this intelligence about precise assault pathways proved invaluable permitting us to focus restricted assets the place they’d produce the best safety consequence somewhat than chasing severity scores that didn’t replicate real-world threat.
The hole between configuration and actuality
We place huge religion in our safety tooling after we allow a characteristic, we assume it’s working. The automated penetration testing platform delivered a sobering lesson: take a look at your controls, don’t simply belief the GUI.
I skilled this firsthand after we enabled a performance to mitigate a selected threat. It regarded good on display, but it surely wasn’t working. The platform methodically examined completely different assault varieties, together with the state of affairs we thought we’d protected towards. The assault succeeded, the safety software’s options weren’t functioning as a result of a bug. We didn’t have the safety we thought we did.
It jogs my memory of the defender’s dilemma: “Defenders have to be right 100% of the time; attackers only have to get it right once.” I’d a lot favor our personal testing instruments spotlight these gaps than have attackers uncover them.
The last word validation: Testing your detection and response
One other highly effective utility is validating your detection instruments and SOC. The primary time I ran a proof of idea, I intentionally didn’t inform our third-party SOC. Our inside SIEM instantly generated quite a few alerts. It took 4 hours for the exterior SOC to contact us — a lifetime in cybersecurity.
Whenever you’re paying for a third-party service, validating their response is invaluable and I strongly advocate operating no less than one unannounced take a look at. The outcomes might shock you, and it’s much better to find gaps throughout your individual testing than throughout an precise incident.
One closing lesson: as your safety resilience improves and also you obtain persistently excessive scores, you attain a plateau. Transferring to a brand new automated penetration testing platform can yield recent findings, as every software takes completely different approaches, offering alternatives to proceed bettering somewhat than changing into complacent.
The decision: Evolution, not elimination
Must you change guide penetration testing with automated platforms? The reply is nuanced. For ongoing safety validation, steady enchancment, and operational resilience, automated testing ought to develop into your main validation technique. The ROI, studying alternatives, and steady suggestions loop far exceed what annual guide testing delivers.
Nevertheless, I wouldn’t utterly eradicate guide testing. There’s nonetheless worth in bringing in specialised human testers for complicated customized purposes, important infrastructure adjustments, or while you want artistic pondering that solely skilled safety researchers present. Consider automated platforms as your day by day coaching routine, with guide checks as occasional specialised assessments.
The actual query is whether or not you’ll be able to afford not to undertake steady automated validation. The hole between annual guide checks leaves you susceptible for 364 days a 12 months. Automated penetration testing fills that hole, transforms your crew’s capabilities, and validates your safety posture constantly, not simply annually when auditors ask.
