Penetration testing, Rules of Engagement, and vulnerability validation

NIST SP 800-115 phases and the Rules of Engagement (RoE)

NIST SP 800-115 (Technical Guide to Information Security Testing and Assessment) structures penetration testing into four sequential phases: Planning, Discovery, Attack, and Reporting. No technical reconnaissance or scanning is ever permitted during Phase 1 (Planning). Instead, Planning establishes the assessment goals, target scope, logistics, and the legally binding Rules of Engagement (RoE) signed by an executive with authority over the target systems (including cloud provider notification where required).

A complete Rules of Engagement (RoE) document specifies: exact in-scope and out-of-scope IP ranges, domains, and physical facilities; permitted and prohibited techniques (for example, banning Denial-of-Service floods or social engineering against mission-critical launch operators); allowed testing time windows; data handling rules for any sensitive data encountered; and the 24/7 emergency escalation and "deconfliction" phone numbers.

Phase 2 (Discovery) gathers OSINT, enumerates hosts and ports, and identifies potential vulnerabilities. Phase 3 (Attack) validates whether those vulnerabilities can actually be exploited and chained: gaining initial access, escalating privileges, pivoting laterally, and - crucially before concluding - removing any test shells, test accounts, or artifacts placed during the engagement so the system is not left more vulnerable than before. Phase 4 (Reporting) delivers both an executive business-risk summary and a technical remediation guide.

Vulnerability scanning vs penetration testing and box models

Practitioners never confuse a Vulnerability Scan with a Penetration Test. A vulnerability scanner (such as Nessus, OpenVAS, or Qualys) runs automated checks against known signatures and version tables to report where weaknesses might exist. Running an Authenticated Scan (giving the scanner read-only host credentials) is far more accurate than an unauthenticated network banner scan because it inspects installed DLLs, packages, and kernel patches directly.

A Penetration Test goes further by using human creativity and controlled exploitation to prove whether a vulnerability is genuinely exploitable in context and whether chaining three medium-severity findings yields domain-admin or crown-jewel compromise. Depending on the threat model, leadership selects Black-Box (external blind attacker), Gray-Box (compromised workforce credential or partner API caller), or White-Box (full source and design transparency).

Red Team, Blue Team, Purple Team, and Breach & Attack Simulation (BAS)

While a penetration test aims to find as many exploitable vulnerabilities as possible within a scoped application or subnet, a Red Team Engagement is an objective-based adversarial simulation (for example, "Can an adversary obtain the orbital command signing key without the SOC noticing?") that tests people, physical security, and detection/response across the enterprise.

The Blue Team is the internal defending SOC and incident response staff. When Red and Blue operate in isolation, adversarial knowledge is often shared only in a PDF at the end of the quarter. A Purple Team exercise removes that wall: offensive operators execute specific MITRE ATT&CK techniques (T1003 credential dumping, T1558.003 Kerberoasting) while sitting side-by-side with Blue Team detection engineers to verify that telemetry arrives in the SIEM and alerts fire in real time. To keep those detections from regressing between human exercises, teams deploy automated Breach and Attack Simulation (BAS) agents that continuously replay safe attack behaviors.