Moving beyond a “check-the-box” approach is critical to maximizing the value that can be derived from pen testing. Successful organizations use the results to identify opportunities for improvement, drive their cybersecurity and IT budgets, and facilitate smart business decisions.
Additionally, it’s important to consider the results of a pen testing exercise holistically, not in isolation. Comparing current findings to previous test reports and recommendations can help identify trends or recurring patterns that can enable more focused and effective risk mitigation efforts.
Just how much value can be added depends on several factors, the most obvious being the skills, experience, and quality of the testing organization. At a minimum, the team that is selected should demonstrate a clear understanding of any specific regulatory requirements that must be met. Ideally, the testing group will have relevant experience with organizations of comparable size in the same or related industries as the testing target. But to truly maximize the value of the experience, both the testing group and the target organization should advance beyond minimum requirements.
Clear communication: The critical component
Unlike a vulnerability assessment, pen testing is a highly interactive process. That means the quality of communication is crucial to the overall value that will be realized.
In a vulnerability assessment, an automated tool scans the IT infrastructure to identify all systems, applications, and services that are running. The automated tool then attempts to identify issues and known exploits, which are listed in a standard report.
Pen testing, on the other hand, mimics a real-world attacker attempting to access sensitive systems and data. In addition to scanning systems and networks, ethical hackers also consider various mitigating and exacerbating factors (such as controls that were disabled beforehand), follow leads and additional unexpected opportunities, and apply social engineering skills to interact with the sponsoring organization’s personnel – just as a malicious hacker would do. Because of the highly variable and customized factors that define a pen test, clear communication of results and methods is crucial to an effective outcome.
The communication effort begins during the project scoping process, the first of the five phases described in the first post in this series. In addition to defining which areas of the organization are to be tested, what types of threats are of greatest concern, and what tactics and techniques the hackers will use, the initial scoping meetings also clarify who within the organization will be kept informed and how information will be delivered.
Reporting protocols will vary according to scope and purpose of the test. The reports that testers provide at the conclusion of a comprehensive, enterprisewide penetration effort will differ significantly from reports on a program that is more limited in scope. In some instances, tests are designed to emulate a particular type of attack that mirrors an actual event. Here again, the post-test communications will differ, based on the initial scoping decisions.
Collaborating with IT/IS
The involvement – or lack of involvement – by the target organization’s IT or IS department needs to be a deliberate decision with regard to penetration testing. In some instances, IT/IS is involved in the determination of scenarios and involved parties. In other scenarios, however, IT/IS cannot be informed when the aim of the test is to determine detection and response capability.
In such cases, the assessment team should engage IT/IS upon completion of the testing. The pen testing results can give IT/IS valuable additional information and help make the case for committing additional resources toward mitigating the risk. Pen testers might be able to identify solutions that are already within reach (or already paid for), drawing on their own industry experience and their newly acquired understanding of the organization’s existing systems and platforms.
Even when penetration testing is conducted as part of an audit plan, involving IT/IS as early as possible can help establish a positive attitude and collaborative mindset. Such an approach can help streamline the organization’s audit response, which inevitably will require significant IT/IS support.
The most successful engagements are those in which the target organization’s IT and IS professionals have an opportunity to collaborate with the testers to brainstorm possible solutions and share relevant experiences. Finally, it’s worth noting that such a collaborative approach helps to engage IT/IS personnel when management begins querying them on test results and remediation plans.
Meetings, debriefings, and reporting
In order to maximize the value of a pen testing project, all parties concerned should have a clear understanding of the reports, briefings, and recommendations the pen testing provider delivers, along with an equally clear understanding of the action plans and follow-up activities the target organization should take in response. When tests are conducted as part of a regulatory compliance effort, the basic reporting formats and content are specified by the relevant regulatory agency. Yet even within these parameters, some variations in approach can optimize the value.
Beyond the delivery of a final written report, the approaches that testing providers use to communicate with stakeholders can vary. In most instances, the assessment team will conduct at least one exit meeting with management and other stakeholders to highlight findings and preview the written report that will follow.
In our own experience, we have found considerable value in providing a separate technical debriefing to IT/IS in advance of the final exit meeting. This technical debriefing helps support the collaborative relationship suggested earlier.
In many instances, this debriefing can extend over a period of hours, as IT/IS and the ethical hackers work together to understand the issues and discuss their findings in considerable technical detail. The final exit meeting typically proceeds much more smoothly when IT has had adequate time to review the test results and begin researching their mitigation efforts.
In terms of the exit meeting itself, there is no single right way to conclude the engagement. In some cases, particularly when the test is a recurring activity and there are only a few new findings, the exit discussions can be brief and routine. In other cases, where the hacking team might have dozens of findings to discuss, it can be helpful to bring in other stakeholders who will need to support the mitigation efforts. In all cases, it is important that the technical details be available and that they be translated into clearly understood statements of risks and potential costs.
Once expectations have been set in the exit meeting, the assessment team begins work on the formal written report, which will include recommendations for addressing issues found. Typically, a first draft will be delivered, at which point management should provide responses and any additional documentation required by auditors or regulators.
The final report generally includes an executive summary (sometimes issued separately) that recaps the overall scope and methodologies used in the test, along with a general summary of the team’s findings. This summary is followed by a complete and detailed listing of specific test findings and an assessment of the relative risk level for each. Each finding also includes an explanation of the testers’ recommendations for remediating shortcomings and management’s action plan for addressing each recommendation. Finally, relevant technical details are attached as an appendix to the report.
To be effective, each action plan item should identify the individual responsible for carrying out the action and a deadline for completion. This reporting and action plan structure is typically mandatory for compliance-based audits, and it is recommended for all other types of penetration assessments, regardless of scope or purpose. Documenting the plan of attack to address each issue is a key exercise to maintain momentum and enact organizational changes after the engagement.
Success factors
Ultimately, the purpose of pen testing is to help an organization identify how to maintain and improve its cyber resiliency. Although ethical hacking requires specific technical skills, the most successful testing teams also possess several nontechnical skills and abilities. Cybersecurity expertise as well as business, audit, and general consulting experience are important pluses, particularly if the objective is to add genuine value rather than mere compliance.
The ability to provide consultative support in implementing recommendations can be particularly valuable, and it distinguishes the most successful penetration and risk assessment teams from more narrowly focused technical providers. A willingness and ability to discuss findings and recommendations is central to deriving genuine added value from the engagement.
Several critical success factors can help organizations get the most from their pen testing investments. They include:
- A clear definition of the testing plan from the outset of the project
- A clear understanding of the test results and relevant risk implications at the conclusion of the effort
- A willingness and ability to create well-defined action plans that designate responsible parties and timetables for completion
- A commitment to using the test reports to drive IT/IS and cybersecurity budgets and priorities
The value of pen testing
Pen testing is a highly effective tool for illustrating today’s cybersecurity risks. By highlighting vulnerabilities that otherwise would go unnoticed and exposing the associated risks to reveal their true potential costs, an effective penetration assessment can provide much needed insight. Moreover, by helping organizations develop workable plans for addressing risks, pen testing can contribute in significant ways to their overall cyber resiliency.
