Anatomy of a pen test

What really happens during penetration testing? Three Crowe professionals detail what to expect in the first of a two-part series.

Penetration testing has become a familiar term in the past few years, but it still generates considerable confusion. Penetration testing – referred to as pen testing or ethical hacking – is specifically required by regulatory agencies in various industries. In addition to being a compliance task, pen testing is a valuable cybersecurity tool. Misunderstandings about pen testing and the various ways in which it can be applied leave many executives wondering, “What should I expect to happen, and what is the value I should expect to derive?” This two-part series will help answer some of those questions. 

The process at a glance

Despite its widespread use, the term itself is often misunderstood. In many cases, those responsible for giving final approval to a pen testing plan are not exactly sure what they’re buying – or what to expect when the project begins. What’s more, they often do not have objective criteria for evaluating the completeness and effectiveness of the test, and they do not fully understand what to do with the results.

Recognized industry standards and various frameworks – such as the Payment Card Industry Data Security Standard – can provide a baseline description of basic pen testing principles and components. Nevertheless, those standards can be applied in countless variations. 

With thousands of pen test practitioners in the cybersecurity industry, the specific approaches and methods employed vary widely. In most cases, a pen testing engagement takes place over a span of one to two weeks, and it occurs in a series of phases. 

These phases can involve significant overlap, however, and the process described here does not necessarily proceed in a step-by-step fashion. In fact, the most effective pen testing practitioners are flexible, adaptable, and able to respond nimbly when a test reveals new information or when the organization’s security environment and threat profile change. 

With that caveat in mind, these are the major pen testing phases: 

1. Scope
2. Reconnaissance
3. Exploitation and lateral movement
4. Escalation 
5. Analysis

Scope

Defining scope is the first phase of any pen testing project, as all concerned seek to answer some basic questions such as: 

• What threats are of greatest concern? 
• What areas should be tested? 
• Is the testing for internal threats only or will it also explore the organization’s vulnerability to outside attackers? 
• Are cloud servers within scope? 
• If so, what is the cloud host’s policy on pen testing?

As these questions are answered, the conversations about scoping the assessment should consider the following:

• Cloud versus on-premises systems. In terms of pen testing in the cloud, the traditional lines between internal and external testing have blurred somewhat with the advent of cloud computing. Today, most cloud servers permit ethical hacking as long as the tests do not cause an actual loss of service. Organizations should determine whether any critical information or infrastructure is stored in the cloud and whether it should be subject to testing.

• Social engineering. Social engineering, which tests employees’ security awareness and compliance with good cybersecurity practices, is another critical area to be considered as part of the scoping process. This aspect of most pen tests involves ethical hackers, or pen testers, attempting to gain access to sensitive data using tactics such as phishing schemes or outright physical intrusion. Does the proposed testing regimen include social engineering tactics? If not, why not?

• Levels of cooperation. Pen testers, also known as the “red team,” should have a defined level of visibility and profile. Will other departments such as IT or IS (known as the “blue team”) be informed of the test? Clarity around the roles and preferred level of interaction between red teams and blue teams is essential before any actual testing can begin.

• Risk alignment and compliance. At this early stage, it is also important to be sure the proposed scope aligns with actual risks and cybersecurity concerns. If the organization hasn’t yet done so, it should perform a risk assessment to determine which areas and technologies are of concern, as well as which threat actors and threat vectors could be part of an attack. If these concerns do not coincide with the testers’ proposed scope or known strengths, now is the time to make adjustments. Based on the threat actors and vectors an organization perceives are most likely, more pointed testing should be discussed to try to gain access to particular targets or systems rather than simply general administrator access to everything.

  Even if the primary purpose of a pen test is regulatory compliance, the specific regulatory requirements about the scope of pen testing should be regarded as the bare minimum – the floor, not the ceiling. To maximize genuine value from the testing exercise, management should expect the testing team to question any out-of-scope areas. 

Reconnaissance

The tactics pen testers use during the initial reconnaissance of an organization can encompass a broad range of activities, from physical reconnaissance and social engineering ploys to extensive behind-the-scenes technical probing. The goal of reconnaissance is not only to identify potential avenues of attack but also to map out the overall structure and extent of the organization’s cybersecurity environment.

To identify exposure to external threats, pen testers often employ a variety of tactics, such as attempting access from public websites, executing phishing attempts and other online ploys, and scanning employee profiles and social media to detect usernames, likely passwords, and insider information about an organization. Resourceful, real-world attackers will use this knowledge to impersonate employees or vendors. In many instances, they succeed in gaining physical entry to secure facilities where they can then access secure portals or plant Raspberry Pis or other small-form-factor devices capable of breaching existing security measures. 

Given enough time, a determined attacker can usually find a way to gain access to a target’s internal networks. In pen testing assignments, the testers do not have the unlimited time that is available to a malicious attacker so they might eventually request the company grant them access in order to begin testing internal measures. More often than not, however, the testers gain access on their own. 

Once they have access, the testers then progress to a combination of both passive reconnaissance, such as exploring the company networks and monitoring online traffic, and active reconnaissance, which involves more advanced technical tools such as internet control message protocol or “ping” sweeps, port scans, and the use of dedicated software to map out the full scope of the company’s networks, directories, and servers. The goal is to identify shortcomings, weaknesses, missing patches, unsecured or outdated endpoints, and other vulnerabilities that can be exploited. 

Often, this reconnaissance will reveal previously unidentified or hidden systems that were not part of the initial scoping discussions. Such discoveries should prompt a reassessment of the project scope to avoid missing any critical vulnerabilities and avoid leaving management with a noncomprehensive view of the computing environment’s risks.

Exploitation and lateral movement

Generally, the transitions between pen testing phases are not precisely defined moments or events. Rather, they are fluid transitions. At some point during the reconnaissance phase, both real-life and ethical hackers will begin exploiting the avenues they find, exploring how one point of access can be used to gain access to others.

During this phase, individual experience and expertise tend to differentiate pen testers. The most successful pen testers take an opportunistic approach, chasing down leads, noting the relative sensitivity of various areas, and prioritizing those avenues that are the most critical due to the value of the data they expose. 

This phase includes a “human-hacking” component as well. For example, accomplished pen testers and attackers alike are adept at getting into users’ heads and guessing passwords by using a combination of experience and publicly available breach information. Likewise, this understanding of human patterns and behaviors helps in identifying users who likely use the same passwords on both their work accounts and their more vulnerable personal accounts. 

By exploiting such behaviors and other weaknesses such as the use of nonhardened personal devices for business purposes, testers expand their access laterally – from one device to another and from one department to another – to gain access to the most sensitive data in the environment. Whether taking advantage of humans making poor security choices or flaws such as missing patches, the actual exploitation of the vulnerabilities to demonstrate the risks firsthand is part of what makes a penetration test a unique and valuable instrument.

Escalation

Concurrent with their lateral exploitation, pen testers also work to gain higher-level authentication that will provide access to more critical controls and information. By exploiting the Microsoft™ Windows™ Active Directory environment commonly found in organizations, pen testers ultimately seek to gain privileged access to the domain controller, which is a database that holds a record of all the devices and users that are part of its domain, including users’ encrypted (hashed) passwords. Access to this controller provides the “keys to the kingdom,” enabling any hacker to identify and abuse other users and devices within the domain. 

Chasing down additional leads as they are discovered, pen testers continue learning more about the environment to form new attack paths. Their methodology involves working continually to gain additional access, particularly access to sensitive data, and usually with the ultimate goal of achieving administrator-level access, which opens up the whole environment to the testing team. Alternatively, it bears repeating that some advanced forms of penetration testing involve setting particular targets or goals at the outset, during scoping, and therefore the focus for escalation might be limited to only those systems or technologies.

Analysis

Once they have achieved the goals of the assessment, which most often include obtaining administrator-level access, pen testers usually have visibility into the entire domain and can begin a comprehensive, retrospective analysis, which involves a combination of “looking back” and “looking down.” Pen testers look back over the paths and techniques the team used to achieve its objective and look down from the highest level attained to get a big-picture view of the security of the entire enterprise.

As a supplement done at the end or beginning of an assessment, automatic vulnerability scans and other technical tools can help identify additional weaknesses or paths that might be exploited by an attacker while also serving as a double-check to verify the assessment team’s findings.

Examples of this retrospective analysis include performing a holistic password analysis once access is gained to centralized authentication servers. This analysis can result in programmatically identifying account password weaknesses and patterns that a knowledgeable attacker could have taken advantage of to achieve unauthorized access. Another such example is analyzing workstations for excessive user permissions, which would inform management of individual workstations that, if compromised, could afford an attacker inordinate levels of permissions to further compromise an environment. These analyses help pen testers determine what else could have gone wrong and enable them to follow potential worst-case scenarios to their ultimate conclusions.

The value of pen testing

To different members of the cybersecurity industry, the phrase “pen test” might have various meanings. Understanding the individual phases and goals of pen tests is important in forming appropriate expectations and appreciating the value they provide. 

For some, allowing pen testers into an environment might seem like a dangerous idea or, worse, just a check-the-box compliance exercise. But the benefits that an organization can reap outweigh any reservations that might be holding decision-makers back from investing in this critical exercise. Moreover, that investment and more thorough understanding can transform a standard or casual compliance-driven exercise into the vehicle that accelerates an organization’s cybersecurity maturity to the next level.

The second post of this series will address another question that executives often ask about pen testing: “What is the value I should expect to derive?” It will explore the types of reports, briefings, action plans, and follow-up activities an organization should expect to receive from its pen testing provider.

Microsoft, Windows, and Azure are either registered trademarks or trademarks of Microsoft Corp. in the United States and/or other countries.