Differences between Vulnerability Scans and Penetration Tests in PCI DSS

As part of regular security status monitoring activities, the PCI DSS standard requires a series of technical assessments to identify potential security issues in compliance and compliance assets at an early stage. security controls deployed. These activities include: vulnerability scans and penetration tests.

Even though each of these tests has a different scope and objectives, They are often confused with each other. To avoid these problems, the differences between each of them are explained below:

Vulnerability scans (req. 11.3)

Vulnerability Scans (vulnerability scans) Internal and external requirements in PCI DSS requirement 11.3 have the following characteristics:

• • They are non-intrusive testing (does not affect confidentiality, integrity or availability of the services analysed).
  • Vulnerability scans are clearly technicians They are aimed at identifying and detecting potential problems in the logical infrastructure of the environment by comparing them with databases of known and reported vulnerabilities.
  • They usually have a high rate of false positives (a vulnerability detected by the scanning software may not really exist), so a detailed review of the reports is required.
  • Verifications are carried out on a formal basis automated.
  • Your runtime is relatively short.
  • They must be carried out on a quarterly basis or after a significant change.
  • In the case of external vulnerability scans, these must be carried out exclusively by companies approved by the PCI SSC as «Approved Scanning Vendors» (ASV).
  • For the Internal Vulnerability Scans, these can be carried out by internal personnel of the company with proven knowledge of security and / or external companies.
  • Likewise, the internal scans must be authenticated.

Some of the tools used to perform vulnerability scans are: Tenable Nessus (commercial), OpenVAS (open source), Qualys (commercial), SAINT (commercial), Nexpose (commercial), etc.

Penetration tests (req. 11.4)

On the other hand, the penetration tests (penetration testing) Internal, external and segmentation verification required by PCI DSS requirement 11.4 have the following characteristics:

• Vulnerability scan results are often used as a starting point to optimize runtimes, although this is not mandatory.
• They are intrusive testing oriented towards the attempted exploitation or direct exploitation of the identified vulnerabilities, simulating the effects of a real attack. Therefore, they can affect the integrity, confidentiality and availability of the assets analyzed so they must be carefully planned so as not to affect the operation.
• Penetration testing goes beyond the technical (logical) environment and may include performing physical evidence such as social engineering or physical access to facilities.
• Requires one pre-preparation to outline the attacks and tools to be used, as well as the definition of a detailed scope, including a "stop point" for when a vulnerability is exploited and allows access to other environments not included in the original scope (pivot).
• Its implementation should be based on a methodology previously defined by the organization (req. 11.4). If this task is delegated to a third party, that third party must provide the information of its methodology as part of this activity.
• Tests executed are usually almost always manuals and specifics environment assets, so their runtime is longer than a vulnerability scan.
• Their reports usually have less false positives Vulnerability scans, because attempts are made to exploit the identified vulnerabilities by verifying the actual existence of the security problem.
• Their reports are more comprehensive and in greater detail than reports from vulnerability scans.
• They must be executed in a annual (for internal, external and segmentation tests) and half-yearly (for segmentation tests for service providers) or after a significant change.
• These tests can be carried out by internal personnel with security knowledge and/or by specialized external companies.
• Depending on the previous knowledge of the internal structure, the design or the implementation of the environment to be evaluated that the professional who executes the test has, this can be cataloged in tests of black box (without prior knowledge), White Box (with detailed information on the environment) or grey box (partial knowledge of the environment). This knowledge can impact test run times (less knowledge of the environment requires more time to run).

Some of the tools used for these tests are: Metasploit (open source/commercial), Core Impact y Cobalt Strike (commercial), Social-Engineer Toolkit (open source), Pentera (commercial), etc. which, generally, are not used individually but are combined with other tools, depending on the environment to be analyzed.

More information in the PCI SSC documentInformation Supplement: Penetration Testing Guidance‘.

Table of differences between a vulnerability scan and a penetration test

Based on the above, the differences between these technical tests can be summarized in the following table:

QSA recommendations

Taking into account the differences between these two types of tests, it is very important that client entities manage their expectations regarding the deliverables of each test to avoid surprises during the formal evaluation:

• It is recommended to validate the suitability, knowledge and experience of the personnel in charge of the tests before their execution.  If possible, add this information in both scan and penetration test reports.
• It is recommended that a list of the tools used during the test be added.
• In penetration testing, it is recommended to request evidence of exploitation attempts (screenshots, videos, images, etc.), as well as the detail of how that conclusion was reached. The intention is that this activity is replicable to check it in a real or controlled environment.
• The mere execution of an automated tool cannot be categorized as a valid deliverable for a penetration test. It is required that there is always a manual process performed by a competent professional.
• Finally, the deliverable of these tests must not only describe what has been found, but must also provide the entity with mitigation and correction alternatives and, in the event that the "standard" corrections cannot be made, list potential compensatory checks, always from a risk management perspective.