Who This Checklist Is For
This article is written for CPAs, auditors and compliance professionals who review penetration test reports as part of SOC 2 engagements. If you are the person deciding whether a pentest report is sufficient evidence for Trust Services Criteria CC6.1, CC7.1 and CC7.2, this checklist is for you.
You do not need to be a security expert to use this checklist. You need to know what to look for, what questions to ask and when a report is not good enough.
The 7-Point Pentest Report Checklist
When you receive a penetration test report from your client's testing firm, verify that it contains all seven of the following elements. If any are missing, the report may not provide sufficient evidence for your SOC 2 examination.
1. Clearly Defined Scope
The report must specify exactly what was tested. Look for:
- IP addresses and/or domains in scope
- Internal vs external network boundaries
- Web applications and APIs tested
- Cloud environments included (AWS, Azure, GCP)
- Date range of testing
- Any exclusions or limitations
Why it matters: If the scope does not match the service organization's system description in the SOC 2 report, the pentest may not cover the systems your examination addresses. A pentest that only covers a marketing website when the service organization processes customer data through a separate application provides no useful evidence.
2. Documented Methodology
The report should reference a recognized testing methodology. Acceptable methodologies include:
- PTES (Penetration Testing Execution Standard)
- OWASP Testing Guide
- NIST SP 800-115
- CREST penetration testing standards
Why it matters: A documented methodology demonstrates that the tester followed a structured, repeatable process rather than running a few automated tools. It also ensures coverage of the vulnerability categories relevant to SOC 2 criteria.
3. Authenticated and Unauthenticated Testing
The report should document that both authenticated (with valid credentials) and unauthenticated (without credentials) testing was performed. Look for:
- External testing from an internet-based perspective
- Authenticated testing with standard user credentials
- Privilege escalation testing from standard to admin roles
- Role-based access control validation
Why it matters: CC6.1 addresses logical access controls. Testing only from an unauthenticated external perspective misses authorization flaws, privilege escalation paths and role-based access control weaknesses that represent significant risk to the service organization's system.
4. CVSS-Scored Findings
Every finding should include a severity rating using the Common Vulnerability Scoring System (CVSS). Verify that:
- Each finding has a CVSS v3.0 or v3.1 base score
- Findings are categorized as Critical, High, Medium, Low or Informational
- The report includes proof-of-concept evidence (screenshots, request/response data)
- Findings are mapped to the Trust Services Criteria they affect (CC6.1, CC7.1, CC7.2)
Why it matters: Standardized scoring allows you to assess severity consistently. Mapping to Trust Services Criteria connects findings directly to the criteria in your examination. Without this mapping, you must interpret each finding's relevance yourself, which introduces subjectivity and increases risk of missing material issues.
5. Remediation Status
The report should document the current remediation status of each finding:
- Open (not yet remediated)
- In progress (remediation underway)
- Remediated (fix applied, pending retest)
- Verified (fix confirmed through retest)
- Accepted risk (risk formally accepted by management)
Why it matters: For Type II examinations, you need to demonstrate that the organization not only identifies vulnerabilities but responds to them. A report showing 15 critical findings with no remediation activity is a material concern. Remediation status tracking shows that the vulnerability management process is functioning.
6. Retest Evidence
For critical and high-severity findings, the report should include retest evidence confirming that remediation was effective. Look for:
- Retest date (separate from the original test date)
- Evidence that the original vulnerability is no longer exploitable
- Confirmation that the fix did not introduce new vulnerabilities
Why it matters: A finding marked "remediated" without retest evidence is an unverified claim. Retesting confirms that fixes are effective and that the vulnerability management lifecycle is complete. This is particularly important for CC7.2, which addresses monitoring and response to anomalies.
7. Attestation Letter
The testing firm should provide a signed attestation letter that includes:
- Testing firm name and contact information
- Tester qualifications and certifications
- Scope of the engagement
- Methodology used
- Testing dates
- Summary of results (number of findings by severity)
- Confirmation that the test was independent of the service organization's development team
Why it matters: The attestation letter is the formal document that connects the pentest report to your SOC 2 examination. It confirms that a qualified, independent party performed the testing. Without it, you are relying on an unsigned technical document with no verification of who produced it or their qualifications.
Red Flags in Bad Pentest Reports
Over years of working with auditors and reviewing pentest reports from other firms, we have identified the most common indicators of a low-quality assessment. If you see any of these, request a new report or a new testing firm.
Automated Scanner Output Disguised as a Pentest
The most common problem. The report is a direct export from Nessus, Qualys or OpenVAS with a cover page added. These reports list hundreds of "findings" that are mostly informational or false positives. There is no manual testing, no exploitation evidence and no business context. A vulnerability scan is not a penetration test. If the report reads like a tool output, it is a tool output.
No Scope Definition
The report jumps straight into findings without documenting what was tested. You cannot determine whether the testing covered the relevant systems. This is the equivalent of receiving an audit report that never identifies the entity being audited.
Missing Methodology
The report does not reference any testing standard or methodology. The tester may have been competent, but without documented methodology, you cannot verify that testing was systematic or complete.
No CVSS Scores
Findings labeled as "High" or "Critical" without standardized scoring. Without CVSS, severity ratings are subjective and inconsistent. One firm's "High" may be another firm's "Medium."
Two to Three Pages Long
A legitimate penetration test report for a SOC 2 engagement should be 20 to 60+ pages depending on scope. A 2-page report indicates minimal testing effort. Even a clean environment with no findings should produce a substantial report documenting scope, methodology, testing activities and negative results.
No Attestation Letter
The testing firm provides a report but no formal attestation. This leaves you with no documented confirmation of tester qualifications, independence or engagement terms.
Test Date Outside the Review Period
For Type II engagements, the pentest must have been performed during the observation period. A test performed six months before the review period started does not demonstrate that controls were effective during the period under examination.
Refer Your Clients to Sherlock Forensics
If your client needs a penetration test that will satisfy your SOC 2 examination requirements, refer them to Sherlock Forensics. Our SOC 2 pentest reports are structured specifically for auditor review and include every element on this checklist.
We work directly with auditors to ensure our report format and content meet your specific requirements. If you need adjustments to the report template, we accommodate those at no additional cost.
- Standard SOC 2 Pentest: $5,000 CAD - external network and web application testing with full deliverables
- Comprehensive SOC 2 Pentest: $12,000 CAD - adds internal network testing, cloud configuration review and retest of critical findings
Both tiers include the findings report, executive summary, remediation roadmap and attestation letter. View our SOC 2 penetration testing service page for full details, or contact us at 604.229.1994.
Frequently Asked Questions
What should a CPA look for in a SOC 2 penetration test report?
Verify seven elements: clearly defined scope matching the service organization's environment, documented methodology based on PTES or OWASP, both authenticated and unauthenticated testing, CVSS-scored findings mapped to Trust Services Criteria, current remediation status for each finding, retest evidence confirming fixes and a signed attestation letter. Sherlock Forensics SOC 2 pentest reports include all seven elements.
What are red flags in a SOC 2 penetration test report?
Red flags include automated scanner output with no manual testing, missing scope definitions, no methodology documentation, findings without CVSS scores, no mapping to Trust Services Criteria, missing attestation of tester qualifications, reports only 2 to 3 pages long and no evidence of authenticated testing. Any of these indicates a report that may not provide sufficient evidence for your examination.
How can CPAs refer clients for SOC 2 penetration testing?
Refer your clients to Sherlock Forensics for SOC 2 penetration testing. We work directly with auditors to ensure report format and content meet their requirements. Standard SOC 2 pentests start at $5,000 CAD. Call 604.229.1994 or visit our SOC 2 pentest page.