Pentest Report Red Flags: What to Look For

You Paid for a Pentest. Did You Get One?

You invested $5,000 to $25,000 in a penetration test. A PDF lands in your inbox. It is 40 pages long. It has charts. It has severity ratings. It looks professional. But is it actually useful? Did you get a real penetration test, or did someone run Nessus, export the results and call it a day?

This guide helps CTOs and CISOs evaluate pentest reports. We cover the red flags that indicate a low-quality engagement, the green flags that signal thorough work and what you should demand from your security vendor.

Red Flag 1: Pure Scanner Output

The most common red flag. The report is clearly automated vulnerability scanner output with minimal human analysis. You can identify this by:

• Findings that reference CVE numbers and patch levels but no application-specific testing
• Descriptions copied verbatim from vulnerability databases
• No screenshots of actual exploitation, only "this port is open" or "this header is missing"
• Remediation advice that says "apply the latest vendor patch" for every finding
• A long list of informational items (SSL cipher preferences, HTTP headers, server version strings) with few or no high-severity findings

If your report looks like it could have been generated without a human ever visiting your application, you received a vulnerability scan, not a penetration test. These are useful but cost a fraction of a pentest and should be priced accordingly.

Red Flag 2: No Evidence

A finding without evidence is an opinion, not a finding. Every vulnerability in a pentest report should include proof that it exists and proof that it is exploitable:

• Screenshots showing the vulnerability in action
• HTTP request/response pairs demonstrating the exploit
• Step-by-step reproduction instructions that your development team can follow
• Sample data extracted (sanitized) to demonstrate impact

If the report says "SQL injection found in search parameter" with no screenshot, no request/response data and no evidence of what data could be extracted, it is possible the tester flagged a potential issue without actually confirming it. That matters because false positives waste developer time and erode trust in the report.

Red Flag 3: Generic Remediation

Good remediation advice is specific to your technology stack and architecture. Bad remediation advice is generic enough to apply to any application.

Bad: "Implement input validation to prevent SQL injection."

Good: "The search endpoint at /api/v1/products?q= is vulnerable to SQL injection through the q parameter. Replace the string concatenation in ProductController.search() with a parameterized query using your ORM's built-in query builder. In Django, change Product.objects.raw('SELECT * FROM products WHERE name LIKE %' + query + '%') to Product.objects.filter(name__icontains=query)."

If remediation advice does not reference your specific codebase, endpoints or technology, it was likely copied from a template. Demand better.

Red Flag 4: No Executive Summary

A pentest report serves two audiences: technical staff who will fix the issues and executives who need to understand the risk. A report without an executive summary fails the second audience entirely.

The executive summary should be one to two pages and include:

• Overall risk rating with a brief justification
• Number of findings by severity (Critical, High, Medium, Low, Informational)
• The top three risks explained in business language (not technical jargon)
• A one-paragraph narrative of the most significant attack path discovered
• A recommendation on remediation priority

If you cannot hand the executive summary to your CEO or board and have them understand the risk, the report failed.

Red Flag 5: Everything Is the Same Severity

If every finding is rated "Medium" or every finding is rated "High," the tester did not properly assess risk. A realistic report has a distribution of severities. Most applications have a few critical and high-severity findings, several medium-severity issues and many low-severity or informational items.

Findings should use a standard scoring system like CVSS (Common Vulnerability Scoring System) that accounts for exploitability, impact, complexity and scope. A missing X-Frame-Options header is not the same severity as a SQL injection that dumps your database. If the report treats them equally, the tester did not understand the difference.

Red Flag 6: No Business Logic Testing

If the report only covers technical vulnerabilities (injection, XSS, misconfigurations) but includes no business logic findings, the tester probably did not understand your application deeply enough to test it properly.

Business logic vulnerabilities include:

• Manipulating prices, quantities or discount codes in e-commerce flows
• Skipping steps in multi-step processes (payment, verification, onboarding)
• Accessing features outside your subscription tier
• Bypassing rate limits on high-value operations
• Exploiting race conditions in financial transactions

These are the vulnerabilities that cause the most business damage and they are the ones automated scanners cannot find. A report with no business logic findings either tested an extremely simple application or did not test deeply enough.

Red Flag 7: No Methodology Section

A professional report documents the testing methodology: what standards were followed (OWASP, PTES, NIST), what tools were used, what testing techniques were applied and what the testing timeline was. Without this, there is no way to evaluate whether the testing was thorough or whether important areas were skipped.

The methodology section is also critical for compliance. SOC 2 auditors, PCI QSAs and ISO 27001 assessors need to see that the pentest followed a recognized framework. A report without methodology documentation may not satisfy your auditor.

Green Flags: What Good Reports Look Like

Now that you know what to watch for, here is what a quality report includes:

• Clear executive summary that non-technical stakeholders can understand
• Scope documentation listing exactly what was tested and what was excluded
• Methodology section referencing industry standards and testing tools
• Findings with evidence including screenshots, request/response data and reproduction steps for every vulnerability
• CVSS scores with a breakdown of the scoring factors for each finding
• Technology-specific remediation referencing your actual code, endpoints and framework
• Business impact analysis explaining what an attacker could achieve by exploiting each vulnerability
• Risk matrix plotting findings by severity and exploitability
• Prioritized remediation roadmap with "fix immediately," "fix within 30 days" and "fix within 90 days" categories
• Positive findings noting security controls that were tested and found effective
• Debrief call to walk through findings, answer questions and discuss remediation strategy

Questions to Ask Before the Engagement

The best way to avoid a bad report is to set expectations before testing begins:

• "Can I see a sample report?" (Any reputable firm will have a sanitized sample)
• "What methodology do you follow?"
• "Will findings include screenshots and reproduction steps?"
• "Will remediation advice be specific to our technology stack?"
• "Is a debrief call included?"
• "How many hours of manual testing are included?"
• "Will you test for business logic vulnerabilities?"
• "Does the report satisfy SOC 2/PCI/ISO requirements?" (if applicable)

What Sherlock Forensics Reports Include

Every Sherlock Forensics penetration test report includes all of the green flags listed above. We produce reports that serve both technical and executive audiences, with evidence for every finding, technology-specific remediation and a live debrief call. Our reports are accepted by SOC 2, PCI DSS and ISO 27001 auditors.

Standard external pentests start at $5,000 CAD. Quick audits start at $1,500 CAD. Every engagement includes the full report and debrief. Learn what to expect or order online.

Frequently Asked Questions

How do I read a pentest report?

Start with the executive summary for the overall risk picture. Then review findings by severity, starting with Critical and High. For each finding, read the description, examine evidence, understand the business impact and review remediation steps. The debrief call included with every Sherlock Forensics engagement walks you through findings in person.

What should a pentest report include?

A quality report includes an executive summary, scope and methodology documentation, findings with CVSS severity ratings, proof-of-concept evidence, business impact analysis, technology-specific remediation guidance, a risk matrix and a prioritized remediation roadmap. Sherlock Forensics reports include all of these plus a walkthrough debrief call.

What are red flags in a pentest report?

Red flags include automated scanner output with no manual validation, findings without evidence or screenshots, generic remediation advice, no executive summary, flat severity ratings (everything the same level), no business logic testing, no methodology section and no debrief. These indicate you received a vulnerability scan rather than a penetration test.