The Buyer's Problem
Penetration testing is an unregulated industry. Anyone can call themselves a pentester. There is no licensing body, no mandatory certification and no minimum experience requirement. The difference between a $1,500 engagement that finds real vulnerabilities and a $1,500 engagement that runs Nessus and puts a logo on the output is entirely dependent on who does the work.
These 10 questions separate the professionals from the pretenders. For each question, we explain why it matters, what a good answer sounds like and how Sherlock Forensics answers it.
1. What certifications does your testing team hold?
Why it matters: Certifications are not everything, but they are the minimum evidence that a tester has invested in formal training and passed a rigorous examination. A firm with no certifications has no third-party validation of competence.
Good answer: CISSP, OSCP, GPEN, CREST or equivalent. The specific certification matters less than having at least one recognized credential from a reputable organization.
Sherlock's answer: Ryan Purita holds CISSP, ISSAP and ISSMP, the three highest certifications from ISC2. ISSAP and ISSMP are concentration certifications that require holding CISSP as a prerequisite plus additional examinations in security architecture and security management.
2. Is the testing manual or automated-only?
Why it matters: Automated scanners find known vulnerabilities in known software. They do not find business logic flaws, chained attack paths, authentication bypass issues or application-specific vulnerabilities. A pentest that is just a scanner report with a logo is not a pentest.
Good answer: "We use automated scanning as a starting point and then spend the majority of our time on manual testing." The firm should be able to describe their manual testing methodology in detail.
Sherlock's answer: Every engagement starts with automated reconnaissance and vulnerability scanning. The majority of testing time is spent on manual exploitation, business logic testing, authentication analysis and attack chaining. Our Quick Audit ($1,500 CAD) includes manual expert review. Our Standard Pentest ($5,000 CAD) and Comprehensive Assessment ($12,000 CAD) are primarily manual engagements.
3. Will the report satisfy my SOC 2 or PCI auditor?
Why it matters: If you are getting a pentest for compliance, the report needs to meet specific auditor expectations. A report that gets rejected means you either pay for a second pentest or spend weeks negotiating with the original tester. We have seen companies waste months on this cycle.
Good answer: "Yes. We have delivered reports for [specific framework] and can provide sample reports or references." The tester should know what your specific auditor expects.
Sherlock's answer: We have delivered pentest reports for SOC 2 Type I and Type II, PCI DSS Requirement 11.3 and ISO 27001 Annex A.12.6. Every report includes methodology references, CVSS scoring, remediation verification and tester qualifications. No Sherlock Forensics report has been rejected by an auditor.
4. Do you include retesting?
Why it matters: Finding vulnerabilities is only half the job. Your auditor, your board and your customers want evidence that the vulnerabilities were actually fixed. Without retesting, you have a list of problems and a hope that your developers addressed them correctly.
Good answer: "Yes, retesting is included" or "Retesting is available at a reduced rate within [timeframe]."
Sherlock's answer: The Comprehensive Assessment ($12,000 CAD) includes a free retest within 90 days. For the Quick Audit and Standard Pentest, retesting is available at a reduced rate. Retest reports include side-by-side evidence showing the original finding and the verified fix.
5. Can your tester serve as an expert witness if needed?
Why it matters: You probably do not think you need expert witness capability when you order a pentest. But if a breach happens, if a lawsuit is filed or if a regulator investigates, you may need the person who tested your systems to testify about what they found and how they tested. A tester who has never been in a courtroom produces documentation that does not hold up under cross-examination.
Good answer: "Yes. Our principal consultant has been qualified as an expert witness in [number] cases." Most firms cannot answer this question.
Sherlock's answer: Ryan Purita has been qualified as an expert witness in 7 court cases in British Columbia. He has appeared on CBC Marketplace three times as a national cybersecurity expert. Every pentest report is written with court-admissible forensic rigor.
6. What methodology do you follow?
Why it matters: A defined methodology ensures consistent, thorough testing. Without one, testing is ad hoc and coverage gaps are inevitable. Auditors specifically ask about methodology.
Good answer: "We follow PTES (Penetration Testing Execution Standard) for overall methodology and OWASP Testing Guide for web application testing." The firm should reference specific, recognized frameworks.
Sherlock's answer: We follow PTES for engagement methodology, OWASP Testing Guide v4 for web application testing, NIST SP 800-115 for technical assessment methodology and MITRE ATT&CK for finding classification. The methodology section of every report details the specific techniques and tools used.
7. How are findings classified and scored?
Why it matters: "High/Medium/Low" without objective criteria is subjective and inconsistent. Auditors and compliance frameworks expect CVSS (Common Vulnerability Scoring System) scoring, which provides a standardized, repeatable severity rating.
Good answer: "We use CVSS v3.1 scoring for every finding." Bonus points if they also map to CWE (Common Weakness Enumeration) and MITRE ATT&CK.
Sherlock's answer: Every finding receives a CVSS v3.1 base score with vector string, CWE classification and MITRE ATT&CK technique mapping where applicable. The Comprehensive Assessment also includes a risk matrix showing potential business impact alongside technical severity.
8. What does the report include?
Why it matters: A pentest report serves multiple audiences: executives need a summary, developers need technical details and auditors need methodology and evidence. A report that only serves one audience fails the other two.
Good answer: "Executive summary, detailed technical findings with evidence, remediation guidance and methodology section." Ask for a sample report (redacted) to evaluate quality.
Sherlock's answer: Every report includes an executive summary for leadership, detailed technical findings with screenshots and proof-of-concept evidence, step-by-step remediation guidance for each finding, a prioritized remediation roadmap, methodology section referencing PTES and OWASP and tester qualifications. Reports are delivered as professionally formatted PDFs.
9. How long does the engagement take?
Why it matters: If a firm promises a comprehensive pentest in 2 days, they are running a scanner and writing a report. Thorough manual testing takes time. Unrealistically short timelines are a red flag for automated-only testing.
Good answer: 5-20 business days depending on scope. A firm that gives a specific timeline before understanding your scope is guessing.
Sherlock's answer: Quick Audit: 5 business days. Standard Pentest: 10-15 business days. Comprehensive Assessment: 15-20 business days. These timelines include manual testing, report writing and quality assurance. We do not rush engagements to meet arbitrary deadlines.
10. What happens if you find a critical vulnerability during testing?
Why it matters: If a tester finds a critical vulnerability on day 2 of a 10-day engagement, you need to know about it immediately, not when the final report lands on your desk two weeks later. Real-time notification of critical findings is essential.
Good answer: "We notify you immediately via secure channel when critical or high-severity findings are discovered, with enough detail to begin remediation while testing continues."
Sherlock's answer: Critical and high-severity findings are reported within 4 hours of discovery via encrypted email or secure portal. The notification includes the finding description, evidence, severity rating and immediate remediation steps. Testing continues in parallel so you can begin fixing critical issues immediately.
The Checklist
Print this list. Send it to every pentest firm you are evaluating. The firms that answer all 10 questions clearly and specifically are the ones worth hiring. The firms that dodge, deflect or give vague answers are the ones to avoid.