the SOC 2 report doesn't mean what you think

A CISO at a mid-market fintech was evaluating vendors in late 2025. The short list included two SaaS tools that would process customer financial data. Both vendors provided SOC 2 Type II reports as part of their security due diligence package. Both reports had clean opinions from reputable audit firms. Both reports covered the trust services criteria for security, availability, and confidentiality.

The CISO’s procurement team recommended going with either vendor, citing the SOC 2 reports as sufficient evidence of security maturity. The CISO asked for the full reports, not just the summary, for both vendors.

The full reports, each running about 80 pages, told a more nuanced story.

The first vendor’s report included a detailed description of controls, testing procedures, and any exceptions noted. The control description was specific: “The Company performs quarterly user access reviews of production systems using a documented procedure, output from which is retained for audit.” The testing procedures were specific: “Auditor selected a sample of 25 terminations and verified that access was removed within the documented timeframe.”

The second vendor’s report also had a clean opinion, but the control descriptions were vaguer: “The Company maintains procedures for user access management.” The testing referenced “inquiry of management” and “inspection of policy documents” more often than it referenced “inspection of system-generated evidence.” The controls were in place, formally. The operation of the controls in practice was less verified.

Both vendors passed their SOC 2. Both had clean reports. The actual security postures behind the reports were different by a margin that the executive summaries did not reveal.

This essay is about that gap. Between what a SOC 2 report certifies and what it’s often assumed to certify, there is a space where a lot of security marketing lives. Understanding the gap is essential for anyone making security decisions based on compliance artifacts.

a SOC 2 audits controls, and says less about security than you think

The literal meaning of a SOC 2 Type II report is that an independent auditor examined a set of controls at a company, tested those controls against the Trust Services Criteria for a specific audit period, and issued an opinion on whether the controls were designed and operating effectively during that period.

The key phrases are “a set of controls,” “against the Trust Services Criteria,” and “during that period.” Each is a constraint that often gets lost in marketing translation.

“A set of controls”, the controls in scope are defined by the company being audited, not by a standard rubric. Two companies can both pass SOC 2 with wildly different control sets. One company might have comprehensive controls across every system; another might have minimal controls on the specific systems in scope, with everything else excluded. Both can be accurate SOC 2 reports.

“Against the Trust Services Criteria”, the TSC is a framework of objectives, not a prescription of specific controls. The company chooses how to meet each criterion. The auditor evaluates whether the chosen approach, as implemented, meets the objective. Different approaches can all meet the same objective.

“During that period”, the audit period is typically six to twelve months. The report certifies the controls during that period. Anything that happened before or after is not in scope. A company can have a clean SOC 2 covering January through June and a massive security incident in July that doesn’t appear in the report.

A SOC 2 report is, correctly read, a structured attestation that a specific set of controls was operating in a specific way during a specific period. It is not a broad certification that the company is secure.

what a SOC 2 report tells you

it tells you

about the controls in scope

• Which controls the company chose to have audited
• Whether they existed, or worked over a period (Type II)
• Which exceptions the auditor found

it doesn't tell you

about the rest

• What was left out of scope
• How the engineering team really operates
• Whether the company is secure

scope limitations are everywhere

The specific aspect most buyers miss is the scope limitation. Every SOC 2 report has a scope section that defines what’s in and what’s out. The in-scope systems, personnel, and processes are what the auditor examined. Anything else is not covered.

A common pattern: the SaaS product that the customer is buying is in scope for the SOC 2, but the corporate infrastructure of the vendor, their internal IT, their development environment, their employee endpoints, is out of scope. The audit certifies the production system where customer data is processed. It says nothing about how that system gets built.

The supply chain is typically out of scope. The vendor’s own vendors, the cloud infrastructure provider, the third-party libraries, the monitoring tools, are not covered by the vendor’s SOC 2. They might have their own SOC 2 reports, but those are separate audits of separate entities.

The employees’ personal devices, if they’re used for work, are often out of scope. The vendor might have a BYOD policy, but SOC 2 rarely covers the specifics of how that policy is enforced.

Reading the scope section is mandatory for interpreting the report. The scope defines what the opinion applies to. An opinion that applies to a narrow scope tells you very little about the parts that weren’t examined.

the exceptions section is where the reality lives

In a SOC 2 Type II report, the description of testing results includes a section on exceptions. An exception is a case where the auditor’s testing found a control operating differently from how it was designed, or not operating at all in some instances.

Exceptions are not automatic failures. A report with exceptions can still have a clean opinion, depending on the nature and significance of the exceptions. A report with no exceptions looks better on paper, but a report with disclosed exceptions is often more honest, it shows that the auditor actually tested, found issues, and the company disclosed them.

Reading the exceptions section tells you where the company’s controls are actually breaking down. A company with zero exceptions in a twelve-month audit either has exceptional controls or is working with an auditor who isn’t testing deeply. The latter is more common than the former.

Specific patterns in the exceptions section reveal control maturity. Exceptions in user access management (terminations not removed in time) indicate weakness in identity hygiene. Exceptions in change management (changes not approved per policy) indicate weakness in release discipline. Exceptions in security monitoring (alerts not reviewed) indicate weakness in operations.

A report with exceptions across multiple TSC categories is telling you that the company’s control operation is patchy. A report with localized exceptions in one area is telling you where to focus your follow-up diligence.

Many buyers don’t read the exceptions section. They look at the opinion (clean), flip through the control descriptions, and stop. This is analogous to reading only the headlines of an audit, the headline tells you the summary, the body tells you the substance.

the difference between Type I and Type II

A SOC 2 Type I report is a point-in-time examination. It covers whether controls were designed appropriately as of a specific date. It does not cover whether the controls operated effectively over time.

A SOC 2 Type II report is a period-of-time examination. It covers whether controls were designed appropriately and operated effectively over a specified period, typically six to twelve months.

The distinction matters. A vendor with only a Type I report has not demonstrated sustained operation of their controls. They have demonstrated that the controls exist. A vendor with a Type II report has demonstrated operation over time.

Type I reports are easier to obtain and are often the first SOC 2 a new company gets. For a company in its first year of security maturity, a Type I is a meaningful milestone. For mature vendors, Type II is the expectation.

If a mature vendor presents a Type I report instead of Type II, that’s a signal. Either the company recently failed a Type II and is regrouping, or the company has chosen to limit the depth of the audit. Either interpretation warrants follow-up.

the audit firm matters

SOC 2 audits are performed by accounting firms (technically by Certified Public Accountants in the US). The quality of audits varies across firms.

Top firms, the Big Four, specialized security auditors, reputable mid-size firms, do thorough audits with substantive testing. Smaller firms vary more widely. Some smaller firms are very good. Some are essentially rubber stamps, providing the report that the client needs for the cost-effective price.

Identifying the rubber-stamp pattern from a report isn’t foolproof, but there are signals. Vague control descriptions. Heavy reliance on inquiry and inspection rather than re-performance and recalculation. Short reports (a thorough SOC 2 Type II is usually 60-100 pages). Absence of disclosed exceptions. A sampling approach that tests small numbers of items.

When evaluating a vendor’s SOC 2 report, noting the audit firm is part of the assessment. A report from a known rigorous firm carries more weight than a report from a firm you’ve never heard of.

the compliance team vs. the engineering team

A specific disconnect within vendor organizations: the SOC 2 report is often produced by a compliance function that is adjacent to, but not integrated with, the engineering function.

The compliance team writes the controls documentation. They work with the auditor. They manage the evidence collection. They handle the report.

The engineering team builds and operates the product. They implement the controls in practice, but their primary focus is on product development, reliability, and customer-facing features.

In a mature organization, the two teams coordinate. The controls documented by compliance reflect what engineering actually does. The evidence provided to auditors is generated by engineering’s day-to-day operations. The SOC 2 is a description of operational reality.

In a less mature organization, the two teams don’t fully align. Compliance documents aspirational controls. Engineering operates approximately but not precisely. The evidence collection is done specifically for the audit, often through a parallel process that doesn’t reflect production operations. The SOC 2 describes the audit workflow, not the operational workflow.

For a buyer evaluating a vendor, the question is which type of organization you’re dealing with. This is hard to determine from the report alone. Useful follow-up questions include: is the control evidence generated automatically from production systems, or manually collected for the audit? Do the engineering leaders understand the controls documented in the SOC 2? Are the controls integrated into the product’s day-to-day operations?

Vendors that are compliance-native can usually answer these questions directly. Vendors where compliance is bolted on have answers that evade specifics.

the penetration test and the SOC 2

Penetration tests and SOC 2 audits are often conflated in vendor security marketing. They are different things.

A penetration test is an offensive security exercise. Ethical hackers attempt to compromise the target system. The output is a report of vulnerabilities found and recommendations for remediation.

A SOC 2 audit is a controls examination. It does not involve offensive testing. It does not produce a vulnerability assessment.

A vendor that has passed SOC 2 may or may not have ever been penetration tested. Penetration testing may be one of the controls examined in the SOC 2 (is there a process for regular pen tests?), but the existence of the process is different from the findings of the most recent test.

When evaluating a vendor’s security, asking specifically for a penetration test report is a useful addition to the SOC 2 review. The two reports cover different aspects. The SOC 2 tells you about process maturity. The pen test tells you about technical vulnerabilities.

Vendors who are secure will typically have both, and will be willing to share both under NDA. Vendors who use SOC 2 as their security story and have no pen test are providing a partial picture.

the specific questions to ask

Here are the specific questions worth asking a vendor whose SOC 2 you’re evaluating.

What is the full scope of the SOC 2? Which systems, personnel, and processes are included? What’s excluded?

Can I have the full report, not just the summary?

What exceptions were noted in the testing? How were they addressed?

When does the current audit period end? When will the next report be available?

Who is the audit firm? How long has the company worked with them?

Is the control evidence generated automatically or manually for the audit?

When was the last penetration test? Can I see the summary findings?

How are production system changes approved and deployed? Can you describe the release process?

How does termination of an employee propagate to production systems? What’s the typical time to access removal?

These questions move the conversation beyond “is your SOC 2 clean?” to “what is your actual security posture?” The vendors who can answer them directly are the ones with actual security. The vendors who deflect are the ones whose SOC 2 is covering for less-mature practices.

read a SOC 2 report in this order

1. 1scopeWhich systems and services are covered, and which aren't.
2. 2exceptionsWhere the reality lives.
3. 3type and periodType I is one day. Type II covers months.
4. 4audit firmWho signed it, and how rigorous they usually are.
5. 5controls versus realityDoes the description match how the product is built and run?

the exercise

If your company uses SOC 2 reports as the primary evidence of vendor security, here is the exercise.

Pick five vendors. Pull their SOC 2 reports. Read them fully, not the executive summary.

For each report, answer: what is the scope? What exceptions were noted? Who is the audit firm? Is the engineering operation reflected in the controls description?

Compare the reports to each other. You will see a spread, some are clearly more rigorous than others.

For any vendor whose report looks weak (narrow scope, no exceptions disclosed, vague control descriptions, small audit firm), pursue the follow-up questions above. If you can’t get direct answers, that’s a data point.

For high-risk vendors (those processing your sensitive data), SOC 2 alone is not enough. You need the pen test, the operational follow-up, and ideally a direct conversation with the vendor’s security team.

SOC 2 is a useful baseline. It filters out vendors who haven’t done the formalization work. It does not certify that a vendor is secure. Using it as proof of security is the mistake that most procurement teams make, because the marketing encourages it.

Read the report. All of it. Then ask the next question.