SCA Metrics for Safe Releases: CVSS, SLAs, Exceptions

What problem SCA metrics solve

SCA scanners are in almost every pipeline today: in CI, in repositories, in container images. Yet releases still ship with risk, because finding a vulnerability doesn’t answer the main question: what to do right now and who is responsible.

A typical scenario: the scanner shows dozens of vulnerabilities in dependencies. Some are not exploitable in your context, some are long-known, and some are only dev dependencies. The team either ignores the report or tries to fix everything at once and misses deadlines.

The common conflict is the same: release speed versus InfoSec requirements. Developers need to deliver features, while InfoSec must not let critical risk reach production. Without shared rules, the conversation quickly becomes a debate of feelings: “this is important” versus “we won’t make it in time”.

SCA metrics turn scans into understandable numbers and thresholds. They can be used as rules, not opinions. They help separate noise from real risk, agree on trade-offs in advance, and make the process predictable.

With such metrics, teams make practical decisions: block a release or allow it under explicit rules; decide what to fix first and within what timeframe; when to issue an exception, for how long, who owns the risk and how to report to InfoSec. Another important outcome: metrics show where the problem is in the process—outdated dependencies, infrequent updates or team discipline.

Simple example. A team prepares a release and the scanner finds a CVE with a high score in a popular library. Without metrics, chaos begins. With metrics it’s clear: the vulnerability affects an optional module only, a patch exists, and policy requires fixing such findings within seven days. The decision becomes obvious: either delay the release by 1–2 days to update, or issue a temporary exception with an expiration date and compensating control (e.g., disable the module). Such a decision can be defended to InfoSec.

What Mend, Snyk and Sonatype actually measure

SCA (Software Composition Analysis) inspects which third-party libraries and components make up your application. The tool checks dependency versions, looks for known vulnerabilities and license issues. Essentially it answers: what are we bringing into production with our code?

Mend, Snyk and Sonatype do similar core tasks: build a dependency list, match it against vulnerability databases, show priorities and help encode rules as policies. Differences are often not in what they "see" but in how they collect data, explain risk and support remediation workflows.

Vulnerability ratings come from multiple sources: typically a mix of NVD, vendor bulletins, community data and the vendor’s own research. Therefore the same CVE can have different scores, different descriptions of affected versions, and even different "exploited/not exploited" statuses. Practically, this means the metric "how many criticals do we have" depends on chosen sources and normalization rules, not only on actual risk.

False positives and gaps most often appear in four areas:

• transitive dependencies: a component is pulled in indirectly and the team doesn’t know where it’s used
• incomplete or nonstandard manifests: the build fetches packages differently than the scanner expects
• misidentified components: similar names, forks, or repackaged artifacts
• exploitation context: a vulnerability exists but the risky feature isn’t used or the component is disabled by configuration

Important: Mend, Snyk and Sonatype measure not the "security of the application" as a whole but the quality of inventory and the level of known supply-chain risk. The clearer you are about the measurement boundaries, the easier it is to choose metrics that actually help ship safer releases.

CVSS policy: thresholds and context, not just the score

CVSS is convenient as a common language, but the score alone does not equal risk to your product. The same CVE with CVSS 9.8 can be critical for a public API and have almost no impact on a closed internal service. So a metric should consider not only the score but where and how the component is used.

A practical approach: set thresholds that the team understands and that predictably affect release decisions. Revisit them when architecture, threat model or InfoSec requirements change.

Thresholds that commonly stick

Teams often start with a simple rule set:

• block a release for CVSS >= 9.0 when there is an exploit or the vulnerability is in an externally reachable boundary
• do not block automatically but create a mandatory ticket for 7.0–8.9 with a clear remediation timeframe
• allow release for 4.0–6.9 if risk is accepted and there is an upgrade plan
• do not treat 0–3.9 as automatically safe if the vulnerability lies in a critical data path

Context: what to add to the score

To make Mend, Snyk or Sonatype signals useful, layer several checks over CVSS: is there a fixed version, is the vulnerable code actually used, and can the component be attacked in your configuration?

Example: a library with CVSS 8.1 was pulled in transitively. If it’s present only in test dependencies or not included in the production build, it’s not a reason to block the release. But if the transitive dependency ends up in the runtime artifact and sits on the perimeter (e.g., a web server), treat it as a full issue and require remediation or a documented risk acceptance with a deadline.

A simple rule for transitive dependencies: always consider them when they are included in production artifacts and reachable at runtime. Ignore them only when this is provable and recorded in policy.

Coverage and inventory quality metrics

Even the best SCA scanner is useless if it sees only part of the code or reads dependencies "incorrectly." So first measure not vulnerabilities but the quality of input data: what gets analyzed and how complete the component inventory is.

Scanning coverage: what actually gets analyzed

Coverage is better expressed not as a single checkbox "scanned/not scanned" but as several ratios that reveal blind spots. For example: what share of repositories are scanned in CI (not manually), which branches are actually checked (main and release branches), what artifact types (containers, packages, images) are included in SCA, and how often scans are skipped due to misconfiguration.

Inventory quality: can you trust the dependency list?

This group of metrics answers: how accurate is our component list. Track the share of projects with correct manifests (package-lock, pom.xml, requirements.txt). If manifests are missing or don’t pin versions, the same project will yield different scan results and debates about risk will never end.

Also measure "unknown dependencies": components without a clear source, version or owner. A practical marker is how many findings cannot be tied to a repository, package or team within a reasonable time.

Separately, track dependency "staleness": the share of components that are more than N major versions behind. You may have no critical CVEs now, but if half the libraries are three majors behind, the next update cycle will be painful; better allocate time while updates still fit release plans.

Remediation timing: SLAs, MTTR and vulnerability age

To make releases safer, "how many vulnerabilities did we find" is less important than how quickly the team closes risks and whether the remediation queue becomes an endless backlog.

Set SLAs not "in general" but by severity and artifact type (application, container, library). A starter framework often looks like:

• Critical: remediate or mitigate within 7 days
• High: within 30 days
• Medium: within 90 days
• Low: planned work, but avoid debt growth

Then the question becomes how to measure speed. MTTR (mean time to remediate) is useful for reports but can be skewed by a few long-lived items. Keep median time to remediate alongside MTTR: it better shows typical team speed. If MTTR is 40 days while the median is 12, most fixes are quick but a few old problems inflate the average.

A third necessary metric for SLAs to work is vulnerability age: how many findings exceed the allowed timeframe. Show not just the count but where they are—by repository, service or team. That turns the conversation specific: not "we have 200 High" but "the payment service has 14 High older than 30 days."

Finally, track backlog dynamics. Imagine scanners find 20 new High each week but the team closes 10. Formally you’re "working," but debt grows. Good reports show the 4–8 week trend: are you shrinking the queue or accumulating it, and which categories (e.g., transitive dependencies) are the main contributors.

Release readiness metrics: what truly blocks delivery

When you plug SCA into CI/CD, builds are usually stalled not because of the vulnerability itself but because of how decisions are made: block, fix now, or issue an exception. Release readiness metrics should highlight process bottlenecks.

A useful starting point is build blocks by policy. Mend, Snyk and Sonatype typically show which rules fired (CVSS, exploit availability, dependency type, license). If a noticeable share of builds fail for the same rule, that’s a signal: either the rule is too strict for the current reality, or the team lacks a quick path to update the dependency.

To understand why delivery stalls, a few indicators are usually enough:

• percent of builds blocked by policy, broken down by reason
• share of releases that pass without exceptions
• time to decision on a critical finding (from appearance in the pipeline to verdict: fix, compensating measure or exception)
• triage quality: share of findings marked irrelevant (false match, unused component, unreachable code)

A good benchmark: if "time to decision" exceeds the time to apply the fix, the delay is caused by approvals and lack of a clear owner. For example, in a government or banking project, a release might stall for a week of back-and-forth with InfoSec over one CVE with unclear status: is there an exploit, does it affect our configuration, who can accept the risk.

Also watch the rise of "irrelevant" findings. A high share usually means the policy is blunt, there’s no context on component usage, or the team spends time arguing with the tool instead of fixing issues. This directly slows delivery and erodes trust in SCA.

Exceptions without chaos: rules, durations and controlled risk acceptance

Exceptions in SCA are necessary, but they are also the thing that most often turns the process into a mess: a vulnerability was "temporarily" allowed and then forgotten. For metrics to work, an exception must be a managed decision: with an owner, an expiry date and a review.

Exceptions are typically acceptable in three cases: there’s no fix (no patch or upgrade breaks compatibility), the finding is false (the CVE is misattributed to the package), or compensating controls reduce risk to an acceptable level (e.g., the service is not externally reachable, WAF rules are enabled, the vulnerable feature is disabled).

Minimum rules for exceptions

A good minimum looks like this:

• duration: exceptions are issued for a limited period (e.g., 30–90 days) and are scheduled for review
• owner: each exception has a responsible product owner who monitors the review
• justification: a short recorded reason and what’s been done to reduce risk
• approval: the risk acceptance is signed by the assigned risk owner (often the product lead or InfoSec, depending on criticality)
• verification: each release checks whether the exception has expired or whether a fix is available

Example: a team prepares a release for a government client. SCA finds a critical vulnerability in a transitive library, but updating breaks integration with an external module. The decision: issue an exception for 45 days, close external access to the vulnerable component and add monitoring. If a patch appears earlier, the exception is revoked immediately.

Metrics for exceptions

Keep it simple. Often four numbers are enough: number of active exceptions, average age of exceptions, share of overdue exceptions, and the breakdown by reason (no fix, false positive, compensating measures). These metrics quickly reveal where risk accumulates unnoticed, even when Mend, Snyk or Sonatype dashboards look "green."

Reports for InfoSec and leadership: formats that get read

An SCA report shouldn’t be a 40-page export from Mend, Snyk or Sonatype. InfoSec and leadership need clear answers: where is risk growing, where are we missing deadlines, and what is blocking releases right now. If a report helps make a decision in five minutes, it will be read.

What InfoSec usually needs to see

InfoSec watches trends and discipline, not one-off snapshots. A good report shows monthly trends, SLA breaches and a list of critical open risks that actually remain in production. Also highlight exceptions (risk acceptances): who approved them, their expiry dates, and what’s being done instead of a fix.

Keep the metrics set short. For example, KPIs can be distilled to:

• number of open Critical/High vulnerabilities in release branches and change over the period
• share of vulnerabilities remediated within SLA
• MTTR for High vulnerabilities
• age in days of the oldest open High vulnerabilities
• number of active exceptions and the share overdue

Scanning coverage and the share of vulnerabilities without available patches are useful as contextual indicators.

Slices that help management

The same KPI set should be shown by product/service, by team, by supplier (which libraries create most debt), and by artifact type (containers, applications, images). That shows whether the problem is in a specific product or in the process as a whole.

For audits, collect evidence in advance: the approved threshold policy (what blocks a release), the exceptions log with expiry dates, and remediation history (tickets, merge dates, release dates). Then the audit is about risk control, not arguing over numbers.

Step-by-step: how to embed SCA metrics into the release process

For metrics to work, they must be tied to a decision: release or not. Start simple: one goal (reduce delivery risk), three stakeholder groups with roles (Dev, DevOps, InfoSec) and a clear escalation rule if a release is blocked.

Select a small set of indicators and thresholds. Too many metrics become unused. Usually 5–7 metrics that truly affect release decisions are enough.

Minimal rollout plan

• assign owners: who reads reports, who accepts risk, who changes pipelines
• define blocking thresholds: e.g., zero critical vulnerabilities without a remediation plan or confirmed risk acceptance
• set up triage: who and how quickly confirms that a finding is real and affects production
• record exceptions as managed entries: reason, decision owner, expiry date, compensating measures
• automate collection in CI/CD and add a regular report for InfoSec and the release manager

After launch, lock in the cadence. Weekly, review SLA breaches, vulnerability age and exceptions nearing expiry. Adjust policy in small steps: if releases are repeatedly blocked by one dependency type, the problem is usually in update rules or missing component ownership.

Example: before a release, a team sees a CVSS 9.8 vulnerability in a transitive library. The decision should not be "postpone" but one of two: urgent upgrade with a fixed date, or an exception for 14 days with explanation of why the risk is acceptable and what measures reduce it now.

Example scenario: one release, three findings and a risk decision

The team releases every two weeks. The project uses dozens of libraries and SCA runs in the pipeline before the build. This time the tool (Mend, Snyk or Sonatype—doesn’t matter) finds three issues in dependencies.

Policy: block release if there is CVSS 9.0+ affecting production with a confirmed exploit or evidence the vulnerable code is used. CVSS 7.0–8.9 does not block automatically but requires a remediation plan and deadline. Everything below goes to the backlog unless special conditions apply.

Findings in this release:

• CVSS 9.8 in a logging library: direct dependency, exploit exists, update available
• CVSS 8.1 in a transitive library: no exploit, update requires a major version bump
• CVSS 6.5 in a library used only in dev tests

Edge cases are resolved via triage: assign an owner (team lead or service owner), set a deadline, and issue an exception with an expiry date if needed. Exceptions without expiry are not accepted.

The release decision updates three metrics:

• share of blocking vulnerabilities per CVSS policy with context (exploit, environment, reachability)
• vulnerability age and MTTR for similar past findings
• number of active exceptions and how many are overdue

Outcome: the first vulnerability is fixed before release (dependency updated), the second is released under a 14-day risk acceptance with a migration task, and the third is marked non-impacting to production and not worked on.

Quick checklist: is your metrics program ready?

When SCA metrics work, they give one clear answer: can we release or not, and what is blocking it. Check for the basic elements without which numbers quickly turn into noise.

Five questions that must have clear answers

• are there formal thresholds by severity (for example, CVSS) and rules for when a release is blocked versus allowed with risk acceptance?
• are remediation timeframes defined by severity, and are SLA breaches visible by product and team?
• are exceptions tracked as managed records: reason, decision owner, expiry date, review conditions?
• does InfoSec receive a regular report in a single format: trends over time and the list of critical open risks in production?
• do you analyze causes of delays: no patch, version conflict, false positive, unused dependency, breaking compatibility?

How to know this is not just formality

A good sign: by the end of the week you can name 2–3 concrete blockers and their owners. A bad sign: discussions degenerate into arguing about "high/medium" and no one owns deadlines.

A simple test: at release planning take one critical finding and ask the team to fill a decision card in 10 minutes: block or accept risk, expiry date, next steps. If it takes hours and ends in email threads, the metrics program is not ready to govern releases.

Next steps: move from scans to managed releases

To stop SCA scans from being noise, start small and make results a required part of the release decision. A common mistake is to enable multiple tools at once, get different numbers and lose the team’s trust.

A practical starter: pick one product (Mend, Snyk or Sonatype) and set a baseline of metrics for the current state. In 2–4 weeks you’ll have an honest picture: how many vulnerabilities actually block releases, how many linger for months, and where most exceptions come from.

Minimal actions that usually deliver quick wins:

• agree on 2–3 CVSS thresholds with context (e.g., a separate rule for exploited vulnerabilities)
• introduce SLAs for Critical and High and assign owners
• limit exceptions in time (e.g., 30–90 days) and require justification and compensating measures
• agree on one concise report for InfoSec and leadership: trend, debt, exceptions, release status

Policy alignment between Dev and InfoSec should focus on concrete product risks, not abstract "perfect security." For example, if SCA finds a High in a library but the component is not used at runtime, the release need not be blocked—but record the decision (and the timeframe) and avoid turning it into an eternal "later."

Consider bringing in a systems integrator when you need more than scanning: enforce rules in CI/CD, unify risk-acceptance templates, define roles, automate reports and train teams.

If you work in the public sector or a large organization where formal reporting and deadline control are important, GSE.kz can help with selecting and implementing SCA as part of system integration: policy and report setup for InfoSec, pipeline integration and ongoing 24/7 support.