QualityWorks Insights
More QA, Not Less
Closing the Verification Gap in AI-Accelerated Software Delivery
A QualityWorks Consulting Group whitepaper
Executive summary
Generative AI is becoming part of mainstream software engineering, but faster code generation does not automatically produce more reliable software. Gartner forecasts that 90% of enterprise software engineers will use AI code assistants by 2028, while Stack Overflow’s 2025 survey found 84% of developers already use or plan to use AI tools.¹²
At the same time, evidence from security benchmarks, developer studies, and software-delivery research shows a consistent verification problem. Veracode’s 2025 benchmark found known security flaws in 45% of generation tasks across more than 100 large language models.⁵ Georgetown CSET’s formal-verification study found that roughly half of tested generated snippets contained bugs under its security-focused benchmark.⁶
A Stanford-led user study found that participants with an AI coding assistant wrote less secure code while being more likely to believe their code was secure.⁷
The implication is not that AI-written code is always worse, or that organizations should slow AI adoption. The stronger conclusion is that AI changes the economics and shape of software risk: teams can produce more code faster, while deep contextual, security, integration, and architectural failures still require independent verification.
QA therefore has to scale with AI-enabled delivery — through automated quality gates, security checks, risk-based human review, and continuous test evidence.
Key findings
AI adoption is accelerating
Gartner forecasts that 90% of enterprise software engineers will use AI code assistants by 2028.
Security remains a concern
Veracode’s 2025 benchmark found a known security flaw in 45% of generation tasks.
Verification must keep pace
Faster code generation increases the need for testing, security validation, human review, and continuous assurance.
1. AI is accelerating software creation
The most defensible story is not that a fixed percentage of all new production code is AI-generated. Public estimates vary substantially depending on whether they measure suggestions, accepted completions, committed code, or self-reported use.
GitHub has reported that Copilot completed about 46% of code in files where it was enabled in earlier product research, while its 2024 Octoverse report documented rapid growth in public generative-AI projects.³ Those figures should not be generalized into a claim that 41% of all new production code is AI-generated.⁴
The broader adoption signal is nevertheless clear. Stack Overflow’s 2025 Developer Survey found 84% of developers use or plan to use AI tools, and Gartner forecasts that 90% of enterprise software engineers will use AI code assistants by 2028.¹²
The practical question for technology leaders is therefore not whether AI coding will enter the software lifecycle, but how to verify its output at the same pace.
Figure 1
AI use is widespread, while confidence remains limited
Selected findings from the Stack Overflow 2025 Developer Survey.
to use AI
AI accuracy
AI accuracy
Figure 1. AI use is widespread, while confidence in output accuracy remains limited.
Source: Stack Overflow 2025 Developer Survey
.
2. Faster generation does not remove quality risk
Security remains a measurable weakness
Veracode’s 2025 GenAI Code Security benchmark tested more than 100 LLMs on 80 coding tasks across Java, JavaScript, Python, and C#. Across the benchmark, only 55% of generation tasks produced secure code; 45% introduced a known security flaw.⁵
Veracode is a security vendor, so the result should be read in that context, but its benchmark is transparent about task construction and is directionally consistent with independent academic work.
Georgetown University’s Center for Security and Emerging Technology (CSET) used formal verification on code generated by five models under security-focused prompts.⁶ The authors found roughly 48% of generated snippets compiled but contained a bug flagged as insecure, while explicitly cautioning that the figure is a rough upper bound and not representative of every production workflow.
That caveat matters: the evidence supports stronger verification, not a blanket claim that every AI-generated function is unsafe.
Figure 2
Code that works is not always code that is secure
Results from Veracode’s 2025 GenAI Code Security benchmark.
Figure 2. Veracode benchmark result across 100+ LLMs and 80 coding tasks.
Source: Veracode 2025 GenAI Code Security Report
.
Human oversight has its own failure mode
Verification cannot depend on confidence alone. In a Stanford-led controlled study, researchers found participants using an AI code assistant wrote significantly less secure code than participants without the assistant, while also being more likely to believe their code was secure.⁷
Stack Overflow’s 2025 survey shows developers themselves recognize the problem: 46% actively distrust AI output accuracy, compared with 33% who trust it, and only 3% highly trust it.²
System-level delivery risk can persist even when individual coding tasks get faster
AI-assisted development can increase coding speed without automatically improving the stability of the overall delivery system. In DORA’s 2024 research, each 25% increase in AI adoption was associated with an estimated 1.5% decrease in delivery throughput and 7.2% decrease in delivery stability.⁸
These results do not establish that AI directly caused the declines. They do, however, reinforce an important point: faster code generation does not remove the need for strong delivery controls.
As AI increases the volume and speed of software creation, organizations still need testing, review, deployment safeguards, and feedback loops capable of keeping pace.
The implication for QA is not to resist AI-enabled development, but to ensure that verification capacity scales alongside development capacity.
Figure 3
Estimated decreases associated with a 25% increase in AI adoption
Magnitude of the estimated decreases identified in DORA’s 2024 research.
Figure 3. Estimated decreases associated with a 25% increase in AI adoption.
DORA 2024 reported an estimated 1.5% decrease in delivery throughput and 7.2%
decrease in delivery stability for each 25% increase in AI adoption. These
findings represent statistical associations and should not be interpreted
as proof of causation.
Source: DORA — Impact of Generative AI in Software Development
.
3. The verification gap is the real scaling problem
AI changes software economics by lowering the marginal effort required to produce code. But verification work does not disappear at the same rate.
Reviews still need context about architecture, authorization boundaries, data integrity, compliance obligations, failure modes, and real user behavior.
The result is a verification gap: production capacity can grow faster than the organization’s ability to validate what it is producing.
“
More QA does not automatically mean more headcount.
It means more verification capacity: stronger automation, broader coverage,
risk-based human review, better observability, independent validation where
appropriate, and more evidence about what is actually reaching production.
That gap is especially important because many high-impact failures are contextual rather than syntactic. A function can compile, pass shallow tests, and still violate a business rule, mishandle permissions, interact badly with another service, or fail under production data.
AI can also help write tests, but test generation and code generation can share the same missing context. Independent evaluation therefore remains essential.
4. What mature assurance looks like
Treat AI output as a draft that must earn trust
Require review and automated checks on AI-touched code before merge. At minimum, use unit and integration tests appropriate to the change, static application security testing, dependency or software-composition analysis, and secrets detection.
- Use risk-based human review for authorization, data integrity, privacy, financial logic, safety-critical workflows, and other high-consequence boundaries.
- Measure the quality of AI-assisted changes separately where feasible: escaped defects, severity distribution, rework, rollback rate, test failures, and mean time to remediate.
- Keep generation and validation logically independent for critical systems. The same agent or workflow that produced a change should not be the only mechanism deciding that the change is safe.
- Increase test automation so verification capacity can keep pace with software-generation capacity, but preserve human judgment where contextual understanding is required.
Use continuous evidence, not point-in-time confidence
NIST’s AI Risk Management Framework Generative AI Profile and its AI-focused Secure Software Development Framework profile both emphasize lifecycle risk management rather than one-time approval.⁹¹⁰
The U.S. Department of Defense’s Software Modernization Strategy similarly places “security, stability, and quality at speed” ahead of speed alone and calls for robust testing integrated into delivery pipelines.¹¹
These are high-assurance examples, not universal regulatory mandates, but the operating principle generalizes:
faster delivery requires faster, persistent evidence of quality.
5. The economics still favor earlier verification
The original draft relied on a widely repeated “15x to 100x” defect-cost curve. That specific multiplier is difficult to trace cleanly to the cited NIST source and should not be used as a headline fact.
A stronger economic anchor is NIST’s 2002 study of inadequate software testing infrastructure, which estimated an annual U.S. economic cost of $59.5 billion and estimated that feasible improvements could reduce about $22.2 billion of that cost.¹²
A broader 2022 estimate from the Consortium for Information & Software Quality placed the annual U.S. cost of poor software quality at $2.41 trillion and accumulated software technical debt at roughly $1.52 trillion.¹³
These are macroeconomic estimates rather than a formula for any one company, but they support the same management conclusion: verification and maintainability are economic controls, not just engineering preferences.
6. A practical 90-day operating model
0–30 days
Establish a baseline
Document where AI-assisted code enters the lifecycle. Add or confirm automated quality and security checks. Define which changes require human approval. Begin measuring escaped defects and rework for AI-assisted changes.
31–60 days
Scale verification capacity
Expand automated regression coverage, integration tests, and risk-based security testing. Add test-generation assistance where useful, but review generated tests for missing context and false confidence.
61–90 days
Move to continuous assurance
Create a lightweight dashboard of quality evidence: deployment failures, rollback rate, high-severity findings, defect escape rate, test coverage where meaningful, and mean time to remediate.
Use trends to decide where stronger gates or more QA capacity are justified.
Conclusion: more QA, not less
AI can make software teams dramatically faster, and that is precisely why QA cannot be treated as expendable overhead.
The more cheaply and quickly organizations can generate code, the more valuable disciplined verification becomes.
The winning model is not humans versus AI, or manual QA versus automation. It is AI-accelerated delivery paired with equally modern assurance: automated where repeatable, independent where necessary, and human where judgment carries the risk.
The goal is not to slow AI down.
It is to make sure software can move at AI speed without moving risk into production at the same speed.
References
- Gartner, “Gartner Identifies the Top Strategic Trends in Software Engineering for 2025 and Beyond.” 2025.
- Stack Overflow, 2025 Developer Survey — Artificial Intelligence.
- GitHub, Octoverse 2024.
- GitHub, “The economic impact of the AI-powered developer lifecycle and lessons from GitHub Copilot.”
- Veracode, 2025 GenAI Code Security Report.
- Jessica Ji, Jenny Jun, Maggie Wu, and Rebecca Gelles, “Cybersecurity Risks of AI-Generated Code.” Center for Security and Emerging Technology, Georgetown University, 2024.
- Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh, “Do Users Write More Insecure Code with AI Assistants?”
- DORA, Impact of Generative AI in Software Development.
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1). 2024.
- National Institute of Standards and Technology, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models: An SSDF Community Profile (NIST SP 800-218A). 2024.
- U.S. Department of Defense, Department of Defense Software Modernization Strategy. 2022.
- National Institute of Standards and Technology, Planning Report 02-3: The Economic Impacts of Inadequate Infrastructure for Software Testing. 2002.
- Consortium for Information & Software Quality, The Cost of Poor Software Quality in the US: A 2022 Report.