WQA

AI-native engineering

Perspective 01

AI is changing the system that builds software.

AI can accelerate production while leaving the path to dependable delivery constrained. The advantage comes from making intent, context, evaluation and accountability part of the engineering system.

By
WQA
Published
26 August 2026
Reading time
13 min read

Why context, evaluation and governance now belong inside the engineering system rather than around it.

Executive summary

  1. 01

    AI can accelerate individual tasks while making the overall delivery system less dependable. The unit of improvement must therefore move from code generation to the complete path from intent to production evidence.

  2. 02

    Context, evaluation, permissions and human accountability are no longer supporting activities around engineering. They are production infrastructure for people and agents.

  3. 03

    Treat the lifecycle as an operating product: define its users, outcomes, interfaces, service levels, ownership and feedback loops, then improve it through evidence from real delivery.

Faster code is not the same as faster delivery

AI is changing software engineering first at the point where work becomes visible: the production of code, tests, designs, documentation and analysis. That is useful, but it can produce a misleading picture of progress. A team may generate an implementation in hours and still wait days for a decision, search across contradictory sources for context, repeat reviews that were never encoded, or discover in production that the wrong problem was solved.

The constraint has moved. When creation becomes cheaper, judgement, coordination and assurance account for more of the elapsed time and more of the risk. Code can arrive faster than the organisation can explain why it is needed, assess whether it is correct, connect it safely to the estate and learn from its operation. Without redesigning that wider system, AI increases the volume of work presented to the same bottlenecks.

This is why the software lifecycle is becoming a product in its own right. It has users: product teams, engineers, designers, operators, risk owners and increasingly agents. It has interfaces: briefs, repositories, tools, policies, environments and release controls. It has outcomes that matter to the business: time to value, reliability, adaptability, cost and confidence. Like any serious product, it must be deliberately designed, operated and improved rather than inherited as a chain of meetings and tools.

The evidence argues for systems thinking

The current evidence is more useful than either the promise that AI will transform every team or the rebuttal that it changes nothing. Google's 2025 DORA research describes AI as an amplifier: it can strengthen a capable organisation, but it can also expose and intensify weak foundations. The report's central management implication is that returns depend on the underlying organisational system, not on tool adoption alone.

METR's early-2025 randomised study provides a productive counterweight. In a set of tasks completed by experienced contributors working in repositories they knew well, the study found that access to then-current AI tools increased completion time by 19%, even though participants expected them to help. METR subsequently reported that a later experiment was affected by strong selection effects and provided only weak evidence about current speedups. This does not resolve the productivity question. It shows why confident universal claims, positive or negative, are a poor basis for investment.

Productivity depends on the task, tool, model, engineer, codebase and surrounding workflow. Perceived speed can differ from measured speed, while local gains can be absorbed by review, integration or rework. A sound executive question is therefore not ‘How much faster can an AI coding tool make an engineer?’ It is ‘Which constraint in our path from idea to dependable production has changed, and what happens to the whole system when it does?’

References010203

Product intent becomes an engineering dependency

Traditional delivery can conceal imprecise intent for surprisingly long periods. People fill gaps through conversation, institutional knowledge and repeated correction. An agent is more literal. Give it an ambiguous objective, incomplete acceptance conditions and a repository full of historical compromise, and it can produce a plausible answer to a question nobody intended to ask.

The practical response is not to write larger specification documents. It is to make the decision chain legible. What user or operational outcome is required? Which behaviours must remain unchanged? What evidence would establish that the work is acceptable? Which constraints are mandatory, and which are preferences? Who can resolve uncertainty? These become inputs to engineering rather than administrative material created before engineering begins.

Consider a team changing the eligibility logic in a regulated customer journey. The task is not merely to alter a rule in code. The delivery system must connect the approved policy, affected customer states, data definitions, exception handling, test scenarios, audit evidence and accountable decision-maker. An agent may assist at several points, but it should not have to infer the commercial or regulatory boundary from fragments. Better intent reduces wasted generation for people as well as machines.

Context becomes production infrastructure

AI systems do not work from intelligence alone. Their behaviour depends on the instructions, retrieved knowledge, available tools, permissions and history assembled around a task. OpenAI's agent guidance describes a basic agent in terms of model, tools and instructions; Anthropic's engineering work repeatedly returns to context engineering, tool design, permissions and evaluation. The implication for an engineering organisation is direct: context is part of the runtime of work.

Useful context may include product priorities, architecture decisions, design rules, source code, domain language, data policy, security requirements, incidents and live service health. More is not automatically better. Excess material can be stale, contradictory or irrelevant. The engineering challenge is to decide which source is authoritative, how it is structured, who owns it, when it expires and how a person or agent retrieves only what the decision requires.

This is a product and platform responsibility. Teams need dependable interfaces to context, not a scavenger hunt across documents, tickets, messages and dashboards. Provenance should remain visible so an output can be traced to the policy, decision or evidence that shaped it. Where sources conflict, the system should expose the conflict rather than silently choosing the most convenient answer. The quality of generated work will increasingly reflect the quality of this context supply chain.

References0405

Evaluation becomes part of product development

Deterministic software is not perfectly predictable, but teams can usually define a known result for a given test. AI-assisted and agentic workflows introduce a different problem: several outputs may be acceptable, behaviour can change when a model or context changes, and success may depend on a sequence of tool calls rather than a single response. Review at the end is too late and too narrow.

Evaluation must begin with the real job. For a coding workflow, compile success or test coverage is only part of the evidence. The team may also need to assess whether the change matches the intended behaviour, respects architecture and policy, handles failure, remains maintainable, and produces the required operational signals. Some checks can be automated. Others need sampling, comparison, specialist review or observation in a controlled production release.

This creates an evaluation system rather than a pass-or-fail gate: representative tasks, known failure cases, quality criteria, security tests, cost and latency thresholds, production outcomes, and a record of why a release decision was made. The evidence should follow changes to models, prompts, tools and context just as conventional test evidence follows code. Evaluation then serves two purposes: it prevents avoidable failure and makes improvement measurable.

References0406

The platform must expose safe, usable interfaces

A software estate designed only for human-operated interfaces is difficult to use safely with agents. Critical actions may be available only through screens, permissions may be broad, service boundaries may be unclear and operational state may be inaccessible. The response is not to connect an agent directly to everything. It is to create governed interfaces through which both people and agents can discover context, invoke tools and observe results.

Current cloud architecture guidance treats agent systems as combinations of models, orchestration, tools, data, memory, identity and observability. That is a useful reminder that the model is one component. An engineering platform also needs scoped identities, explicit tool contracts, isolated execution, audit trails, limits on consequential actions and routes to human intervention. These controls should be designed with the workflow, not added after a demonstration succeeds.

The same work improves the estate beyond AI. A well-defined deployment interface, authoritative service catalogue or machine-readable policy reduces friction for engineers and automation alike. Conversely, asking an agent to compensate for undocumented dependencies and unreliable environments simply automates exposure to existing weaknesses. The platform should make the safe path the usable path.

References0706

Human accountability becomes more explicit, not less

The usual debate frames autonomy as a choice between human control and machine speed. In practice, dependable autonomy requires a more precise allocation of responsibility. A person does not need to approve every low-consequence action, but the organisation must decide which actions an agent may take, within what limits, using which evidence and under whose authority.

That boundary should reflect consequence. An agent that classifies internal maintenance tasks may operate with broad freedom and retrospective sampling. An agent that changes production infrastructure, customer entitlements or regulated decisions needs tighter permissions, stronger pre-release evidence, explicit confirmation or a controlled execution environment. Escalation is not a generic ‘human in the loop’ button. It needs an owner, sufficient context, a response expectation and a safe state while a decision is pending.

Accountability also includes the lifecycle itself. Someone must own the quality of the context, the relevance of the evaluations, the integrity of the tool interfaces and the response to production signals. If every component has a technical owner but nobody owns the behaviour of the complete workflow, failures will continue to fall between teams. Designing the lifecycle as a product makes that end-to-end accountability visible.

References0406

The economics have to be measured end to end

AI changes the cost structure of delivery. The marginal cost of generating another design option, code change or test may fall, while consumption charges, review demand and platform investment rise. If the organisation measures only output or licence adoption, it can report activity as value and overlook costs transferred to other teams. A cheaper first draft is not an economic gain when it creates more integration work, production incidents or support demand.

Measure at the workflow level. Establish the elapsed time from a committed decision to a dependable production outcome, including waiting and rework. Track change failure and recovery, but also the product result the change was intended to produce. Add the cost of model use, environments, specialist review and operating the new controls. The purpose is not to assign every minute to AI. It is to determine whether the redesigned system produces its intended outcome with less delay, risk or total effort.

Quality deserves particular attention because faster generation can change the volume presented for review. Teams may merge more small changes while architecture coherence, security or maintainability deteriorates gradually. Useful leading signals include the proportion of generated work substantially revised by a person, evaluation failures by category, repeated exceptions, review queue time and the age of context sources. Production signals remain decisive: reliability, customer outcomes, recovery, cost and the ability to make the next change.

This measurement also protects useful experimentation. A workflow that produces no immediate labour saving may still be valuable if it shortens time to evidence, enables a previously impractical service or reduces concentration of knowledge in a few people. Conversely, high adoption and positive sentiment do not prove a business return. The lifecycle product needs a balanced scorecard tied to its intended outcome, not a single universal productivity number.

References010203

Do not turn adoption into another transformation programme

There is a legitimate counterargument: teams learn new tools by using them, and a large programme to redesign the engineering system could delay useful gains. That is correct. Not every workflow needs an agent, not every use of AI needs enterprise architecture, and governance that is disconnected from consequence can become another source of latency.

The answer is proportionality. Low-risk assistance such as explanation, search, drafting or local test generation can begin with lightweight controls and clear data rules. Higher-responsibility workflows need stronger context, permissions, evaluation and observation. The organisation should invest in shared infrastructure when repeated use shows that multiple teams face the same constraint, not because an abstract target architecture demands it.

Equally, buying licences and waiting for productivity to emerge is not a strategy. It leaves teams to solve context, integration and assurance independently and makes results difficult to compare. The better route is directed experimentation: select a delivery workflow with a measurable business effect, establish its baseline, change the way work moves through it, and decide from evidence what should be retained, stopped or turned into a reusable platform capability.

References0103

Manage the lifecycle as an operating product

A lifecycle product needs a product definition. Its users are the people and agents moving an idea into production. Its outcome is not more generated output; it is an intended product change operating dependably. Its interfaces connect intent, context, tools, environments, controls and evidence. Its service levels describe how quickly decisions are made, how reliably changes move, how failures are contained and how learning returns to the next cycle.

WQA calls the lifecycle-wide delivery system Agentic Product Engineering, expressed through Value, Experience, Production and Evolution. WQA uses MPLR to organise the shared context, agents, skills, workflows, evaluation and governance that connect those movements. WQA can also establish or adapt the platform inside an organisation. This is not a proposal to replace every existing tool or create an autonomous factory. It is a way to design the connections that existing delivery models often leave implicit.

The starting point should be one workflow where delay, rework or lost context has a measurable business effect. Map the current path, including waits and reversals. Establish a baseline. Introduce agents only where they improve a defined decision or action. Build the minimum context and control needed for that responsibility. Observe the result in delivery and production, then reuse only the capabilities that prove valuable.

A product roadmap for the lifecycle should therefore be organised around constraints removed and outcomes improved, not a catalogue of AI features. One quarter may focus on making product intent machine-readable and reviewable; another on dependable environment access or continuous evaluation. Each investment should make several future workflows easier while remaining accountable to evidence from the workflow that justified it.

  • Outcome: name the customer or operational change the workflow must produce, plus measures for time, quality, cost and confidence.
  • Users and responsibilities: identify who creates, decides, reviews, operates and learns, including the bounded role of each agent.
  • Context and interfaces: define authoritative inputs, permitted tools, state changes, provenance and the safe path through the platform.
  • Evaluation and service levels: test the real task and set expectations for decision time, reliability, recovery and escalation.
  • Feedback and ownership: connect production evidence to the next decision and give one accountable owner responsibility for the complete workflow.

Sources and further reading

  1. Google Cloud DORA2025 DORA Report: State of AI-assisted Software Development
  2. METRMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
  3. METRUpdate on measuring AI's impact on experienced developer productivity
  4. OpenAIA practical guide to building agents
  5. AnthropicAnthropic Engineering
  6. NISTArtificial Intelligence Risk Management Framework
  7. Google Cloud Architecture CenterChoose components for your agentic AI architecture

Continue the conversation

What does this change for your organisation?

Connect this perspective to the product, operation or technology foundation your organisation needs to change.

Talk to us