Functional testing can show whether a digital service works as expected, but passing those tests does not on its own mean the service is high quality.
A service can function as expected but still be difficult to use with assistive technology, slow when demand increases, hard to recover when something fails or risky to change.
Quality is shaped by decisions made throughout the life of a service: what we design, how we build, how we release, what we measure and how we respond when something changes.
Building quality in early is not just about reducing defects. It helps teams identify risks when change is easier to make, giving leaders greater confidence and reducing the likelihood of more costly or disruptive problems later.
To help teams think about quality consistently, the cross-government testing community has developed a set of Quality Engineering Principles.
A shared starting point
The creation of the principles started with a workshop bringing together testing and quality engineering practitioners from across government. That conversation highlighted a common challenge: teams were often working towards similar quality outcomes, but we did not have a shared set of principles to describe the approach.
Since then, an active cross-government working group has developed and refined the principles, bringing together experience from different organisations, services and delivery contexts.
We have brought that work together into seven Quality Engineering Principles. These principles set out a shared approach: engineer quality into the way we provide services, take a holistic view of quality, define it around user needs and outcomes, apply effort proportionate to risk, automate repeatable activities, shorten feedback loops and use evidence to make quality measurable, visible and actionable.
The principles are currently in alpha, which means they are a starting point and will continue to evolve. We want teams across government and the wider public sector to try them in real work, challenge them and help us improve them. You can share feedback and suggest improvements directly through the GitHub space.
Quality is more than whether something works
Functional correctness matters, but it is only one part of quality. For a public service, quality can also mean being accessible, secure, resilient and efficient. It should be easy to operate, maintain and change safely. The characteristics that matter most will depend on the service, its users and the consequences if something goes wrong.
This changes the question teams need to ask. Instead of starting with ‘What testing do we need?’, we can start with: ‘What does quality mean for our users and this service, and what evidence will give us confidence that we are achieving it?’. Then teams must consider the wider technical and operational characteristics needed to support them.
For leaders and service owners, this means creating the conditions for teams to make good quality decisions: being clear about the outcomes that matter, understanding risk, and using evidence to know where attention and investment are needed.
Quality is created across the team
Many decisions that determine service quality are not testing decisions. Product teams decide which outcomes matter most. Designers influence usability and accessibility. Engineering and architecture choices affect reliability, security, performance and how easily a service can change. Operational practices determine how quickly teams can detect and recover from problems. Quality engineers bring expertise in areas such as risk analysis, test design, automation and assurance.
Quality engineering therefore works best as part of multidisciplinary delivery. These principles are a way to take the quality conversation beyond testing and quality engineering. They give the wider digital and data community a shared language for thinking about quality and reinforce that it is shaped by decisions across the whole team.
Get useful feedback earlier and more often
One of the strongest ways to improve those decisions is to shorten the time between making a change and learning about its effect. That feedback might come from automated tests, accessibility checks, security controls, performance testing, monitoring, operational data or what we learn from users.
Automation helps where activities are repeatable and where automating them provides value. But automation itself is not the outcome. The outcome is useful feedback at the point where people can act on it.
The same applies to measurement. More dashboards and metrics do not automatically create confidence. Evidence needs to help teams understand whether a service is healthy, where risks are increasing and whether it is providing the outcomes users need.
Apply effort where it matters most
Building quality into delivery does not mean applying the same controls to every change. Different services and changes carry different risks. Teams should consider factors such as complexity, the likelihood of failure and the potential effect on users or the service. Testing and assurance can then be proportionate to that risk. This helps concentrate effort where it provides the most value and confidence rather than creating unnecessary gates.
What happens next
Our aim is to make quality decisions more deliberate, evidence-led and focused on the people who rely on our services. The principles are now available for feedback, and we want to learn from how teams use them in practice. Share your feedback and suggested improvements through the GitHub space. We’ll review what we hear, refine the principles where needed and share updates as they evolve.
Leave a comment