How to measure software quality – Starting out

Process quality has been very popular over the past two decades.  From CMM and ISO9001 to 6 sigma, Lean, BPM & beyond we have seen a large body of work produced on the premise that “the quality of the process largely determines the quality of the product”.

All of the models, methods and techniques used for improvement are based on this process-improvement concept.

But are we building the right quality?

What do people mean when they ask, nay demand, high quality products?  More often than not they mention “relatively free from defects” or “great functionality”.  And that’s where the majority of projects leave it.

For software, there is a standard model defined for product quality – ISO9126 (and it’s subsequent derivations).  It provides a set of characteristics which can enable the definition and assessment of various quality attributes of a software product.

Whilst the standard itself is still relatively unknown, most people are familiar with the concepts contained in it – the ‘ilities’:

  • Functionality
  • Reliability
  • Usability
  • Efficiency
  • Maintainability
  • Portability

And each of these is then broken down into more specific attributes.

Yet familiar as people might be with these concepts, they are generally not used – except in a binary sense (important or not e.g. portability not important in an embedded system).

The difficulty faced by most engineers is threefold:

  1. the metrics and indicators used to specify and assess the ‘ilities’ are generally not known
  2. the characteristics are specified in a generic manner and don’t really provide practical examples are how they can be applied
  3. they were created when procedural languages and transaction based systems were the focus – web and mobile based computing was not yet on the scene

So how do we improve?

By identifying 2 or 3 ‘ilities’ that are important to the project – we all know what they are intuitively – and then agreeing on some metrics and targets that provide a benchmark to assess product performance.

The ‘ilities’ are sometimes included in projects as non-functional requirements, but even then it is often not measured using metrics and is more qualitative e.g. “the system needs to be maintainable vrs the maintainability metric to be used is cyclomatic complexity and coherence index is to not exceed 15”.

This also makes it easier for developers to plan their design and test their code.  It also enables users to perform UAT and BRT more effectively.  Since the targets are quantified, you don’t receive comments like:

“hmmm…that seems ok, but maybe it could be a *bit* more maintainable”   😉

This entry was posted in agile, BPMN, CMMI, defects, Effectiveness & Efficiency, improvement, ISO9001, ISO9126, KPI, method, metrics, process improvement, product quality, productivity, quality, software quality and tagged , , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s