Thomas A. Alspaugh
Kinds of Software Quality (Ilities)

Various researchers and experts have produced lists of kinds of software quality, often informally called ilities since most of their names end in -ility.

Here we examine three lists of ilities, produced by three different groups of researchers.

Boehm, Brown, and Lipow's 23 Quality Characteristics (1976)

Why ilities are important

‘Suppose you receive a software product which is delivered on time, within budget, and which correctly and efficiently performs all its specified functions. Does it follow that you will be happy with it? For several reasons, the answer may be no. Here are some of the common problems you may find:

  1. The software product may be hard to understand and difficult to modify. This leads to excessive costs in software maintenance, and these costs are not trivial. For example, a recent paper by Elshoff (1) indicates that 75 percent of General Motors' software effort is spent in software maintenance, and that GM is fairly typical of large industry software activities.
  2. The software product may be difficult to use, or easy to misuse. A recent GAO report (2) identified over $10,000,000 in unnecessary Government costs due to ADP problems; many of them were because the software was so easy to misuse.
  3. The software product may be unnecessarily machine-dependent, or hard to integrate with other programs. This problem is difficult enough now, but as machine types continue to proliferate, it will get worse and worse.

[Boehm+Brown+Lipow1976-qesq p.592]

The 23 quality characteristics

The 23 characteristics are organized hierarchically in a tree, in which each parent's child nodes are sub-characteristics of the parent characteristic. The sub-characteristics are necessary but not sufficient for achieving the parent characteristic (so that for example Usability requires Reliability, Human Engineering, and Efficiency, but those three aren't sufficient by themselves to achieve Usability).

Because the characteristics are hierarchical, they are easier to think about and deal with than a flat list (such as that of Blundell, Hines, and Stach).

The leaf characteristics are also described as metrics, to be evaluated by an expert human being or (if possible) by a software tool.

Each characteristic (except for one present in the diagram but omitted in the text) is given an unusually full definition, compared to those of most authors.

Unfortunately, some of the definitions are dated and grounded in FORTRAN (such as that for Conciseness which talks of overlays), but an insightful reader can come up with the modern-day equivalent.

Boehm et al. ility tree

Figure 1. Boehm et al.'s Figure 1 (page 595), corrected to match the qualities named in the text.
Note: General Utility and Device Efficiency appear in the figure but are not defined in the text. [Original figure]

Code possesses the characteristic accessibility to the extent that it facilitates selective use of its parts. (Examples: variable dimensioned arrays, or not using absolute constants.) Accessibility is necessary for efficiency, testability, and human engineering.
Code possesses the characteristic accountability to the extent that its usage can be measured. This means that critical segments of code can be instrumented with probes to measure timing, whether specified branches are exercised, etc. Code used for probes is preferably invoked by conditional assembly techniques to eliminate the additional instruction words or added execution times when the measurements are not needed.
Code possesses the characteristic accuracy to the extent that its outputs are sufficiently precise to satisfy their intended use. Necessary for reliability.
Code possesses the characteristic augmentability to the extent that it can easily accommodate expansion in component computational functions or data storage requirements. This is a necessary characteristic for modifiability.
Code possesses the characteristic communicativeness to the extent that it facilitates the specification of inputs and provides outputs whose form and content are easy to assimilate and useful. Communicativeness is necessary for testability and human engineering.
Code possesses the characteristic completeness to the extent that all its parts are present and each part is fully developed. This implies that external references are available and required functions are coded and present as designed, etc.
Code possesses the characteristic conciseness to the extent that excessive information is not present. This implies that programs are not excessively fragmented into modules, overlays, functions and subroutines, nor that the same sequence of code is repeated in numerous places, rather than defining a subroutine or macro; etc.
Code possesses the characteristic internal consistency to the extent that it contains uniform notation, terminology and symbology within itself, and external consistency to the extent that the content is traceable to the requirements. Internal consistency implies that coding standards are homogeneously adhered to; e.g., comments should not be unnecessarily extensive or wordy at one place, and insufficiently informative at another, that number of arguments in subroutine calls match with subroutine header, etc. External consistency implies that variable names and definitions, including physical units, are consistent with a Glossary; or, there is a one-one relationship between functional flow chart entities and coded routines or modules, etc.
Code possesses the characteristic device-independence to the extent it can be executed on computer hardware configurations other than its current one. Clearly this characteristic is a necessary condition for portability.
Code possesses the characteristic efficiency to the extent that it fulfills its purpose without waste of resources. This implies that choices of source code constructions are made in order to produce the minimum number of words of object code, or that where alternate algorithms are available, those taking the least time are chosen; or that information-packing density in core is high, etc. Of course, many of the ways of coding efficiently are not necessarily efficient in the sense of being cost-effective, since portability, maintainability, etc., may be degraded as a result.
Code possesses the characteristic human engineering to the extent that it fulfills its purpose without wasting the users' time and energy, or degrading their morale. This characteristic implies accessibility, robustness, and communicativeness.
Code possesses the characteristic legibility to the extent that its function is easily discerned by reading the code. (Example: complex expressions have mnemonic variable names and parentheses even if unnecessary.) Legibility is necessary for understandability.
Code possesses the characteristic maintainability to the extent that it facilitates updating to satisfy new requirements or to correct deficiencies. This implies that the code is understandable, testable and modifiable; e.g., comments are used to locate subroutine calls and entry points, visual search for locations of branching statements and their targets is facilitated by special formats, or the program is designed to fit into available resources with plenty of margins to avoid major redesign, etc.
Code possesses the characteristic modifiability to the extent that it facilitates the incorporation of changes, once the nature of the desired change has been determined. Note the higher level of abstractness of this characteristic as compared with augmentability.
Code possesses the characteristic portability to the extent that it can be operated easily and well on computer configurations other than its current one. This implies that special language features, not easily available at other facilities, are not used; or that standard library functions and subroutines are selected for universal applicability, etc.
Code possesses the characteristic reliability to the extent that it can be expected to perform its intended functions satisfactorily. This implies that the program will compile, load, and execute, producing answers of the requisite accuracy; and that the program will continue to operate correctly, except for a tolerably small number of instances, while in operational use. It also implies that it is complete and externally consistent, etc.
Code possesses the characteristic robustness to the extent that it can continue to perform despite some violation of the assumptions in its specification. This implies, for example, that the program will properly handle inputs out of range, or in different format or type than defined, without degrading its performance of functions not dependent on the non-standard inputs.
Code possesses the characteristic self-containedness to the extent that it performs all its explicit and implicit functions within itself. Examples of implicit functions are initialization, input checking, diagnostics, etc.
Code possesses the characteristic self-descriptiveness to the extent that it contains enough information for a reader to determine or verify its objectives, assumptions, constraints, inputs, outputs, components, and revision status. Commentary and traceability of previous changes by transforming previous versions of code into non-executable but present (or available by macro calls) code are some of the ways of providing this characteristic. Self-descriptiveness is necessary for both testability and understandability.
Code possesses the characteristic structuredness to the extent that it possesses a definite pattern of organization of its interdependent parts. This implies that evolution of the program design has proceeded in an orderly and systematic manner, and that standard control structures have been followed in coding the program, etc.
Code possesses the characteristic testability to the extent that it facilitates the establishment of verification criteria and supports evaluation of its performance. This implies that requirements are matched to specific modules, or diagnostic capabilities are provided, etc.
Code possesses the characteristic understandability to the extent that its purpose is clear to the inspector. This implies that variable names or symbols are used consistently, modules of code are self-descriptive, and the control structure is simple or in accordance with a prescribed standard, etc.
Code possesses the characteristic usability to the extent that it is reliable, efficient and human-engineered. This implies that the function performed by the program is useful elsewhere, is robust against human errors (e.g., accepts either integer or real representations for type real variables), or does not require excessive core memory, etc.

[Boehm+Brown+Lipow1976-qesq pp. 603-605]

Cavano and McCall's 11 Quality Factors (1978)

Cavano and McCall ilities diagram

Cavano and McCall organize their quality factors by the lifecycle phase in which they are important for a developed system, namely product revision, product transition, and product operation.

They give a brief definition of each of their 13 factors, plus (most usefully) give a question for each factor, in the figure.

Does it do what I want?

Extent to which a program satisfies its specifications and fulfills the user's mission objectives.

Does it do it accurately all of the time?

Extent to which a program can be expected to perform its intended function with required precision.

Will it run on my hardware as well as it can?

The amount of computing resources and code required by a program to perform a function.

Is it secure?

Extent to which access to software or data by unauthorized persons can be controlled.

Can I run it?

Effort required to learn, operate, prepare input, and interpret output of a program.

Can I fix it?

Effort required to locate and fix an error in an operational program.

Can I change it?

Effort required to modify an operational program.

Can I test it?

Effort required to test a program to ensure it performs its intended function.

Will I be able to use it on another machine?

Effort required to transfer a program from one hardware configuration and/or software system environment to another.

Will I be able to reuse some of the software?

Extent to which a program can be used in other applications - related to the packaging and scope of the functions that programs perform.

Will I be able to interface it with another system?

Effort required to couple one system with another.


Blundell, Hines, and Stach's 39 Quality Measures (1997)

Blundell, Hines, and Stach list 39 quality measures, most defined in terms of one or more of 18 software characteristics, each defined in terms of one or more of seven design attributes.

Perhaps the most appealing feature of this list of ilities is how it is grounded (to some extent) in seven fundamental design attributes, most of which are common in the software design literature if not clearly defined, on which are organized the 18 software characteristics of source code written to the design described by the design attributes, all crowned by the 39 measures of the quality of the resulting running system.

Levels of BHS qualities

Organization of Blundell, Hines, and Stach's
quality measures, software characteristics,
and design attributes

But the lists could be said to be an example of dividing and defining software qualities to an extreme. Because the 39 measures are not organized or related to each other, in contrast to Boehm, Brown, and Lipow's hierarchy and Cavano and McCall's division by lifecycle phase, they are difficult to think about and work with. And the relationships among the three levels are not clear. One could argue that these authors have gone beyond the point of usefulness, at least for ordinary informal description of software requirements.

Blundell, Hines, and Stach's 7 Design attributes

The design attributes are intended by Blundell et al. to be measurable by software metrics, but are not very clearly defined in the paper. The design attributes are listed below along with quotations about each one.

COH: Cohesion
… cohesion measures the singularity of function of a single module.
Cohesion or average modular strength is a measure of local or internal modular complexity. Such module strength derives from the strength of the connections between program units.
In modular design, cohesion should be maximized while coupling is minimized.
Cohesion measures the singularity of purpose of a module.
COM: Program complexity
Not well defined; presumably the complexity within modules.
Cyclomatic complexity recognizes that compound predicates increase program complexity … and there is evidence to suggest a connection between decision nodes and complexity.
A measure of problem complexity needs to be derived from a functional specification. Currently, there is no method of measurement that establishes the degree of difficulty of a problem.
COU: Coupling
Coupling measures the simplicity of the connection between modules …
In modular design, cohesion should be maximized while coupling is minimized.
Coupling is maximized when the inter-connectivity of a module is minimized.
DAS: Data structures
The choice of data structure significantly affects the quality of a design. Specification languages may be used to express optimum data types based upon functional requirements. Optimal data structures should be derived from intelligent systems based upon experience analysis.
ITA: Intra-modular complexity
Not well defined; presumably the complexity within modules.
A lesser number of modules produce lower inter-modular complexity but higher intra-modular complexity.
ITE: Inter-modular complexity
Not well defined; presumably the complexity of the way modules are connected to each other.
A lesser number of modules produce lower inter-modular complexity but higher intra-modular complexity.
TOS: Token selection
Not well defined; presumably the number of distinct lexical tokens in the program code.
The vocabulary of a program is considered to be the sum of the number of unique operators and unique operands the program contains [Grier 1981]. Such operators and operands are collectively referred to as tokens [Shen et al. 1983]. Tokens may be considered to be atomic programming units.
Token selection relates to the number and variety of data values and functions needed for a given solution.

Blundell, Hines, and Stach's 18 Software Characteristics

Blundell, Hines, and Stach Table 3, p.243
# Software characteristic Design attributes
coh com cou das ita ite tos
1 Conciseness
2 Ease of change
3 Ease of checking conformance
4 Ease of coupling to other systems
5 Ease of introduction of new features
6 Ease of testing
7 Ease of understanding
8 Freedom from error
9 Functional independence of modules
10 Precise computations
11 Precise control
12 Shortest loops
13 Simplest arithmetic operators
14 Simplest data types
15 Simplest logic
16 Standard data types
17 Ease of maintenance
18 Functional specification compliance

Blundell, Hines, and Stach's 39 Quality Measures

Several of their quality measures are essentially indistinguishable (for example, Adaptability and Expandability).

Blundell, Hines, and Stach Table 1, p.236
1 Accuracy • precision of computations {10} and control {11}
• assessment of freedom from error {8}
2 Adaptability • ease of introduction of new features {5}
[part of Modifiability]
3 Auditability • ease of checking conformance to standards {3}
4 Availability • percentage of time that a program is operating according to requirements at a given point in time
5 Chang[e]ability • ease of program change {2}
[apparent synonym of Modifiability]
6 Completeness • degree of implementation of required function {18}
[degree of Correctness, in part]
7 Conciseness • [fewer] lines of code [for same function and quality] {1}
[Blundell et al. erroneously have 10 here]
8 Consistency • uniformity of design and documentation
9 Correctness • program satisfies specification {18}
• [program] meets user expectations {18}
• [program is] fault-free {8}
[cf. Completeness Utility]
10 Data commonality • use of standard data structures and types {14} {16}
11 Dependability = Reliability
12 Efficiency [these are all means of achieving the quality]
• simplify arithmetic and logical expressions {13} {15}
• shorten nested loops {12} {15}
• avoid multidimensional arrays {14}
• avoid pointers and complex lists {14}
• use fast arithmetic operations {13}
• do not mix data types {16}
• use integer arithmetic Boolean expressions {14}
• minimize I/O requests {9}
13 Error tolerance • [minimizing] damage occurring due to an error
[very similar to Robustness]
14 Expandability • degree to which program can be extended {2}
[closely related to Adaptability]
15 Flexibility • effort required to modify {2}
[closely related or identical to Changeability, inverse of Modifiability]
16 Functionality • capability, Generality, and security of system {18}
17 Generality • breadth of application {4} {9}
18 Hardware independence • degree of decoupling from hardware
19 Human factors • quality of the user interface
20 Integrity • control of access [preventing] unauthorized users
21 Interoperability • [minimizing] effort required to couple to other programs and systems {4}
22 Maintainability [these are time spans contributing to lack of the quality]
• problem recognition time {7} {17}
• administrative delay time {17}
• maintenance tools collection time {17}
• problem analysis time {7} {17}
• change specification time {2} {17}
• active correction time {17}
• local testing time {6} {17}
• global testing time {6} {17}
• maintenance review time {17}
• total recovery time {17}
23 Modifiability • ease of changing a program {2}
24 Modularity • functional independence of program components {9}
25 Operability • ease of operation {7}
26 Portability • [minimal] effort to transfer the program from one virtual machine to another {17}
27 Reliability • performance of intended function for a given period of time {18}
28 Reusability • ease of reference {7} {9}
• standardization {7}
• ease of integration {4}
• reuse of objects {9}
29 Robustness • ability to continue operation despite invalid inputs {18}
[very similar to error tolerance]
30 Security • availability of mechanisms that protect the program/data
[very similar to Integrity]
31 Self‑documentation • degree to which code provides documentation {7}
32 Simplicity • ease of understanding {7}
[almost identical to Understandability]
33 Supportability • ease of extending, adapting, and servicing a program {2}
[almost identical to Maintainability]
34 Testability • [minimal] effort required to test a program to ensure correctness
35 Traceability • ability to trace program components back to requirements
36 Transportability = Portability
37 Understandability • ease of understanding the function of a program and its functional implementation
38 Usability • [minimal] effort to learn, operate, prepare input, and interpret output
39 Utility • degree [to which the] program satisfies its intended function
[very similar to Correctness]

Distinguishing two ilities

How could we decide two qualities are distinct?


James Kenneth Blundell, Mary Lou Hines, and Jerrold Stach. The measurement of software design quality. Annals of Software Engineering, 4:235–255, 1997.
B. W. Boehm, J. R. Brown, and M. Lipow. Quantitative evaluation of software quality. In Second International Conference on Software Engineering (ICSE ’76), pages 592–605, 13–15 Oct. 1976.
Joseph P. Cavano and James A. McCall. A framework for the measurement of software quality. In software quality assurance workshop on Functional and performance issues, pages 133–139, 1978.

flip bgunflip