• Home
• Foundations
• ⁘
• Mathematical
  • Sets
  • Relations
  • Correspondences
  • Ordered Sets
  • Lattices
  • Graphs
  • Powersets
  • Cartesian Products
  • Binary Strings
  • aia
  • Greek &c.
• Logic
  • Logic
  • Math. Induction
  • Strong Induction
  • Recurs. Defs., Derivs.
  • Structural Induction
  • Regular Expressions
  • Protégé 4
  • Macro Processing
• Writing/Thinking
  • External Glossary
  • Abstracts
  • Argument
  • Formality
  • Inquiry Cycle
  • Legal Relations
  • LATEX
  • Negotiation
  • Presentations
  • Quotations
  • Teamwork
• SWD
  • Quotations
  • Kinds
  • Architecture
  • Data Flow
  • Evolution
  • IP
  • Object Orientation
  • Process
  • Reviewing
  • Testing
• Requirements
  • Kinds
  • Quick Start
  • Glossaries
  • Goals
  • i*
  • Ilities
  • Narratives
  • SCR
  • Tracing
• Specifications
  • Alloy
  • E-R
  • MSCs
  • Statecharts
  • UML
• Java
  • Coding Standards
  • Design Patterns
  • Javadoc
  • JUnit
  • Nested Classes
  • Packages
  • Types
• Web
  • (X)HTML
  • XML Schemas
  • XSLT
  • MathML

Various researchers and experts have produced lists of kinds of software quality, often informally called ilities since most of their names end in -ility.

Here we examine three lists of ilities, produced by three different groups of researchers.

Boehm, Brown, and Lipow's 23 Quality Characteristics (1976)

Why ilities are important

‘Suppose you receive a software product which is delivered on time, within budget, and which correctly and efficiently performs all its specified functions. Does it follow that you will be happy with it? For several reasons, the answer may be no. Here are some of the common problems you may find:

1. The software product may be hard to understand and difficult to modify. This leads to excessive costs in software maintenance, and these costs are not trivial. For example, a recent paper by Elshoff (1) indicates that 75 percent of General Motors' software effort is spent in software maintenance, and that GM is fairly typical of large industry software activities.
2. The software product may be difficult to use, or easy to misuse. A recent GAO report (2) identified over $10,000,000 in unnecessary Government costs due to ADP problems; many of them were because the software was so easy to misuse.
3. The software product may be unnecessarily machine-dependent, or hard to integrate with other programs. This problem is difficult enough now, but as machine types continue to proliferate, it will get worse and worse.

[Boehm+Brown+Lipow1976-qesq p.592]

The 23 quality characteristics

The 23 characteristics are organized hierarchically in a tree, in which each parent's child nodes are sub-characteristics of the parent characteristic. The sub-characteristics are necessary but not sufficient for achieving the parent characteristic (so that for example Usability requires Reliability, Human Engineering, and Efficiency, but those three aren't sufficient by themselves to achieve Usability).

Because the characteristics are hierarchical, they are easier to think about and deal with than a flat list (such as that of Blundell, Hines, and Stach).

The leaf characteristics are also described as metrics, to be evaluated by an expert human being or (if possible) by a software tool.

Each characteristic (except for one present in the diagram but omitted in the text) is given an unusually full definition, compared to those of most authors.

Unfortunately, some of the definitions are dated and grounded in FORTRAN (such as that for Conciseness which talks of overlays), but an insightful reader can come up with the modern-day equivalent.

Figure 1. Boehm et al.'s Figure 1 (page 595), corrected to match the qualities named in the text.
Note: General Utility and Device Efficiency appear in the figure but are not defined in the text. [Original figure]

ACCESSIBILITY

Code possesses the characteristic accessibility to the extent that it facilitates selective use of its parts. (Examples: variable dimensioned arrays, or not using absolute constants.) Accessibility is necessary for efficiency, testability, and human engineering.

ACCOUNTABILITY

Code possesses the characteristic accountability to the extent that its usage can be measured. This means that critical segments of code can be instrumented with probes to measure timing, whether specified branches are exercised, etc. Code used for probes is preferably invoked by conditional assembly techniques to eliminate the additional instruction words or added execution times when the measurements are not needed.

ACCURACY

Code possesses the characteristic accuracy to the extent that its outputs are sufficiently precise to satisfy their intended use. Necessary for reliability.

AUGMENTABILITY

Code possesses the characteristic augmentability to the extent that it can easily accommodate expansion in component computational functions or data storage requirements. This is a necessary characteristic for modifiability.

COMMUNICATIVENESS

Code possesses the characteristic communicativeness to the extent that it facilitates the specification of inputs and provides outputs whose form and content are easy to assimilate and useful. Communicativeness is necessary for testability and human engineering.

COMPLETENESS

Code possesses the characteristic completeness to the extent that all its parts are present and each part is fully developed. This implies that external references are available and required functions are coded and present as designed, etc.

CONCISENESS

Code possesses the characteristic conciseness to the extent that excessive information is not present. This implies that programs are not excessively fragmented into modules, overlays, functions and subroutines, nor that the same sequence of code is repeated in numerous places, rather than defining a subroutine or macro; etc.

CONSISTENCY

Code possesses the characteristic internal consistency to the extent that it contains uniform notation, terminology and symbology within itself, and external consistency to the extent that the content is traceable to the requirements. Internal consistency implies that coding standards are homogeneously adhered to; e.g., comments should not be unnecessarily extensive or wordy at one place, and insufficiently informative at another, that number of arguments in subroutine calls match with subroutine header, etc. External consistency implies that variable names and definitions, including physical units, are consistent with a Glossary; or, there is a one-one relationship between functional flow chart entities and coded routines or modules, etc.

DEVICE-INDEPENDENCE

Code possesses the characteristic device-independence to the extent it can be executed on computer hardware configurations other than its current one. Clearly this characteristic is a necessary condition for portability.

EFFICIENCY

Code possesses the characteristic efficiency to the extent that it fulfills its purpose without waste of resources. This implies that choices of source code constructions are made in order to produce the minimum number of words of object code, or that where alternate algorithms are available, those taking the least time are chosen; or that information-packing density in core is high, etc. Of course, many of the ways of coding efficiently are not necessarily efficient in the sense of being cost-effective, since portability, maintainability, etc., may be degraded as a result.

HUMAN ENGINEERING

Code possesses the characteristic human engineering to the extent that it fulfills its purpose without wasting the users' time and energy, or degrading their morale. This characteristic implies accessibility, robustness, and communicativeness.

LEGIBILITY

Code possesses the characteristic legibility to the extent that its function is easily discerned by reading the code. (Example: complex expressions have mnemonic variable names and parentheses even if unnecessary.) Legibility is necessary for understandability.

MAINTAINABILITY

Code possesses the characteristic maintainability to the extent that it facilitates updating to satisfy new requirements or to correct deficiencies. This implies that the code is understandable, testable and modifiable; e.g., comments are used to locate subroutine calls and entry points, visual search for locations of branching statements and their targets is facilitated by special formats, or the program is designed to fit into available resources with plenty of margins to avoid major redesign, etc.

MODIFIABILITY

Code possesses the characteristic modifiability to the extent that it facilitates the incorporation of changes, once the nature of the desired change has been determined. Note the higher level of abstractness of this characteristic as compared with augmentability.

PORTABILITY

Code possesses the characteristic portability to the extent that it can be operated easily and well on computer configurations other than its current one. This implies that special language features, not easily available at other facilities, are not used; or that standard library functions and subroutines are selected for universal applicability, etc.

RELIABILITY

Code possesses the characteristic reliability to the extent that it can be expected to perform its intended functions satisfactorily. This implies that the program will compile, load, and execute, producing answers of the requisite accuracy; and that the program will continue to operate correctly, except for a tolerably small number of instances, while in operational use. It also implies that it is complete and externally consistent, etc.

ROBUSTNESS

Code possesses the characteristic robustness to the extent that it can continue to perform despite some violation of the assumptions in its specification. This implies, for example, that the program will properly handle inputs out of range, or in different format or type than defined, without degrading its performance of functions not dependent on the non-standard inputs.

SELF-CONTAINEDNESS

Code possesses the characteristic self-containedness to the extent that it performs all its explicit and implicit functions within itself. Examples of implicit functions are initialization, input checking, diagnostics, etc.

SELF-DESCRIPTIVENESS

Code possesses the characteristic self-descriptiveness to the extent that it contains enough information for a reader to determine or verify its objectives, assumptions, constraints, inputs, outputs, components, and revision status. Commentary and traceability of previous changes by transforming previous versions of code into non-executable but present (or available by macro calls) code are some of the ways of providing this characteristic. Self-descriptiveness is necessary for both testability and understandability.

STRUCTUREDNESS

Code possesses the characteristic structuredness to the extent that it possesses a definite pattern of organization of its interdependent parts. This implies that evolution of the program design has proceeded in an orderly and systematic manner, and that standard control structures have been followed in coding the program, etc.

TESTABILITY

Code possesses the characteristic testability to the extent that it facilitates the establishment of verification criteria and supports evaluation of its performance. This implies that requirements are matched to specific modules, or diagnostic capabilities are provided, etc.

UNDERSTANDABILITY

Code possesses the characteristic understandability to the extent that its purpose is clear to the inspector. This implies that variable names or symbols are used consistently, modules of code are self-descriptive, and the control structure is simple or in accordance with a prescribed standard, etc.

USABILITY (AS-lS UTILITY)

Code possesses the characteristic usability to the extent that it is reliable, efficient and human-engineered. This implies that the function performed by the program is useful elsewhere, is robust against human errors (e.g., accepts either integer or real representations for type real variables), or does not require excessive core memory, etc.

[Boehm+Brown+Lipow1976-qesq pp. 603-605]

Cavano and McCall's 11 Quality Factors (1978)

Cavano and McCall organize their quality factors by the lifecycle phase in which they are important for a developed system, namely product revision, product transition, and product operation.

They give a brief definition of each of their 13 factors, plus (most usefully) give a question for each factor, in the figure.

CORRECTNESS

Does it do what I want?

Extent to which a program satisfies its specifications and fulfills the user's mission objectives.

RELIABILITY

Does it do it accurately all of the time?

Extent to which a program can be expected to perform its intended function with required precision.

EFFICIENCY

Will it run on my hardware as well as it can?

The amount of computing resources and code required by a program to perform a function.

INTEGRITY

Is it secure?

Extent to which access to software or data by unauthorized persons can be controlled.

USABILITY

Can I run it?

Effort required to learn, operate, prepare input, and interpret output of a program.

MAINTAINABILITY

Can I fix it?

Effort required to locate and fix an error in an operational program.

FLEXIBILITY

Can I change it?

Effort required to modify an operational program.

TESTABILITY

Can I test it?

Effort required to test a program to ensure it performs its intended function.

PORTABILITY

Will I be able to use it on another machine?

Effort required to transfer a program from one hardware configuration and/or software system environment to another.

REUSABILITY

Will I be able to reuse some of the software?

Extent to which a program can be used in other applications - related to the packaging and scope of the functions that programs perform.

INTEROPERABILITY

Will I be able to interface it with another system?

Effort required to couple one system with another.

[Cavano+McCall1978-fmsq]

Blundell, Hines, and Stach's 39 Quality Measures (1997)

Blundell, Hines, and Stach list 39 quality measures, most defined in terms of one or more of 18 software characteristics, each defined in terms of one or more of seven design attributes.

Perhaps the most appealing feature of this list of ilities is how it is grounded (to some extent) in seven fundamental design attributes, most of which are common in the software design literature if not clearly defined, on which are organized the 18 software characteristics of source code written to the design described by the design attributes, all crowned by the 39 measures of the quality of the resulting running system.

Organization of Blundell, Hines, and Stach's
quality measures, software characteristics,
and design attributes

But the lists could be said to be an example of dividing and defining software qualities to an extreme. Because the 39 measures are not organized or related to each other, in contrast to Boehm, Brown, and Lipow's hierarchy and Cavano and McCall's division by lifecycle phase, they are difficult to think about and work with. And the relationships among the three levels are not clear. One could argue that these authors have gone beyond the point of usefulness, at least for ordinary informal description of software requirements.

Blundell, Hines, and Stach's 7 Design attributes

The design attributes are intended by Blundell et al. to be measurable by software metrics, but are not very clearly defined in the paper. The design attributes are listed below along with quotations about each one.

COH: Cohesion

… cohesion measures the singularity of function of a single module.
Cohesion or average modular strength is a measure of local or internal modular complexity. Such module strength derives from the strength of the connections between program units.
In modular design, cohesion should be maximized while coupling is minimized.
Cohesion measures the singularity of purpose of a module.

COM: Program complexity

Not well defined; presumably the complexity within modules.
Cyclomatic complexity recognizes that compound predicates increase program complexity … and there is evidence to suggest a connection between decision nodes and complexity.
A measure of problem complexity needs to be derived from a functional specification. Currently, there is no method of measurement that establishes the degree of difficulty of a problem.

COU: Coupling

Coupling measures the simplicity of the connection between modules …
In modular design, cohesion should be maximized while coupling is minimized.
Coupling is maximized when the inter-connectivity of a module is minimized.

DAS: Data structures

The choice of data structure significantly affects the quality of a design. Specification languages may be used to express optimum data types based upon functional requirements. Optimal data structures should be derived from intelligent systems based upon experience analysis.

ITA: Intra-modular complexity

Not well defined; presumably the complexity within modules.
A lesser number of modules produce lower inter-modular complexity but higher intra-modular complexity.

ITE: Inter-modular complexity

Not well defined; presumably the complexity of the way modules are connected to each other.
A lesser number of modules produce lower inter-modular complexity but higher intra-modular complexity.

TOS: Token selection

Not well defined; presumably the number of distinct lexical tokens in the program code.
The vocabulary of a program is considered to be the sum of the number of unique operators and unique operands the program contains [Grier 1981]. Such operators and operands are collectively referred to as tokens [Shen et al. 1983]. Tokens may be considered to be atomic programming units.
Token selection relates to the number and variety of data values and functions needed for a given solution.

Blundell, Hines, and Stach's 18 Software Characteristics

Blundell, Hines, and Stach's 39 Quality Measures

Several of their quality measures are essentially indistinguishable (for example, Adaptability and Expandability).

Distinguishing two ilities

How could we decide two qualities are distinct?

• If different stakeholders benefit from each of the two qualities
• If the effect of the two qualities are seen in different phases of the lifecycle
• If the qualities are manifested in two different ways
• If there are systems or situations for which only one of the two qualities might be desirable

References

Blundell+Hines+Stach1997-msdq

James Kenneth Blundell, Mary Lou Hines, and Jerrold Stach. The measurement of software design quality. Annals of Software Engineering, 4:235–255, 1997.

Abstract

Software quality involves the conformance of a software product to some predefined set of functional requirements at a specified level of quality. The software is considered valid when it conforms to these “quality factors” at some acceptable level. There are a large number of quality factors against which software may be validated. This paper discusses the development of traditional software metrics in relation to the anticipated structure of a software system. The taxonomy of a software system primarily relies upon the dissection of the software system into modules. Modular design is the cornerstone of quality software, and metrics that can predict an optimum modular structure are critical. By examining the theoretical bases on quality metrics, a base set of common quantitative metrics can be devised and mapped to quality metrics in which they reside. This paper surveys existing metrics and suggests the derivation of software design metrics from software quality factors. Measurable software attributes are identified and suggested as potential design metrics.

doi:10.1023/A:1018914711050

Boehm+Brown+Lipow1976-qesq

B. W. Boehm, J. R. Brown, and M. Lipow. Quantitative evaluation of software quality. In Second International Conference on Software Engineering (ICSE ’76), pages 592–605, 13–15 Oct. 1976.

Abstract

The study reported in this paper establishes a conceptual framework and some key initial results in the analysis of the characteristics of software quality. Its main results and conclusions are:

• Explicit attention to characteristics of software quality can lead to significant savings in software life-cycle costs.
• The current software state-of-the-art imposes specific limitations on our ability to automatically and quantitatively evaluate the quality of software.
• A definitive hierarchy of well-defined, well-differentiated characteristics of software quality is developed. Its higher-level structure reflects the actual uses to which software quality evaluation would be put; its lower-level characteristics are closely correlated with actual software metric evaluations which can be performed.
• A large number of software quality-evaluation metrics have been defined, classified, and evaluated with respect to their potential benefits, quantifiability, and ease of automation.
• Particular software life-cycle activities have been identified which have significant leverage on software quality.

Most importantly, we believe that the study reported in this paper provides for the first time a clear, well-defined framework for assessing the often slippery issues associated with software quality, via the consistent and mutually supportive sets of definitions, distinctions, guidelines, and experiences cited. This framework is certainly not complete, but it has been brought to a point sufficient to serve as a viable basis for future refinements and extensions.

url

Cavano+McCall1978-fmsq

Joseph P. Cavano and James A. McCall. A framework for the measurement of software quality. In software quality assurance workshop on Functional and performance issues, pages 133–139, 1978.

Abstract

Research in software metrics incorporated in a framework established for software quality measurement can potentially provide significant benefits to software quality assurance programs. The research described has been conducted by General Electric Company for the Air Force Systems Command Rome Air Development Center. The problems encountered defining software quality and the approach taken to establish a framework for the measurement of software quality are described in this paper.

doi:10.1145/800283.811113

• thomas · alspaugh @ acm · org
• 
• 2019May11Sa10:37
• Attribution