This note describes a novel approach to improving the productivity of people who design, develop, and maintain computer software. The productivity of these people is important because overruns in cost and delivery time for software are common, quality is low, and maintenance cost is often greater than that of the original development. Barry Boehm, at the 1973 Tri-Service Symposium on the High Cost of Software held in Monterey, California, estimated the 1972 Air Force software costs to have been between $1 billion and $1.5 billion, while hardware costs were in the range of $0.3 to $0.4 billion. He predicted the ratio of software to hardware costs will increase until it reaches a level of about 9 to 1 in the 1980s because hardware costs will continue to go down while the costs of personnel increase.1a

We believe that by making more and better tools easily available to software engineers, by making the tools consistent with each other, and by developing better techniques and methodologies, we can reduce the costs associated with the software life cycle and improve the quality of the products. This note describes a method for achieving these goals: by building upon the Augmentation Research Center's NLS system, by defining and implementing a few new tools, and by providing consistent access to already existing tools.1b

A number of tools and techniques are currently available to assist in the various phases of the software life cycle. We use the term 'tool' to refer to a software system or program that helps someone develop other systems or programs, much as a hammer is a tool used to build a house. We use the terms 'techniques' and 'methods' interchangably to refer to the manner in which the tools are applied. The term 'methodology' is used to refer to a systematic use of a collection of tools and techniques. Finally, we use the term 'environment' to refer not only to the collection of software tools, techniques, and methodologies available to a software engineer, but also to the people with whom the software engineer interacts, and the hardware tools and systems available (such as computers and terminals).1c

The software life cycle, as defined by Boehm (in the December 1976 issue of the IEEE Transactions on Computers), consists of the following seven phases:2a

System requirements
2a1
Software requirements
2a2
Preliminary design
2a3
Detailed design
2a4
Coding and debugging
2a5
Test and preoperations
2a6
Operation and maintenance.
2a7

In addition to these distinct phases, there are threads that pervade the entire life cycle: the management and control of people, resources, and the cycle itself; the production and distribution of documentation; and the training of designers, developers, maintainers, and end users.2b

The later in the life cycle an error is detected, the more expensive it is to correct. Or conversely, the more effort that is expended in the frontend of the cycle (Phases 1-4), the less time and funding are necessary for implementation and integration (Phases 5 and 6) and for operation and maintenance (Phase 7).2c

A number of tools and techniques are already available to aid in all phases of the software life cycle: 2d

Formal languages exist for specifying system and software requirements, and tools are available to validate these requirements to ensure their completeness and correctness;2d1

Formal languages exist for specifying the preliminary and detailed design of a program, and tools are available to prove the completeness and correctness of these designs;2d2

High-level languages for implementation are abundant, and a number of other tools and techniques, such as the use of program support libraries and the use of programming teams, help the implementation process; and2d3

A wide variety of debuggers are available, including systems for automatic testing of programs, automatic generation of data with which to test programs, symbolic program execution, and automatic correctness proving of programs. 2d4

Until there exists one tool that will automatically generate a program given the end users' requirements (stated loosely and in a natural language), no single tool will be sufficient to guarantee correct, error-free, software to meet all the users' (changing) requirements and intents. Additionally, many existing tools have both practical and theoretical limits as to what each can accomplish. For example, many program proving tools are capable of proving only small to moderate length programs, and only when such programs are written in highly specialized languages. Tools that exercise all statements or control paths of a program, in addition to having many of the above practical limitations, are theoretically incapable of determining the absence of a necessary control path.2e

Thus, for the present, and the foreseeable future, we are left with the need for a number of tools and techniques. This collection is needed for each individual phase of the life cycle and for the threads that pervade the cycle itself. Current practices (for the most part) consist of the haphazard application of the tools known to local groups of programmers. What is needed is the widespread knowledge about existing tools, and a well-defined methodology for applying these for maximum payoff.2f

It is not sufficient merely to provide a wide spectrum of tools. The tools should be used. A tool might not be used for many reasons: it may not solve the problem it was designed to attack; it may create more problems than it solves; it may be too expensive to use; it may be too difficult to use and/or learn; it may be solving a problem that is not perceived by those who must use it (e.g., the imposition of a tool on programmers by a manager). Conversely, we may define the ideal tool as one that solves a need perceived by the users of the tool without creating any new problems and is cheap to use. The difficulties associated with learning and using it should be commensurate with the perceived payoffs to the user.2g

If a multiplicity of tools is needed, then these tools certainly should complement each other. The results produced by a tool that assists in one phase of the cycle should be directly usable by other tools that assist in the same phase and by the tools that support the next phase. Ideally, the techniques and discipline would be the same for using all tools. Thus, knowing how to use one tool, would be almost sufficient for using all tools. The specifics of a new tool may be different but the discipline would be the same. This might be analagous to increasing one's vocabulary (a relatively easy task) rather than having to learn an entirely new language (a relatively more difficult task). 2h

A large project consisting of a number of modules and being worked on by a number of individuals requires tools just to keep track of who is doing what to which module and where in the cycle each module currently is. Such information ought to be maintained in a data base associated with the project. The individual tools that support specific phases of the cycle could then interact with this common "project data base" to indicate where a module currently is in the life cycle. It is easy to imagine other tools that would then interact with this data base to extract information such as who is working on what module, what percentage of the system has been designed, what percentage of the system has had its module specifications verified, what percentage of the system has been coded, etc.2i

We propose to define, implement, and study a software engineering environment containing a collection of consistent and complimentary tools, technigues, methodologies, and supporting data bases.2j

The tasks required to provide a software engineering environment fall roughly into the following three areas:3a

Defining and implementing the environment;3a1

Providing the tools that will be used by the software engineers; and3a2

Providing consistent access to the above tools from the software engineers' environment, and ensuring that any results produced by one tool are useable by other tools, including the project data base maintenance tool.3a3

We intend to build upon the NLS system (see below) to provide the framework for a software engineering environment. NLS was designed and implemented with its own evolution as one of its primary goals. This fact will make it possible to expand NLS to provide the framework for a software engineering environment. NLS already provides many of the capabilities that would be needed in such an environment and thus by using NLS we reduce the number of new tools that need to be developed.3b

We intend to define a project data base and design and implement the tools for the maintenance and management of project data bases.3c

Since many good tools already exist to assist in all phases of the software life cycle, we do not intend to redevelop such tools. Rather, we intend to adopt existing tools into our environment by using NLS as a frontend for these tools. This approach will ensure consistent access to these tools. A means will be defined for using NLS as a frontend system and for capturing the results of a tool for incorporation into the appropriate project data base. Ultimately, tools will be modified and designed to fit cleanly in the environment.3d

NLS provides a wide spectrum of tools and techniques for the creation, viewing, editing, and dissemination of textual and pictorial information. At the user level, NLS is organized into a number of subsystems. Each subsystem contains a number of related commands. Although the semantics vary widely from subsystem to subsystem, the syntactic rules for all commands are the same, and the user interaction discipline is the same for all subsystems.4b

NLS is an extendable system. By writing new subsystems, individual users can extend the basic capabilities of NLS to meet the needs of specific goals. Those user generated extensions that have been found to be of general utility have been incorporated into basic NLS and thus made available to a wide number of users. In writing a new subsystem, the user takes advantage of a number of tools already part of NLS. The use of these tools ensures that the new subsystems will continue to provide a consistent and coherent user interface.4c

Internally, NLS is organized into a number of hierarchically structured modules. At the most basic organizational level, NLS is split into two distinct modules: a Frontend (FE) for all interaction with a user; and a Backend (BE) for actually performing the commands specified by the user in his/her interaction with the FE. Communication across the FE-BE interface is supported by protocol modules in both the FE and the BE. This organization allows the FE to be executing on a machine close to the user (to provide responsive command interaction and feedback) and allows the BE to be executing on a different (perhaps more powerful, perhaps geographically remote) machine.4d

The FE and BE are themselves composed of a number of functional modules. For example, the FE consists of a Command Language Interpreter (CLI) for parsing user-specified commands, a virtual terminal handling module, the above-mentioned protocol module, an Operating System Interface (OSI) module for performing operating system and machine dependent operations, a physical terminal handling module, and other modules. The BE includes the above mentioned protocol module, an Operating System Interface (OSI) module, a file system module for handling the NLS hierarchical file system, a formatting module for formatting the information to be displayed to the user, a subsystem backend module for each supported subsystem, and other modules. 4e

Most of these modules are themselves divided into a number of different levels. At the topmost levels are those routines and data structures that are available for use by other modules. These routines and data structures maintain a constant interface to the outside world. Underneath these levels are the necessary supporting data structures and routines.4f

This internal structure of NLS, i.e., the organization into a number of functional modules, with each module divided into a number of levels, has proved to be valuable for the maintenance and development of a large system -- NLS itself -- that is under constant evolution by a variety of individuals. It also provides a readily available wealth of existing code upon which users can build to create their own special purpose subsystems. The evolution of NLS has included the continual addition of new features and subsystems, more efficient implementation, and the transporting of NLS to a number of different computers and operating systems.4g

NLS is and has been used extensively by the software engineers at ARC. It is used for both its own maintenance and development as well as for the maintenance and development of other systems developed by ARC. In addition to the many features of NLS itself which support software engineers, ARC has developed a number of tools and techniques to assist in the software life cycle. Some of the capabilities and features of NLS, available to all users, and particularily useful for software engineers, include:5a

In addition to the capabilities of NLS useful for all information workers, there are a number of facilities specifically for software engineers. These include:5b

Other tools and techniques used by ARC software engineers include: meta-compilers; an informal team approach to programming with informal walk throughs; standards for coding styles and for documenting changes to NLS; a feedback system for dealing with user suggestions and complaints; the exclusive use of high level languages.5c

Most software groups have their own sets of tools, techniques, and methodologies. What is unique about ARC is that it provides a systematic approach to the problems. New tools and techniques are not developed in a vacuum; rather, they are designed with the goal of being incorporated in the existing system. The payoffs have been obvious: tools are used readily (and thus costs reduced) because they fit in naturally with the other tools the users are accustomed to using and fulfill needs perceived by these users.5d

We propose to use NLS as the basic framework upon which to build a software engineers' environment. We need to examine and determine how NLS has to be expanded in order to enable better support of this environment.5e

The tools and techniques developed and in use at ARC are heavily oriented towards the later phases of the software life cycle. Other groups (at SRI and elsewhere) have complementary (and competitive) tools and techniques, e.g., specification languages and program proving tools and techniques. We need to be able to provide consistent access to these tools and techniques. Such tools may at first be encapsulated under NLS to demonstrate both the feasibility and desireability of providing a coherent environment at the possible expense of efficiency. This would involve using NLS as a frontend for these tools. Later, existing tools may be re-written to mesh more closely with the environment, and we would expect that tools would be written explicitly for the environment.6a

Central to the concept of a software engineers' environment is the concept of the project data base to keep track of the status of the modules comprising a system. All tools in the environment should either participate in the generation of this data base or take advantage of the information in the base. Information is often easily available only at the time a tool (e.g., a compiler) is executed. Thus, tools should either interact directly with this data base, or the act of encapsulating a tool should include capturing data known to the tool.6b

A rich environment for software engineers can decrease the costs associated with the software life cycle, while at the same time providing higher quality products. We propose to provide the basic framework for such an environment by expanding and building upon the NLS environment. We propose to define a means for incorporating existing tools and techniques into this environment so that these tools can all be accessed in a consistent and coherent manner. We will initially encapsulate a few tools developed by groups other than ARC to demonstrate the feasibility and benefits of our approach. Finally, we propose to define the project data base, and the means for interfacing to this data base. We are not proposing to develop any new tools, other than those for the maintenance and manipulation of the project data base. We feel that there is already a wealth of existing tools, and that much can be gained by providing access to a wide variety of tools in one consistent environment.7a