Benifits of Implementing Level 2 Capability Maturity Model Processes with the Personal Software Process
Author: John Frankovich
Advanced Information Services
1605 Candletree Drive Suite 114
Peoria, Il 61614
Abstract
This paper describes the practical application of the Personal Software Process (PSP) in supporting the implementation of the Capability Maturity Model (CMM). The CMM is a process maturity framework designed to improve an organization’s process capability [Paulk 1993]. The PSP is a proven technique to improve the individual engineer’s performance and productivity [Humphrey 1995]. The benefits of using the bottom-up PSP methodologies to supplement CMM Level 2 Key Process Areas (KPAs) are described.
Content
1.0 Overview of the Capability Maturity Model(CMM)
2.0 Overview of the Personal Software Process(PSP)
3.0 Supplementing the CMM with the PSP
4.0 A PSP based Software Project Planning Process
5.0 A PSP based Software Project Tracking and Oversight Process
6.0 Benefits of the PSP-CMM approach
7.0 Conclusions
Appendix A: References
1.0 Overview of the Capability Maturity Model(CMM)
After two decades of unfulfilled promises about productivity and quality gains from applying new software methodologies and technologies, industry and government organizations are realizing that their fundamental problem is the inability to manage the software process [DoD 1987]. In 1987, the Software Engineering Institute (SEI) designed an organization assessment model that later developed into the Capability Maturity Model (CMM) [Paulk 1993].
The CMM is a framework that characterizes an evolutionary process improvement path toward a more mature organization. An organization can use the CMM to determine their current state of software process maturity and then to establish priorities for improvement. An organization's current state of maturity can be categorized as Initial, Repeatable, Defined, Managed, or Optimizing.
Initial: The software process is characterized as ad hoc and occasionally even chaotic. Few process are defined and success depends on individual effort
Repeatable: Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier success on projects with similar applications.
Defined: The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organizations. All projects use an approved, tailored version of the organization's standard process for developing and maintaining software.
Managed: Detailed measures of software process and product quality are collected. Both software process and products are quantitatively understood and controlled.
Optimizing: Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies [Humphrey 1995].
The following chart graphically displays the relationships between each of the CMM maturity levels and also shows the Key Process Areas (KPA) associated with each level:
The CMM defines five maturity levels and each level is composed of several key process areas. Each key process area identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important for enhancing process capabilities [Humphrey 1995]. The following are descriptions of each KPA’s purpose as defined in the CMM [Paulk 1993]:
The purpose of Requirements Management is to establish a common understanding between the customer and the software project of the customer’s requirements to be addressed.
The purpose of Software Project Planing is to establish reasonable plans for performing the software engineering and for managing the software product.
The purpose of Software Tracking and Oversight is to provide adequate visibility into actual progress so that the managers can take effective actions when the software project’s performance deviates significantly from the software plan.
The purpose of Software Subcontract Management is to select qualified software subcontractors and manage them effectively.
The purpose of Software Quality Assurance is to provide management with appropriate visibility into the process being used by the software project and the products being built.
The purpose of Software Configuration Management is to establish and maintain the integrity of the products of the software project through out the project’s software life cycle.
The purpose of Organization Process Focus is to establish the organizational responsibilities for software process activities that improve the organization’s overall software process capability.
The purpose of Organization Process Definition is to develop and maintain a usable set of software process assets that improve process performance across the projects and provide the basis for cumulative, long-term benefits to the organization.
The purpose of Training Program is to develop the skills and knowledge of individuals so they can perform their roles effectively and efficiently.
The purpose of Integrated Software Management is to integrate the software engineering and management activities into a coherent, defined software process that is tailored from the organization’s standard software process and related process assets, which are described in the Organization Process Definition.
The purpose of Software Product Engineering is to consistently perform a well defined engineering process that integrates all the software engineering activities to produce correct, consistent software products effectively and efficiently.
The purpose of Intergroup Coordination is to establish a means for the software engineering group to participate activity with the other engineering groups so the project is better able to satisfy the customer’s needs effectively and efficiently.
The purpose of Peer Reviews is to remove defects from the software work products early and efficiently.
The purpose of Quantitative Process Management is to control the process performance of the software project quantitatively.
The purpose of Software Quality Management is to develop a quantitative understanding of the quality of the project’s software products and achieve specific quality goals.
The purpose of Defect Prevention is to identify the cause of defects and prevent them from recurring.
The purpose of Technology Change Management is to identify beneficial new technologies (i.e., tools, methods, and processes) and transfer them to the organization in an orderly manner.
The purpose of Process Change Management is to improve continually the software processes used in the organization with the intent of improving quality, increasing productivity, and decreasing the cycle time for product development.
The CMM represents a substantial gain in the understanding of software engineering processes and lays the ground work to help organizations develop their own processes that will continually improve over time. Its objectives, goals, and activities are designed to institutionalize organizational structure and policies. However, it does not address what tools, techniques, and methodologies an engineer should use in order to implement and ultimately satisfy the policies. The CMM does lay the ground work for personal methodologies by providing an organizational structure but it does not directly address the requirements of the individuals. This personal gap between the CMM and the individual software engineers will continue to exist until organizations can:
- Identify Key Process Areas that can be performed by the individual
- Define a subset of the CMM that will be useful to a small development team
2.0 Overview of the Personal Software Process(PSP)
The Software Engineering Institute has developed a Personal Software Process [Humphrey 1995] that addresses the gap that exists between the Capability Maturity Model and the individual.
The PSP has a process evolution framework similar to the CMM. The PSP partially addresses 12 of the 18 KPAs defined in the CMM. The following figure shows the CMM along with its key process areas and those areas that are partially addressed in the PSP have been noted with an asterisk.
In order to develop high quality software, each individual component that is integrated into the overall product must also be of high quality. The overall strategy of the PSP is to make sure all the individual components are of high quality. The PSP accomplishes this by providing a defined personal process framework that the software engineer can use to:
- Develop a plan for every project / component
- Record their development time
- Track their defects
- Retain their data in project summary reports
- Use their data to plan future projects / components
- Analyze their data to evolve their processes
- To improve their performance
2.1 PSP0: The Baseline Process
This is the initial stage the personal software process. Its fundamental objective is to document the engineer’s personal process as it exist today. This baseline will provide a consistent basis for measuring progress. Additionally, it provides a defined structure on which to improve. The only modification that the engineer does to his own process at this level is to record measures of performance. The PSP guides this data gathering by providing standard forms and templates. This initial phase is very similar to the CMM initial level of maturity. This reemphasizes that fact that the PSP and CMM approach Software Process Improvement in a similar way, but from different perspectives (Top-Down or Bottom-Up).
2.2 PSP1: The Personal Planning Process
PSP1 supplements the Baseline Process by the addition of software planning activities. These activities are geared toward elucidating the relationship between the size of programs and the amount of time that is needed to develop them. This is accomplished by developing a plan and estimating the resources required. After the completion of the project, the plan’s estimated metrics are compared to actual metrics that were gathered over the project’s life span. The comparison of this statistical data allows the engineer to gain insight into his own software process. This knowledge allows them to produce more accurate estimates. Additionally, it allows them to begin to manage their personal software process and improve those areas that will result in the largest gains.
2.3 PSP2: The Personal Quality Management Process
An early PSP objective is to help engineers to deal realistically and objectively with the defects they inject. These defects are usually syntax or simple semantic errors that even the most seasoned software engineer will commit. The amount of time that is required to track down and fix these errors can become extreme in very large and complex projects. PSP2 ,the Personal Quality Management Process, addresses this critical issue through defect management. Defect data collection started in PSP0, The Baseline Process. With this information engineers construct and use checklists for design and code review. This allows the software engineer to focus on those personal areas that are causing problems. With the checklist they can effectively review design and code while modifying the checklist as their own Personal Software Process improves.
2.4 PSP3: The Cyclic Personal Process
The Cyclic Personal Process is the final level in the PSP. Its purpose is to scale up the PSP2, the Personal Quality Management Process, for use on large scale projects. One of the fundamental problem solving approaches in science is Divide and Conquer. This refers to taking a complex problem and subdividing it into smaller more manageable pieces. This is the fundamental concept that is behind PSP3. The following figure illustrates this:
The strength of this approach is that each sub-module is designed as a full PSP 2 project. Fundamentally, the PSP was designed to aid in the development of high quality software. If each module is designed to meet this requirement, then integration of each individual component into the overall product will also satisfy the requirement.
3.0 Supplementing the CMM with the PSP
The CMM is a framework that characterizes an evolutionary process improvement path toward a more mature organization. The CMM is comprised of five maturity levels, each containing several KPAs. All KPAs in the CMM have the following features:
| KPA Structure |
Definitions [Paulk 1993] |
| Goals |
The goals summarizes the key practices of a key process area and can be used to determine whether an organization or project has effectively implemented the key process area. |
| Commitment to Perform |
Commitment to Perform describes the actions the organization must take to ensure that the process is established and will endure. |
| Ability to perform |
Ability to perform describes the preconditions that must exist in the project or organization to implement the software process competently. |
| Activities performed |
Activities performed describes the roles and procedures necessary to implement the key process area. |
| Measurement and Analysis |
Measurement and Analysis describes the need to measure the process and analyze the measurements. |
| Verifying Implementation |
Verifying Implementation describes the steps to ensure that the activities are performed in compliance with the process that has been established. |
The CMM is intended to be a high-level description of very complex software processes and thus should be viewed as a guideline and its goals should be considered as key criteria. An organization is considered to have reached a certain maturity level if it can demonstrate that it satisfies all of the goals for all KPAs at that level. The SPI must be designed to institutionalize the organizational policies of the CMM and at the same time give the engineers the tools, techniques and methodologies needed to implement the policies.
It is a fact, that a Software team is made up of individuals, and they are the ones that actually get the work done. It is at this lower level of abstraction that the benefits of software engineering practices will be most productive. The day-to-day operation of an organization rest on the shoulders of the individuals who construct, build, and implement the company's future. The CMM lays the ground work for personal methodologies, but it does not directly address the requirements of the individuals. This Personal Gap can be mitigated if the SPI is supplemented with the PSP.
One of the fundamental difficulties in designing a SPI initiative using the CMM, is that the model describes what to build but not how to build it. For example, the CMM can be thought of as a blueprint for a house. It tells the construction worker where the load bearing walls, support beams, and trusses should be placed in order to construct a stable house. The problem is that the carpenter may not know how to build a truss or what constitutes a state of the art of truss.
The following demonstrates this difficulty in implementing the CMM KPA of Software Project Planning:
| KPA: Software Project Planning |
Description [Paulk 1993] |
| Goal 1 |
Software estimates are documented for use in planning and tracking the software project. |
| Commitment 1 |
A project software manager is designated to be responsible for negotiating commitments and developing the project’s development plan. |
| Ability 1 |
A documented and approved statement of work exists for the software project. |
| Activity 1 |
The software engineering group participates on the project proposal team. |
| Measurement 1 |
Measurements are made and used to determine the status of the software planing activities. |
| Verification 1 |
The activities for software project planning are reviewed with senior management on a periodic basis. |
The above describes many aspects of what makes up a good Software Project Planning process. With this description an organization would be hard pressed to implement an effective project planning process. There exists issues that are not covered in the CMM that could severely impact the overall effectiveness of the SPI initiative like the following:
- What can we estimate besides schedule?
- Is the size of the software produce related to the effort needed to build it?
- How do we generate estimates from previous experience?
- What should be included in a project’s development plan?
- What constitutes a "good" project plan?
- How do we measure the status of the software planing activities?
- Why do we need to measure anything?
The Software Engineering Institute developed the PSP [Humphrey 1995] to addresses these and other important issues that exist in the Capability Maturity Model. The PSP is a bottom-up approach to Software Engineering which focus on the individual software engineer, their personal practices, and how they relate to an organization wide Software Process Improvement plan. In the context of the house building example, the PSP describes what issues are involved in building a truss and offers directional support in its construction.
The underlying philosophy is that the CMM is a top-down approach to improving an organization’s ability to engineer software. If an organization trains its software engineers and project managers in the PSP, using A Discipline for Software Engineering [Humphrey 1995] as the training textbook, then the organization as a whole will have a better understanding of the key issues in implementing many of the CMM related initiatives.
4.0 A PSP Based Software Project Planning Process
A Software Project Plan documents estimates for the work to be performed, the necessary commitments, and defines a plan to perform the work. One of the most important factors in satisfying the Key Process Area of Software Project Planning, is that the process of constructing, documenting, and maintaining the Software Project Plan must be in compliance with the Capability Maturity Model. The PSP can help form a subset of these software processes in a highly integrated manner. The following chart graphically displays the relationship between the PSP and the CMM based Software Project Planning processes:
The Preliminary Design Phase is undertaken after the Software Requirements Specification (SRS) is completed. In this phase the software development team constructs a high level design which can include Entity Relationship Diagrams (ERD), Data Flow Diagrams (DFD), and Data Dictionaries. After the high level design is completed, inspected, and baselined, the project manager divides the design into individual components that are suitable for an individual PSP trained engineers to work on. These components are then given to the PSP engineer who use their own defined personal process to estimate size, time, and effort required for implementation of the component. After all components have been estimated in this faction, the information is passed back to the project manager. The Software Project Plan is developed by integrating the individual estimates for all of the components of the high level design.
5.0 A PSP Based Software Project Tracking and Oversight Process
The Key Process Area of Software Project Tracking and Oversight focus on management’s ability to determine project status. In order to provide adequate visibility into the process of developing software, actual results and performances must be tracked against the Software Project Plan. When the actual results and the plan significantly differ, correction action should be taken. One way to perform this type of detailed tracking is with earned value project scheduling.
Earned value (EV) tracking is a mechanism to evaluate the progress of a project [Boehm 1981]. EV works by establishing a value for each task in the software project plan. This value represents the percent of effort required to complete the task relative to the overall project effort. The EV tracking mechanism provides a common value scale for each task regardless of the type of work involved. As each task is completed, the project will be awarded that task’s earned value and when the project reached 100% earned value, then all of the planned tasks are completed.
The following chart graphically displays the relationship between the PSP and the CMM based Software Project Tracking and Oversight processes:
As mentioned earlier in this paper (section 3.0), The Software Project Plan is developed by integrating the individual estimated for all of the components of the high level design. In the Critical Design / Code / Test phase, each of these components is implemented by engineers performing full PSP 2.1 cycles on the components. Each engineer tracks to completion each of the implementation tasks with earned value. This low level earned value is then passed back to the project level via weekly or monthly project status update meetings.
Project level earned value can be constructed in one of two ways depending on the granularity of tracking that is required by the project: Component Level earned value or Engineer Level earned value.
Component Level earned value
The project can track component level earned value by defining an earned value task to be the completion of a PSP 2.1 Cycle. Once the engineer has completed the implementing of a component (When the engineers earned value is 100%) then the project is awarded the earned value associated with the component.
Engineer Level earned value
The project can track the engineering level earned value by defining an earned value task to be the completion one of the PSP phases of the PSP 2.1 cycle (Planning, Design, Implementation, Testing, Postmortem). Once the engineer has completed a PSP phase, the project is awarded the earned value associated with the phase.
This type of low level earned value tracking can greatly increase the visibility into the process of developing software. This type of an approach allows the project to track earned value not only at the project level but also at the individual engineer level. The complexity of defining a software process to calculate earned value is masked by the PSP since the process is already defined for the engineering level.
6.0 Benefits of the PSP-CMM approach
The focus of this paper is on the organizational benefits of using the PSP to supplement the CMM Level 2 Key Process Areas. This support can be divided into to sections: Engineer Level Support and Project Level Support. Even though an examination of Engineer Level Support is beyond the scope of this paper, the following is a brief summary of results from an excellent technical report on the PSP’s impact on individual engineers [Hayes 1997]:
- Effort estimates improved by a factor of 1.75
- Size estimates improved by a factor of 2.5
- Product quality, the percent of defects found before compile, increased by 50%
6.1 Project Level Support: Case Study
In 1992, Advanced Information Services Inc. (AIS) realized the need for a paradigm shift in the way that they developed software. AIS is an independent software contracting organization with offices in Peoria and Northbrook, Illinois and a subsidiary in Madras, India. AIS has successfully adopted and institutionalized the CMM and PSP into their defined software process, which has been internationally acknowledged by their selection as a finalist for the 1997 IEEE Computer Society Award for Software Process Improvement.
The effect of an organizational CMM - PSP software process improvement effort is visible in the following chart. The implementation of the CMM provided measurable improvements in AIS’s ability to provide "Faster, Better, Cheaper" software. The inclusion of the PSP further refined the fidelity of the AIS Defined Process and has been successful in sustaining continual improvement.
This chart show the Schedule predictability, as measured by the number of months late, of developmental projects at AIS. The x-axis represents the start data of each of the projects that are plotted. The chart is divided into 3 sections:
- 1988-1992. This represents the time period when AIS did not have a defined software process.
- 1992-1995. This represents the time period when AIS utilized CMM based software process.
- 1995-1997. The represents the time current time period as AIS follows CMM based software process that include the PSP.
There is obviously a dramatic improvement in schedule predictability that is associated with the introduction of the CMM based software process. The inclusion of the PSP does not have a large impact of the schedule predictability. The major benefit gained from the introduction of the PSP is evident in the following effort predictability chart:
This chart show the Effort predictability, as measured by the percent of effort deviation of developmental projects at AIS. The x-axis represents the start data of each of the projects that are plotted. The chart is divided into the same 3 sections as the Schedule Predictability chart. The benefits of supplementing CMM based software process with the PSP are clearly evident in the "PSP-CMM Process" section of the effort predictability chart.
The CMM based process had an impact of the Schedule Predictability but did not have the same impact of the Effort Predictability. The reason for this is that the CMM gave the organization the ability to determine the status of the projects and determine if the schedule was slipping. Thus, if a problem was detected then more hours were "donated" to the project in an attempt to maintain the schedule.
It was not until the PSP was introduced that the Effort Deviation experienced a marked reduction. The PSP equipped the individual engineers with the tools and techniques to needed meet their effort estimates. The combination of the CMM and the PSP seems to be an effective way to manage the software process.
7.0 Conclusions
The PSP and CMM are mutually supportive. The CMM provides the orderly support environment engineers need to do superior work, and the PSP equips engineers to do high quality work and participate in organization process improvement [Iyer 1996].
The underlying philosophy is that the CMM is a top-down approach to improving an organization’s ability to engineer software. The PSP is the bottom-up approach that enables the engineers to improve the quality of their software. The combination of these macro and micro methodologies can empowered organization to improve profitability of development projects by meeting cost estimate and schedule commitments with reasonable consistency.
Appendix A: References
[AIS]
Advanced Information Services. 1605 Candletree Dr. Suit 114. Peroria IL USA 61614
[Boehm 1981]
Boehm, B. (1981). Software Engineering Economics. Englewood Cliffs, NJ: Prentice-Hall, Inc.
[DoD 1987]
Report of the Defense Science Board Task Force on Military Software, Office of the Under Secretary of Defense for Acquisition, Washington, D.C., September 1997
[Hayes 1997]
Hayes, W (1997), The Personal Software Process (PSP): An empirical study of the impact of PSP on Individual Engineers. Technical Report, CMU/SEI-1997-97-TR-001, Software Engineering Institute, Carnage Melon University, Pittsburgh, PA.
[Humphrey 1995]
Humphrey, W.S (1997), A Discipline for Software Engineering. Addison Wesley, Reading, M.A
[Iyer 1996]
Iyer, S. (1996), PSP and The Capability Maturity Model. Technical Report, Software Engineering Institute, Carnage Melon University, Pittsburgh, PA.
[Paulk 1993]
Paulk, M.C., Bill Curtis, M. B. Chrisis (1993), "Capability Maturity Model for Software Version 1.1," Technical Report, CMU/SEI-93-TR-24, Software Engineering Institute, Carnage Melon University, Pittsburgh, PA.