General
The SPICE suite of documents is concerned with the processes, people issues and the application of technology relevant to software. It provides a framework for the assessment of software processes. This framework can be used by organizations involved in planning, managing, monitoring, controlling and improving the acquisition, supply, development, operation, evolution and support of software.
Process assessment examines the processes used by an organization to determine whether they are effective in achieving their goals. The assessment characterizes the current practice within an organizational unit in terms of the capability of the selected processes. The results may be used to drive process improvement activities or process capability determination by analysing the results in the light of the organization's business needs, identifying strengths, weaknesses and risks inherent in the processes.
The documents provide a structured approach to software process assessment for the following purposes:
The high level view of the relationships between process assessment, process improvement and process capability determination is shown in the figure below, along with an indication of the places of the various components of the SPICE document set in the processes.
Overview of relationships of elements of the SPICE suite
The requirements of the SPICE suite are designed to afford assessment results that are repeatable, objective, comparable within similar contexts, and able to be used for either process improvement or process capability determination.
Dependable assessment results are achieved through the definition of a standard framework for the conduct of assessments. The framework includes an architecture for rating practices and processes and for presenting assessment ratings. The assessment framework also provides guidance on the conduct of the assessment, supported by tools to assist in the objective rating of processes based on indicators of practice adequacy. The SPICE suite provides guidance in the contexts of both process improvement and process capability determination. It further provides a definition of the required skills and experience for assessors.
This paper describes how the parts of the Suite can be used to conduct and use process assessments. The key determinant in the use of the Suite is the purpose for which the assessment is being conducted. This may be:
The context of process assessment
The context of a process assessment is summarized in figure 4. Part 3 of the SPICE Suite defines the requirements for conducting an assessment, sets out the basis for rating, scoring and profiling process capabilities, and defines the circumstances under which assessment results may be compared. Part 4 provides guidance on conducting a team-based assessment and interpreting the requirements in part 3. This guidance is generic enough to be applicable across all organizations, and for conducting assessments using a variety of methods, techniques and tools.
Process assessment is an activity that is performed either during a process improvement initiative as described in part 7 of the SPICE suite, or as part of a process capability determination exercise as described in part 8. In either case, the formal entry to the assessment processes occurs with the compilation of the assessment input which defines the purpose of the assessment (why it is being carried out), the scope of the assessment (which processes are to be assessed) and what constraints, if any, apply to the assessment. The assessment input also defines the responsibility for carrying out the assessment and gives definitions for any processes within the scope of the assessment that are extensions of the processes defined in part 2.
An assessment is carried out by assessing selected processes against the process model defined in part 2 of the SPICE Suite. This model consists of a set of process-specific base practices on one hand and a set of generic practices on the other hand. The generic practices apply across all processes. The generic practices are grouped into common features and capability levels that define how well the process is managed. The assessment output is a vector (or process profile) of process capability level ratings for each process instance assessed.
An assessment is supported by an assessment instrument, or set of instruments, constructed according to the guidance contained in part 5 of the SPICE Suite. The process assessment is carried out either by a team with at least one qualified assessor who has the competence described in part 6; or, on a continuous basis using suitable tools for data collection and verified by a qualified assessor.
An architecture for software processes
Part 2 of the SPICE Suite defines, at a high level, the fundamental activities that are essential to good software engineering. It describes what activities are required, not how they are to be implemented. The baseline practices may be extended through the generation of application or sector specific practice guides to take account of specific industry, sector, or other requirements.
Each process in part 2 is described by base practices, which are the essential activities of the specific process. Processes are grouped in turn into five process categories as shown in the table below.
Description of process categories
Customer-Supplier
Engineering
Project
Support
Organization
Evolving process capability is expressed in terms of capability levels, common features, and generic practices. Generic practices are applicable to all process. These practices represent the activities necessary to manage a process and improve its capability to achieve desired outputs. They are grouped into common features and capability levels which help define how effective the process will be at achieving its defined purpose.
Tools to support process assessment
Part 5 of the SPICE Suite provides the framework for building an assessment instrument. An assessment instrument is a tool, or set of tools, used during the performance of an assessment to assist the assessor in obtaining reliable, consistent and repeatable results.
An assessment instrument built according to requirements and guidance in part 5 contains a set of assessment indicators which help the assessor to analyse the process under review, and to make consistent judgements about the implemented practices. In addition, an assessment instrument provides a mechanism for the assessor to record notes and results, and may provide a means of capturing other types of information for use in process improvement or process capability determination. An assessment instrument may also provide assistance to the assessor in analysing ratings and compiling process profiles.
Part 5 does not attempt to prescribe a particular format for an assessment instrument, which could be implemented as a manual, paper-based tool; a questionnaire; an automated, on-line tool; or even as an expert system. It does, however, provide a common framework, and prescribes a set of elements that should be incorporated into any type of assessment instrument. In addition, it provides guidance for designers, users and acquirers of assessment instruments about the characteristics and usability of different types of instrument.
Assessor training and qualification
The qualified assessor in a team has the pivotal role of ensuring that other team members collectively have the right blend of specialized knowledge and assessment skills. The qualified assessor provides the necessary guidance to the team, and helps to moderate the judgements and ratings made by the other members of the team to ensure consistency of interpretation.
Part 6 of the SPICE Suite provides guidance for the preparation and qualification of assessors to perform assessments. Specifically, its purpose is to define initial and ongoing qualification of assessors. It is concerned with assessor competencies and appropriate education, training and experience and include mechanisms that may be used to demonstrate competence and to validate education, training and experience.
Process improvement context
Successful software process improvement occurs in a business context by addressing specific business goals that are clearly defined in a process improvement requirement specification.
Part 7 of the SPICE Suite provides guidance on using software process assessment as part of a complete framework and method for performing software process improvement in a continuous cycle although there is no reason why the organization could not employ the guidance for a single cycle of improvement activity. The guidance covers:
The guidance provided does not presume specific organizational structures, management philosophies, software life cycle models or software development methods. The concepts and principles are appropriate for the full range of different business needs, application domains and size of organization, so that they may be used by all types of software organizations to guide their improvement activities.
Process capability determination context
The procedure for process capability determination is described in part 8 of the SPICE Suite. Process capability determination is mainly built upon process assessment as described in the Suite. Processes are rated against the process model defined in part 2, using the measurement and rating framework defined in part 3. The context of process capability determination is shown in the figure.
An acquirer of software products or services has technical and other needs as expressed in the specified requirements. Before making a contract the acquirer may need to determine the process capability of the prospective contractor, or a supplier may want to ascertain its own process capability before responding to an acquirer's proposal. The technical and other needs for process capability determination are documented in the specified requirements.
The specified requirement is translated into a target capability that represents the required process capability, and process assessment input that will scope the process assessment. The supplier may put forward a proposed process capability as a set of process-by-process capability level ratings to be offered by the organizational unit concerned. In a straightforward situation, the proposed process capability may be based on a recent self-assessment or by other means. In more complex cases, a supplier may propose a process capability to be achieved in the future based on the supplier's current profile and relevant improvement plans, backed up if possible with improvement records, or a constructed capability including the capability of one or more sub-contractors or partners.
The credibility of the proposed process capability is analysed together with the risks involved and reported on in the process capability report. Part 8 of the SPICE Suite provides guidance on how to use the results of an assessment for the purpose of determining the process capability of suppliers. The guide specifically addresses process capability determination both for use within an organization to determine the risks associated with undertaking a new project (sometimes called first party use) and for use by an acquirer for assessing external suppliers (sometimes called second party or contractual use).
Conformance
The SPICE Suite contains three principal areas where conformance may be claimed: in the conduct of a software process assessment, in the development of processes which extend those listed in the defined model, and in the construction of assessment instruments. This section explains the nature of conformance in these three areas.
Overview of the requirements
This guidance describes how to conduct assessments in such a way that conformance to the requirements can be readily demonstrated.
The requirements, set out in part 3 of the SPICE document set for conducting a software process assessment are designed to ensure that the results are reliable, consistent and repeatable. This is important where an organization wishes to compare its assessment results with those of other similar organizations. It is especially important in process capability determination in a contractual situation where competing or collaborating suppliers are being compared.
In general terms, an assessment meeting the requirements of this standard is one which:
It is the responsibility of the assessment team, and specifically the qualified assessor, to ensure that the requirements for conducting an assessment are met. The sponsor of the assessment will normally be the party requiring that the assessment conforms to the requirements.
Assessment team membership
When a decision is taken to perform an assessment, the sponsor of the assessment should be responsible for assembling the assessment inputs, as described in part 7 or part 8 of the SPICE suite. In particular, the assessment input includes the nomination of an assessor qualified as described in part 6 (the qualified assessor).
The qualifications of an assessor may be verified by following the procedure defined in part 6. This may be performed by the organization being assessed; by the employer of the assessor (if different); or by a third party. Details of the verification should be available, on request, to the sponsor of the assessment. Sufficient records of the assessor's personal history should be retained.
The assessment process
When performing an assessment, the team is responsible for ensuring that the requirements for this process are followed and documented. Specific requirements cover documenting the
Some of the items - particularly assessment scope and constraints - contain a number of elements, and care must be taken to ensure that all of these have been addressed.
The requirements for rating are contained in part 3 of the SPICE Suite. Guidance on reviewing the inputs of an assessment is contained in part 4. The assembly of the inputs themselves will depend on the purpose of the assessment, and guidance is contained in parts 7 and 8. Documentation of the assessment inputs should be retained in the assessment record, and traceability provided to ensure that it can be verified that the necessary reviews have been performed.
Selecting processes for assessment
The Assessment scope sets out which of the organizational units processes are to be assessed, and their mapping into the corresponding processes as defined in part 2 of the SPICE Suite. The mapping will normally be a task for the qualified assessor. Defining the scope in this way ensures that there is a common basis for rating and measurement.
The requirements for identifying processes and process instances and for selecting process instances for the assessment are set out in part 3. Guidance on mapping to the defined processes, and for rating the process instances is contained in part 4.
Apart from guiding the assessment, the mapping of organizational processes to the model defined in part 2 forms an essential part of the assessment record. It should be possible, after the assessment has been completed, for any person examining the assessment record to be able to relate the mapping to records of organizational structure, procedures, and standards within the assessed organization.
Reporting assessment results
One of the main reasons for conducting a conformant assessment is to ensure comparability with other assessment results. This is made possible by the requirements for rating processes and calculating results within the measurement framework, and reporting them in a way that makes the results of the calculation obvious.
Part 3 of the SPICE Suite defines the requirements for conducting ratings, and for calculating practice adequacy ratings, process capability ratings and process profiles. Detailed guidance on deriving these ratings is contained in part 4. Requirements for recording the assessment outputs are contained in part 3. It should be noted that the SPICE Suite does not mandate any specific format for the process profile; a variety of numerical or graphical presentation formats would meet the requirements.
Whatever the final format of the process profile, it is essential that clear traceability to the practices and processes contained in the model defined in part 2 is provided, to enable the process of calculation to be verified. In addition, it should be noted that assessment output contains full details of the process context in the assessment record. This record will also include additional information collected as part of the assessment, and required as inputs to the process improvement or process capability determination activities to follow on from assessment.
Extensions to the baseline practices
Part 2 of the SPICE Suite provides a model for assessing and improving processes. These processes identify critical attributes that a process should have to be considered complete and effective, but without unduly constraining the implementation of the process. Further guidance on implementing processes may be found in related software standards such as DIS 12207-1 or ISO 9000-3.
Variant process models may be built that address the unique needs of an industry sector or organization by selecting specific processes from the model in part 2, providing guidance on how to interpret the practices, and/or developing extended processes. Extended processes may include additional base practices, guidance on how to interpret practices for adequacy, or be an entirely new process.
Extended processes and variant models may be developed by organizations for their own internal use; by acquirers of software systems for use in specific acquisition situations; or by professional organizations defining requirements for specific application domains or use situations.
The requirements for building conformant variants and extended processes are set out in Annex A of part 2, together with a style guide in Annex G. The essence of the requirements is to allow only variants that are equivalent to or extend the process model in part 2, and to provide traceability of the base practices of variants and extended processes to the base practices in part 2.
Documentation of the extended process or variant and its differences from the model in part 2 is essential for assessment and rating, and for demonstrating conformance to the requirements. New processes and new practices in extended processes should be identified. The assessment results generated for an extended process should explicitly identify the variations from the standard model. In an individual assessment, the team leader should document any extended processes used, and refer to the location of evidence of their conformance.
Constructing and selecting an Assessment Instrument
An assessment instrument is used during an assessment to assist assessors in identifying work products, practices and processes, in making consistent judgements, and in analysing and presenting the results. The instrument may be in the form of a simple questionnaire, or it may be an automated tool. The instrument for a given assessment might be constructed specifically by the assessment team; it might be a tailored version of an available tool; or it might be a commercial product.
The minimum set of requirements to be met by an assessment instrument of any type are defined in part 5 of the SPICE Suite.
It is the responsibility of the qualified assessor to ensure that the assessment instrument chosen meets the requirements expressed in the part 5, and to document the relevant evidence demonstrating conformance. Documentation may consist of point-by-point analysis of the chosen tool against the requirements in part 5; alternatively, it may be that the supplier of a tool provides self or third-party certification of conformance.
Documentation of the conformance of the assessment instrument to the requirements forms part of the records of the assessment. Where software-based tools are developed or used to support assessments, ISO 12119 - 1995 may provide a useful mechanism for demonstrating or verifying their conformance to the requirements of part 5.