2.3 Information Security Risk Evaluation Attributes

We now turn our attention directly toward information security evaluation, moving from the more abstract nature of risk management principles to information security risk evaluation attributes. The remainder of this chapter focuses on the attributes and outputs of the OCTAVE approach.

First, we examine the tangible characteristics of information security risk evaluations and define what is necessary to make the evaluation a success from both the process and organizational perspectives. We begin by exploring the primary relationships between the principles and attributes, illustrated in Table 2-2.

Table 2-2. Mapping OCTAVE Principles to Attributes

Principle	Attribute
Self-direction	Analysis team Augmenting analysis team skills
Adaptable measures	Catalog of practices Generic threat profile Catalog of vulnerabilities
Defined process	Defined evaluation activities Documented evaluation results Evaluation scope
Foundation for a continuous process	Next steps Catalog of practices Senior management participation
Forward-looking view	Focus on risk
Focus on the critical few	Evaluation scope Focused activities
Integrated management	Organizational and technological issues Business and information technology participation Senior management participation
Open communication	Collaborative approach
Global perspective	Organizational and technological issues Business and information technology participation
Teamwork	Analysis team Augment analysis team skills Business and information technology participation Collaborative approach

Note that some of the attributes map to more that one principle, as is to be expected in such a complex activity as an information security risk evaluation. By looking at the attribute names, you will notice that they are focused on tangible characteristics of the evaluation process and are process oriented rather than activity oriented. We will start looking at activities in the next section when we present the outputs. Let's turn our attention now to the information security risk evaluation attributes, starting with the analysis team.

Analysis Team

An analysis team staffed by personnel from the organization must lead the evaluation activities. The analysis team must be interdisciplinary in nature, including people from both the business units and the information technology department. The analysis team must manage and direct the information security risk evaluation for its organization, and it must be responsible for making decisions based on the information gathered during the process.

This attribute is important because it ensures that ultimate responsibility for conducting the evaluation is assigned to a team of individuals from the organization. Using an analysis team to lead it helps to ensure the following results:

• People who understand the business processes and who understand information technology work together to improve the organization's security posture.

• The evaluation is run by personnel who understand how to apply all worksheets and tools used during the evaluation.

• The method is applied consistently across the organization.

• People in the organization feel "ownership" of the evaluation results, making them more likely to implement the recommended strategies and plans.

Augmenting Analysis Team Skills

The evaluation process must allow the analysis team to augment its skills and abilities by including additional people who have specific skills required by the process or who possess needed expertise. These additional people can be from other parts of the organization, or they can be from an external organization.

The analysis team is responsible for analyzing information and making decisions during the evaluation. However, the core members of the analysis team may not have all of the knowledge and skills needed during the evaluation. At each point in the process, the analysis team members must decide if they need to augment their knowledge and skills for a specific task. They can do so by including others in the organization or by using external experts. This attribute is important because it ensures that the analysis team has the required skills and knowledge to complete the evaluation. This attribute also allows an organization to conduct an information security risk evaluation even when it does not have all of the required knowledge and skills within the organization. It provides an avenue for working with external experts when appropriate.

Catalog of Practices

The evaluation process must assess an organization's security practices by considering a range of strategic and operational security practice areas. These are formally defined in a catalog of practices. The catalog of practices used by an organization should be consistent with all laws, regulations, and standards of due care with which the organization must comply. A more detailed description of the catalog of practices appears in Chapter 5 and in Appendix C.

Using a catalog of practices is important because it allows an organization to evaluate itself against a known and accepted measure. This helps the organization to understand what it is currently doing well with respect to security (its current security practices) and what it is not doing well (its organizational vulnerabilities). The catalog of practices is also important because it creates the structure for an organization's protection strategy. Finally, the catalog also provides a basis for selecting actions to include in risk mitigation plans.

Generic Threat Profile

The evaluation process must assess threats to the organization's critical assets by considering a broad range of potential threat sources that are formally defined in a generic threat profile. The profile contains potential threat sources ranging from insiders deliberately modifying critical information to power outages, broken water pipes, and other dangers beyond the organization's control.

Using a generic threat profile is important because it allows an organization to identify threats to its critical assets based on known potential sources of danger. The profile also uses a structured way of representing potential threats and yields a comprehensive summary of threats to critical assets, thus providing a complete and simple way to record and communicate threat information. A detailed look at the generic threat profile is presented in Chapter 6.

Catalog of Vulnerabilities

The evaluation process must assess the current technological weaknesses (technology vulnerabilities) in the key components of the computing infrastructure by considering a range of technology vulnerabilities based on platform and application. Vulnerability evaluation tools (software, checklists, scripts) examine infrastructure components for technology vulnerabilities contained in the catalog. Two examples of catalogs of vulnerabilities are CERT®^[1] Knowledgebase^[2] and Common Vulnerabilities and Exploits (CVE).^[3]

^[1] CERT is registered in the U.S. Patent and Trademark Office.

^[2] The CERT® Knowledgebase contains a public database describing vulnerabilities and a restricted access catalog containing descriptive information regarding more than 1,300 vulnerabilities. It can be accessed at http://www.cert.org/kb/.

^[3] CVE is a community effort led by the MITRE Corporation. It can be accessed at http://www.cve.mitre.org.

Using a catalog of vulnerabilities is important because it allows an organization to evaluate its technology base against known technology vulnerabilities. Identifying which vulnerabilities are present in the organization's key components provides the organization with information about how vulnerable its computing infrastructure currently is. Chapters 7 and 8 discuss how to use the catalog of vulnerabilities.

Defined Evaluation Activities

The procedures for performing each evaluation activity and the artifacts (worksheets, catalogs, etc.) used during each activity must be defined and documented. These include

• Procedures for preparing for the evaluation

• Procedures for scoping the evaluation

• Procedures for completing each evaluation activity

• Specifications for all tools and worksheets required by each activity

• Specifications for catalogs of information that define accepted security practices, known sources of threat, and known technological weaknesses

Implementing defined evaluation activities helps to institutionalize the evaluation process in the organization, ensuring some level of consistency in the application of the process [GAO 99]. It also provides a basis upon which the activities can be tailored to fit the needs of a particular business line or group.

Documented Evaluation Results

The organization must document the results of the evaluation, either in paper or electronic form. Organizations typically document and archive risks to the organization's critical assets as well as security strategies and plans to improve the organization's security posture.

It is important to establish a permanent record of evaluation results. A database of information can serve as source material for subsequent evaluations and is also useful when tracking the status of plans and actions after the evaluation. For example, the information recorded can also be used as lessons learned. When risks to a critical asset are identified, staff members can look at the mitigation plans for risks to similar assets. Organizational personnel can then understand which mitigation actions have been effective in the past and which haven't, enabling them in turn to create more effective mitigation plans.

Evaluation Scope

The extent of each evaluation must be defined. The evaluation process must include guidelines to help the organization decide which operational areas (business units) to include in the evaluation. Determining the scope of an evaluation is important for ensuring that its results are useful to the organization. If the scope of an evaluation becomes too broad, it is often difficult to analyze all of the information that is gathered. If it is too small, it will not yield an accurate picture. Setting a manageable scope for the evaluation reduces the size of the evaluation, making it easier to schedule and perform the activities. In addition, the areas of an organization can be prioritized for the evaluation. Essentially, the highest-risk areas can be examined first or more frequently.

Next Steps

The evaluation must include an activity whereby organizational personnel identify the next steps required to implement security strategies and plans. This activity often requires active sponsorship and participation from the organization's senior managers. Next steps typically include the following information:

• What actions the organization will take to follow up on the results of the evaluation

• Who will be involved in implementing security strategies and plans

• Plans for future activities to evaluate information security risks

The task of identifying the next steps that people in the organization must take to implement the protection strategy and the mitigation plans is essential for security improvement. The people in the organization need to build upon the results of the evaluation. Getting senior management sponsorship is the first critical step toward making this happen.

Focus on Risk

The evaluation must focus on assessing an organization's information security risks by examining the interrelationships among assets, threats to the assets, and vulnerabilities (including both organizational and technological weaknesses). This attribute is important because it requires the organization's personnel to focus on security issues and their effect on the organization's business objectives and mission. Personnel must look beyond the current organizational and technological weaknesses and examine how those weaknesses relate to the organization's critical assets and the threats to those assets, thus establishing the risks to those assets.

Focused Activities

The evaluation process must include guidelines for focusing evaluation activities, for example:

• Workshops that efficiently elicit security-related information from an organization's staff members

• Analysis activities that use asset information to focus threat and risk identification activities

• Analysis activities that use asset and threat information to set the scope of the technology vulnerability evaluation

• Planning activities that establish risk priorities using risk measures (impact, probability)

Focusing each activity on the most critical information security issues is important to ensure that the organization applies its resources efficiently. If you gather too much information, it may be difficult to analyze. Focusing on the most important information reduces the size of the evaluation, making it easier to perform the activities while still collecting the most meaningful data and producing the most significant results.

Organizational and Technological Issues

The evaluation process must examine both organizational and technological issues. Information security risk evaluations typically include the following practice- and vulnerability-related information:

• Current effective security practices used by staff members

• Missing or ineffective security practices (also called organizational vulnerabilities)

• Technological weaknesses present in key information technology systems and components

Because security has both organizational and technological components, an evaluation must deal with both organizational and technological issues. When creating the organization's protection strategy and risk mitigation plans, the analysis team considers both types of issues in relation to the mission and business objectives of the organization. By doing so, the team is able to address security by creating a global picture of the information security risks the organization must confront.

Business and Information Technology Participation

The evaluation process must include participants from both the business units and the information technology department, allowing for the establishment of an interdisciplinary analysis team (see the analysis team attribute). Participants from key areas (business units) of the organization also need to contribute their perspectives on security-related issues during activities designed to elicit knowledge. Note that participants must include representatives from multiple organizational levels (senior management, middle management, and staff).

Incorporating multiple perspectives is essential to ensure that a broad range of risk factors is considered. Staff members who work in the business lines of an organization understand the relative importance of business operations and the systems and information that support them. In general, they are in the best position to understand the business impact of disruption or abuse to business systems and operations and the impact of potential mitigation actions. It is information technology personnel and information security experts who best understand the design of existing systems and the impact of technology-related vulnerabilities, just as it is they who are also in the best position to evaluate the trade-offs of mitigation actions when evaluating their effect on system performance.

Senior Management Participation

Senior managers in the organization must have defined roles during the evaluation process. Typically, an organization's senior managers demonstrate active sponsorship of the evaluation, participate in workshops to contribute their understanding of security-related issues and their effect on business processes, review and approve security strategies and plans, and define the steps required to implement security strategies and plans.

Senior management participation is the single most important success factor for information security risk evaluations, as it demonstrates strong sponsorship of the evaluation. This level of sponsorship helps to ensure that staff members are available and willing to participate in the evaluation, take the evaluation seriously, and are prepared to implement the findings after the evaluation.

The senior managers' active participation in an information security risk evaluation is also important to the success of the initiative. Senior managers can help to define the scope of the assessment and to identify participants. If senior managers support the evaluation, people in the organization tend to participate actively. If senior managers do not support the evaluation, staff support for the evaluation will dissipate quickly.

Collaborative Approach

Each activity of the evaluation process must include interaction and collaboration among the people who are participating in that activity. Collaboration can be achieved through the use of workshops or other interactive methods.

A collaborative approach is an essential attribute of information security risk evaluations. Because security is interdisciplinary in nature, completing the evaluation activities requires interdisciplinary knowledge and skills. It is therefore important for each evaluation activity to require all participating individuals to interact and collaborate, thus ensuring that the necessary skills and knowledge are applied to complete that activity satisfactorily.

As you can see, all of the attributes just described focus on the evaluation process and how that process is implemented in an organization. Next, we build on this view by exploring the results of information security risk evaluations.