6
An approach to quantitatively measure Information security PEDRO GIOVANNI LEON Carnegie Mellon University, Pittsburgh, (PA), USA [email protected] ASHUTOSH SAXENA SETLabs, Infosys Technologies Ltd., Hyderabad (AP) 500019, INDIA [email protected] ABSTRACT Measuring information security has traditionally been daunting task due to the lack of proper tools. Even more, organizations are concerned about suffering security breaches but, most of the time, justifying security investment is a tough task in the absence of a tangible measurement. In this paper, we propose an approach to quantitatively measure different aspects of information security. The proposal leverages different aspects of risk management, software assurance, audit and control to provide a flexible, easily- updated, goal-oriented and risk-based information security measurement. We hope that this work is helpful to evaluate the effectiveness of countermeasures and perform benchmark against industry standards and regulations. C.4 [Performance Of Systems]: Measurement techniques General Terms : Security. Keywords: Information security, quantitative measurement. 1. INTRODUCTION In order to achieve a reasonable level of security organizations focuses on implementing different types of security measures to mitigate some of their risks and invest considerable amount of resources aimed to achieve the desired level. Furthermore, security breaches can occur either because of systems bad configurations, defective software designs, vulnerable communication protocols, improper procedures, lack of awareness on security aspects and so on. That is, information security goes beyond the implementation of technical controls and it has to be seen as an organizational-wide process. Even more, as a process, in order to be able to improve it, we should have the means to evaluate its performance. Our paper is focused on providing a measurement for the different aspects of information security, namely, confidentiality, integrity, availability and accountability. It also provides an individual measurement for each aspect of information security (i.e. confidentiality, integrity, availability and accountability). Therefore, it is helpful in the identification of control gaps between technical, operational and organizational controls; it also allows stakeholders to identify strengths and weaknesses on each of the aspects of security. In addition, it eases the benchmark with a number of information security best practices, standards and regulations. Finally, it is useful to monitor the performance of implemented security countermeasures. The organization of the paper is as follows: In section 2 we present the related works in this area; section 3 describes the Description of the Scheme. Elements of the Security Control Matrix; the definition of Security Performance Metrics is included in section 4 and section 5 respectively. Section 6 introduces the Measurement Model. In section 7, concluding remarks and future works are presented followed by References in section 8. 2. Related Work National Institute of Standards and Technology have published insightful Special Publications dealing with topics of Risk Management, Security Controls and Information Security Metrics [1,2,3,4]. In [5] an approach for measuring Software Assurance combining different measurement methodologies is presented. ISO/IEC Standards addressing Information Security management, Software Measurement and Information Security Measurement Performance Metrics are [6], [7] and [8], respectively. A well known process oriented methodology used to evaluate the maturity of software development processes is [9]. The Build Security in [10] is an initiative of the Department of Homeland Security (DHS) implemented by the Software Engineering Institute (SEI) at Carnegie Mellon University. Its objective is to provide software developers and security architects with practices, tools, guidelines, rules, principles, and other resources to be used to build security into the software development process. The Making Security Measurable Project [11] presents a comprehensive database of software vulnerabilities, attacking patterns, configurations problems. The main goal of the project is to present a standard definition of these security topics in order to ease the automation of security measurements. Common software vulnerabilities, weaknesses, attacking patterns, platforms classification and configuration problems databases, are used to establish enumeration techniques to evaluate and measure the security of software. Several research papers addressing topics of measurement models and metrics have been written in the last few years. Savola proposed taxonomy for security metrics in [12] and [13]. Heyman et al. proposed to use the notion of security patterns to determine security metrics and interpreting their results [14]. Andrew Jaquith well known book [15] provides good ideas about security metrics and performance. In [16], qualities of risk as a security metric are analyzed, further criteria-compliance, intrusion- detection, policy and incident based metrics are proposed as Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ISEC2010, 3rd India Software Engineering Conference, Mysore, Feb 25-27, 2010. Copyright 2010 ACM x-xxxxx-xxx-0/00/000x…$x.00.

Information assurance and security metrics

Embed Size (px)

Citation preview

Page 1: Information assurance and security metrics

An approach to quantitatively measure Information security

PEDRO GIOVANNI LEON Carnegie Mellon University,

Pittsburgh, (PA), USA [email protected]

ASHUTOSH SAXENA SETLabs, Infosys Technologies Ltd.,

Hyderabad (AP) 500019, INDIA [email protected]

ABSTRACT Measuring information security has traditionally been daunting task due to the lack of proper tools. Even more, organizations are concerned about suffering security breaches but, most of the time, justifying security investment is a tough task in the absence of a tangible measurement. In this paper, we propose an approach to quantitatively measure different aspects of information security. The proposal leverages different aspects of risk management, software assurance, audit and control to provide a flexible, easily-updated, goal-oriented and risk-based information security measurement. We hope that this work is helpful to evaluate the effectiveness of countermeasures and perform benchmark against industry standards and regulations.

C.4 [Performance Of Systems]: Measurement techniques

General Terms : Security.

Keywords: Information security, quantitative measurement.

1. INTRODUCTION In order to achieve a reasonable level of security

organizations focuses on implementing different types of security measures to mitigate some of their risks and invest considerable amount of resources aimed to achieve the desired level. Furthermore, security breaches can occur either because of systems bad configurations, defective software designs, vulnerable communication protocols, improper procedures, lack of awareness on security aspects and so on. That is, information security goes beyond the implementation of technical controls and it has to be seen as an organizational-wide process. Even more, as a process, in order to be able to improve it, we should have the means to evaluate its performance.

Our paper is focused on providing a measurement for the different aspects of information security, namely, confidentiality, integrity, availability and accountability. It also provides an individual measurement for each aspect of information security (i.e. confidentiality, integrity, availability and accountability). Therefore, it is helpful in the identification of control gaps between technical, operational and organizational controls; it also allows stakeholders to identify strengths and weaknesses on each

of the aspects of security. In addition, it eases the benchmark with a number of information security best practices, standards and regulations. Finally, it is useful to monitor the performance of implemented security countermeasures.

The organization of the paper is as follows: In section 2 we present the related works in this area; section 3 describes the Description of the Scheme. Elements of the Security Control Matrix; the definition of Security Performance Metrics is included in section 4 and section 5 respectively. Section 6 introduces the Measurement Model. In section 7, concluding remarks and future works are presented followed by References in section 8.

2. Related Work National Institute of Standards and Technology have

published insightful Special Publications dealing with topics of Risk Management, Security Controls and Information Security Metrics [1,2,3,4]. In [5] an approach for measuring Software Assurance combining different measurement methodologies is presented. ISO/IEC Standards addressing Information Security management, Software Measurement and Information Security Measurement Performance Metrics are [6], [7] and [8], respectively. A well known process oriented methodology used to evaluate the maturity of software development processes is [9].

The Build Security in [10] is an initiative of the Department of Homeland Security (DHS) implemented by the Software Engineering Institute (SEI) at Carnegie Mellon University. Its objective is to provide software developers and security architects with practices, tools, guidelines, rules, principles, and other resources to be used to build security into the software development process.

The Making Security Measurable Project [11] presents a comprehensive database of software vulnerabilities, attacking patterns, configurations problems. The main goal of the project is to present a standard definition of these security topics in order to ease the automation of security measurements. Common software vulnerabilities, weaknesses, attacking patterns, platforms classification and configuration problems databases, are used to establish enumeration techniques to evaluate and measure the security of software.

Several research papers addressing topics of measurement models and metrics have been written in the last few years. Savola proposed taxonomy for security metrics in [12] and [13]. Heyman et al. proposed to use the notion of security patterns to determine security metrics and interpreting their results [14]. Andrew Jaquith well known book [15] provides good ideas about security metrics and performance. In [16], qualities of risk as a security metric are analyzed, further criteria-compliance, intrusion-detection, policy and incident based metrics are proposed as

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ISEC2010, 3rd India Software Engineering Conference, Mysore, Feb 25-27, 2010. Copyright 2010 ACM x-xxxxx-xxx-0/00/000x…$x.00.

Page 2: Information assurance and security metrics

alternative metrics; finally a set of characteristics for good security metrics are listed; namely, they have to measure the right thing (goal oriented), quantitatively measured, be able to be measured accurately, compared against a benchmark, inexpensive to execute, independently refereed, repeatable and scalable. In [17], the author summarizes some issues about security metrics; namely qualitative against quantitative metrics, subjectivity and objectivity, modeling, validity over time, weaknesses of logic values (true and false) for security metrics. The author also presents a formal model of Information Security considering three different abstraction levels: users, services and infrastructure. He also presents a number of axioms describing the desired properties of a good metric. The authors in [18] present a systematic methodology to derive security metrics, it starts from specifying metric requirements, identifying vulnerabilities and software characteristics, analyze security models and categorize and develop security metrics. NIST SP [19] provides a brief description of the SDLC phases and presents an insightful set of security controls desired to be included in each of the phases. Moreover, FIPS documents [20], provide guidelines to classify Federal Information Systems according to the type of information they process or deal with; accordingly [21] provides a set of minimum generic security requirements for Federal Information Systems. In [22] a set of security requirements are presented to evaluate the secure development of web applications. The proposal includes four security levels and two sublevels to classify Web Applications. To be labeled with a certain security level, the Web application should satisfy the set of requirements on the corresponding security level and the levels below it. The CVSS [23] is a standard methodology focused on measuring the negative impact that vulnerabilities can impose on Information Systems, its main goal is to be a standard worldwide methodology to evaluate the impact of vulnerabilities and ease measurement initiatives by establishing a common language.

Our work in this paper is comprised of three main elements: A security control matrix, a set of security performance metrics and a measurement model. The security control matrix is a compilation of security controls from security best practices, regulations and National Institute of Standards and Technology guidelines. The controls are organized in seven processes as proposed in [24] and categorized in 17 families as proposed in [2]. Furthermore, priorities to each control according to the particular needs of the entity (e.g. organization, information system or application) under evaluation have to be assigned. The performance metrics are a subset of security controls with two particular features; they are high priority controls which should be quantitatively measurable. The measurement model receives input from the control matrix and security metrics, taking into consideration security controls and information criteria priorities to derive the levels of security for each information criteria and perspective.

3. Description of the Scheme Stated earlier that our scheme is comprised of three elements:

A security control matrix (SCM), which is a comprehensive list of security controls collected from a number of NIST guidelines, the ISO 27001:2005 standard and industry standards and best practices such as PCI and OWASP. A selection of security performance metrics (SPM), high priority security controls selected from the SCM, these are quantitatively measurable

controls which can be directly mapped with security goals and intended to be periodically reported to senior management. The last element is the measurement model. It takes into consideration the priorities for each aspect of security, namely, confidentiality, integrity, availability and accountability (i.e. information criteria) and security controls to deliver levels of security from different perspectives: Organizational, Operational and Technical and for each information criteria.

The evaluation use as benchmark what we call perfect security. Perfect security is defined as the desired level of security for the entity under evaluation [27]. This level of security is derived from a risk analysis performed by the adequate stakeholders, in which business needs and threats, legal issues, industry regulations and best practices are considered to determine the security goals for the entity. Once the security goals are defined, the stakeholders can select and prioritize from the security controls listed in the SCM that they believe are needed to comply with the desired level of security.

After determining the desired level of security (the security controls that are going to be evaluated), the stakeholders are required to select the security performance metrics (SPM). The SPM are designed to be periodically reported to senior management so that the performance of the security process is evaluated. The measurement model receives as inputs the desired level of security (benchmark), the results of the security evaluation and the values of the SPM. Then, it determine the gap based on the level of compliance and considers priorities for the information criteria and controls to deliver different levels of security for each information criteria under technical, operational and organizational perspectives.

The results of the evaluation are supposed to be analyzed by the stakeholders and, based on that, actions might be taken to improve the level of security for the entity. Corrective actions might include adding or removing security controls for the level of perfect security, changing priorities of the security controls, enforcing implementation of missing security controls, select a different set or modify the current security performance metrics, etc. A graphical view of the scheme is shown in figure1.

4. Security Control Matrix The Security Control Matrix classifies security controls in

seven security processes taken from [24]. We believe that this classification of controls is useful to perform targeted evaluations to particular security processes. Moreover, organizing security controls in processes and families ease the updating of the security control matrix since new security controls can be added accordingly. For each control added to the control matrix a number of fields have to be filled, namely, security perspective, priority, stakeholder, impact on information criteria (confidentiality, integrity, availability and accountability), SDLC phase, quantitatively measurable, compensatory control, security guideline / framework / regulation. The proposed process oriented classification is defined as follows:

Security Management: Focused mainly on high level security risk management practices such as implementation of procedures and policies, assigning responsibilities, monitoring as well as controls oriented to comply with statutory requirements and best practices.

Page 3: Information assurance and security metrics

Access Control: These controls include aspects of logical and physical security focused on controlling the access to information system resources. Different levels of controls are considered (i.e. organizational, operational or technical) and they address a variety of security aspects such as system boundaries, identification and authentication, user authorization, sensitive system resources and, audit and monitoring.

Configuration Management: Focused on guaranteeing a secure information system configuration (i.e. the systems operate as intended and are securely configured) and ensuring that efficient configuration change processes exist and are followed.

Contingency Planning: Controls related with continuity of information system operations under unexpected events.

Business Process Application Controls: Controls applied to business transaction flows, essentially related with completeness,

accuracy, validity and confidentiality of data during its processing within the application. Areas of control include Data Input such as data validation and edit checks; Data Processing such as completeness of transactions and log reviews; Data Output and Master Data Setup and Maintenance.

Interface Application Controls: Focused on guaranteeing an adequate data transmission between applications as well as clean migration of data during conversion procedures.

Data Management System Controls: Data management systems include database management systems, data warehouse software and data extraction and reporting software. These controls enforce authentication/authorization, data access privileges to the data management systems.

Figure 1: The flow of Information Security Scheme.

In the following we list the criteria that might help stakeholders (appropriate authority in place for support to achieve that control) to assign priorities to the different controls.

High: Controls associated with high level risks. If the security control fails, the information system is exposed to a high negative impact for the operation. Fails in high impact controls might cause the company to go to bankruptcy. Regardless the type of organization, all the controls associated with preserving human lives have to be prioritized as High.

Medium: Controls associated with medium level risks. If the security control fails the company might incur in considerable economic losses, lost of reputation or lost of customers.

Low: Controls associated with low level risks. Productivity looses and minimum economic impacts are incurred if the security control is not in place.

Normally, Information Criteria refers to the fundamental aspects of Information Security, namely confidentiality, integrity and availability. Additionally, in our scheme, we consider Accountability. Each security control should address directly or indirectly one or more of these aspects. Furthermore, each entity might have specific security requirements imposed by its particular security goals. The reason why information criteria is considered is because that would allow us to measure the level of security for each aspect of information security, identify strengths

Page 4: Information assurance and security metrics

and weaknesses and evaluate if the security requirements for each aspect of security are being met. In addition, it permit us to weight accurately the importance of security controls so that we can come up with a final security level taking into consideration the particular security goals and values of the fields for each security control are primary (P) or secondary (S). When the field is marked as primary; that means that the security control is directly related with that aspect of information security; similarly, the field is marked as secondary when the security control is indirectly related with that information criterion. No mark is filled in the field if the security control does not address that aspect at all. Primary and secondary marks allow to wisely weight controls, assigning a higher weight to the ones marked with “P”, lower weight to the ones marked with “S” and no weight to the ones markless. The numerically measurable field is included to enrich our scheme with the inclusion of security performance metrics. We differentiate between two areas of measurement: Measuring the security level of a certain Information System and, establishing performance metrics to evaluate efficiency/ effectiveness and impact of specific security controls the two areas are nevertheless tight and the security level is determined based on the level of compliance with the established benchmark (i.e. perfect security), considering priorities and performance for applicable metrics. On the other hand, performance metrics require specific data to be collected and analyzed as well as the establishment of thresholds and a rule of measurement (i.e. percentage, relative or absolute value). If a security control is marked with “TRUE” in the field, this particular control becomes a candidate to be a performance metric. The actual election of the Performance Metrics has to be done considering High priority controls and the particular requirements of the stakeholder.

5. Performance Security Metrics Security controls that are quantitatively measurable with a

high priority are good candidates to be performance security metrics. Performance security metrics should be goal oriented and carefully chosen. There is no need to have a big number of performance security metrics but a small number that represent important aspects of the security and provide insightful information about the implemented security countermeasures which should help to evaluate the achievement of the security goals.

Once a metric is chosen, it should be matched with a top level security or organizational goal. A measurement method should be established, that is, the process to convert collected data into a percentage or number. In [4], the measurement method is called “Formula”. In addition, the collection method should be defined (i.e. automatic/manual collection, survey or interview based, etc.). The source of data must be specified (i.e. database, system log, personnel, etc.). The threshold criteria define the different security levels that can be assigned depending on the actual value obtained. A metric should be defined, either percentage or number and, based on that, a security level will be established. The frequency of collection and reporting should be established according to the particular needs and resources. Collecting and analyzing data could be a tough task; therefore, a trade-off should be considered between the benefit and the resources required. In the stakeholder field, the responsible(s) of this metric should be listed, the same way it is listed on the control matrix.

6. Measurement Model The measurement model has to process many inputs to derive

a final security level. The inputs for the model can be grouped as shown in table 1. Security Performance Metrics are selected high priority security controls quantitatively measurable. We assign a value between 0 and 1 to these kinds of controls considering that each security performance metric should have defined thresholds to evaluate the corresponding performance of that control. The Security Performance Metric Value (SPMV) is the value between 0 and 1 assigned to the corresponding security control. It is calculated as the ratio between the measured and maximum values for that metric.

Table 1: Security Performance Metrics Considerations

Prim & Secon controls

Confidentiality

Integrity

Availability

Accountability

Technical H,M,L H,M,L H,M,L H,M,L

Operational H,M,L H,M,L H,M,L H,M,L

Organizational H,M,L H,M,L H,M,L H,M,L

Sec Perf Metrc Value (SPMV) = Current value of metric / Maximum possible value for that metric

Where, 0 ≤ SPVM ≤ 1.

When a Security Performance Metric (SPM) is given a value of 0, we basically consider that security control not implemented or not effective at all. Similarly, when a SPM has a value of 1, we consider it as totally implemented. Values between minimum and maximum represent the level of efficacy of the control. Then, we can add up the values of all the SPM:

Total Value of SPMs = TVSPM = Σ SPMV

Therefore, to calculate the percentage of implemented controls, we use the general formula:

Ratio of IC = (Total number of non-quantatively measurable

passed controls + TVSPM) / Total number of evaluated controls

6.1 Priority Weight Factors To calculate the Weight Factors for each Priority (High,

Medium, Low), we use a linear scale from 1 to 10 and assign a value to each priority. In addition, we need to consider the case where at least one of the priorities has not been assigned to any of the controls listed on the control matrix. For example, since the priorities are set based on a previous risk assessment, there exist the possibility that all the controls are assigned only high and medium priorities but not low priorities. Similarly, any priority might not be assigned to a particular triple. Therefore, to obtain normalized weight factors, we need to figure out which priorities of controls are going to be used during the analysis. Hence, we need to post the following question: Are there any high /medium /low priority control listed on the control matrix? These questions have to be answered for each information criteria as well. The results for each information criteria should be summarized as :

Page 5: Information assurance and security metrics

Where each row corresponds to one of the different types of controls (i.e. Technical, Operational, Organizational and Total) and each column correspond to a different priority (i.e. High, Medium, Low). Therefore, the values in row number one in the matrix above should be interpreted as follows: The control matrix includes high priority technical controls, and low priority technical controls but no medium priority technical controls.

Then, using the linear scaled values for High, Medium and Low priorities, we can obtain the weighted factors as follows:

Where HSV, MSV and LSV are scalars values assigned to prioritize high, medium and low level security controls. Then we obtain the scalars used to normalize the priority weight factors as follows: Technical Normalizing scalar (TeNS) = PM(1,1) + PM(1,2) + PM(1,3) Operational Normalizing scalar (OpNS) = PM(2,1) + PM(2,2) + PM(2,3) Organizational Normalizing scalar (OrNS )= PM(3,1) + PM(3,2) +P M(3,3) Total Normalizing scalar (ToNS) =PM(4,1) + PM(4,2) + PM(4,3)

Finally, we derive the Priorities Normalized Weight Matrix (PNWM) by multiplying each row of Priority Matrix by the corresponding inverses of the normalizing scalars, as follows:

PNWM (1, 1-3) = PM (1, 1-3) / TeNS PNWM (2, 1-3) = PM (2, 1-3) / OpNS PNWM (3, 1-3) = PM (3, 1-3) / OrNS PNWM (4, 1-3) = PM (4, 1-3) / ToNS Where, each element of the PNWM represents the weight

factors for technical, operational, organizational and total security controls for all the three priorities (high, medium and low). When all the elements of the Priority Existence Matrix are “TRUE”, the weight factors are all the same for technical, operational, organizational and total controls. For example, if the user assigns scale values of 10, 5, 2 to high, medium and low priorities controls, the weighted factors can be calculated as follows:

HPWF = 10/17 = 0.5882 MPWF = 5/17 = 0.2941 LPWF = 2/17 = 0.1176 The Priority Weight Factors are then used to calculate the

different security levels as follows: SL = (HPWF) (%HPIC) + (MPWF) (%MPIC) Where, HPIC = High Priority Implemented Controls MPIC = Medium Priority Implemented Controls LPIC = Low Priority Implemented Controls

6.2 Information Criteria Weight Factors Similar as we calculate the Priority Weight Factors, we use a

linear scale for 1 to 10 and assign a value to each aspect of Information Criteria (i.e. Confidentiality, Integrity Availability and Accountability).

We also need to consider the case where at least one of the aspects of Information criteria is not represented in the control matrix. This is not likely to happen but we consider and solve this in a similar way as in the priority weight factors. For example, it might be the case when the Control Matrix do not list

organizational accountability controls. Therefore, to obtain normalized weight factors, we need to figure out if all the information criteria aspects are represented in the control matrix for all the perspectives. Hence, we need to post the following question: Are there any control listed on the control matrix addressing Confidentiality/ Integrity/ Availability/ Accountability? The results can be summarized in a matrix as:

Where each row corresponds to one of the different types of controls (i.e. Technical, Operational, Organizational and Total) and each column correspond to a different aspect of information criteria (i.e. Confidentiality, Integrity, Availability, Accountability). Therefore, the values in row number two in the matrix above should be interpreted as follows: The control matrix includes confidentiality, availability and accountability operational controls but not integrity operational controls. Then, using the linear scaled values for Confidentiality, Integrity, Availability and Accountability, we can obtain the weighted factors as follows:

Where CSV, ISV, ASV and AcSV are scalars values assigned to prioritize confidentiality, integrity, availability and accountability information criteria. Then we obtain the scalars used to normalize the weight factors as follows: Technical Normalizing scalar (TeNS) = ICM(1,1) + ICM(1,2) + ICM(1,3)+ ICM(1,4) Operational Normalizing scalar (OpNS) = ICM(2,1) + ICM(2,2) + ICM(2,3) + ICM(2,4) Organizational Normalizing scalar (OrNS )= ICM(3,1) + ICM(3,2) + ICM(3,3) + ICM(3,4) Total Normalizing scalar (ToNS) =ICM(4,1) + ICM(4,2) + ICM(4,3) + ICM(4,4)

Finally, we derive the IC Normalized Weight Matrix (ICNWM) by multiplying each row of IC Matrix by the corresponding inverses of the normalizing scalars, as follows:

ICNWM(1, 1-4) = ICM(1,1,-4)/TeNS ICNWM(2, 1-4) = ICM(2,1,-4)/OpNS ICNWM(3, 1-4) = ICM(3,1,-4)/OrNS ICNWM(4, 1-4) = ICM(4,1,-4)/ToNS Where, each element of the ICNWM represents the weight

factors for technical, operational, organizational and total security controls for all the aspects of Information Criteria. When all the elements of the Information Criteria Existence Matrix are “TRUE”, the weight factors are all the same for technical, operational, organizational and total controls. For example, if the user assigns scale values of 8, 10, 7 and 6 to Confidentiality, Integrity, Availability and Accountability, respectively; the weigh factors can be calculated as follows:

CWF = 8/31 = 0.2580 IWF= 10/31 = 0.3225 AWF = 7/31 = 0.2258 AcWF = 6/31 = 0.1935

Page 6: Information assurance and security metrics

The Information Criteria Weight Factors are used to calculate the final security level as follows:

ISSL = (CWF)(CSL) + (IEF)(ISL) +(AWF)(ASL) + (AcWF)(AcSL) Where, IS_SL = Information System Security Level CSL = Confidentiality Security Level ISL = Integrity Security Level ASL = Availability Security Level AcSL = Accountability Security Level

7. Concluding Remarks and Future Work We have presented a new approach to measure and evaluate

security at organizational, information system or applications with different evaluation perspectives (i.e. Technical, Operational or Organizational). Our scheme leverages many of the features of current Risk, IT Management and Audit frameworks, Security Guidelines and Standards. This allows us to include essential elements of Risk Management, Control and Audit and a process oriented classification of security controls and metrics to deliver a flexible, goal-driven and easily updatable measurement.

An important improvement to this scheme is the creation of a catalog of security goals. In a similar fashion [25] links Business Goals with IT Goals, we can match Business Goals with IT Security Goals. Furthermore, once the catalog is defined, we can map security goals with security controls so that, according with the security goals required, priorities will be assigned to each security control. This will help to automate the prioritization task and at the same time evolve the scheme to be also Business Oriented. Graphical depiction may help in the interpretation of the results. The collection of the information for security performance metrics is out of the scope of this scheme; however, methods for automatic collection, analysis and reporting are implemented.

The main benefits of our work are as follows: customized and risk based, helpful to valuate effectiveness of countermeasures and perform benchmark against industry standards and regulations, easy identification of weaknesses and mapping with stakeholders, and view of security from different perspectives. Information security is a dynamic field; new technologies, vulnerabilities, threats, security standards, etc. emerge constantly. Therefore, a constant update of the security controls and metrics should be performed. The security controls in our proposal are nor unique neither exhaustive and can be improved by experts in different areas of information security. Experts in network security, software vulnerabilities, risk management, software assurance, etc, are welcome to collaborate in the creation of security control and in the general improvement of the scheme.

8. References: [1] National Institute of Standards and Technology Special

Publication 800-30, Risk Management Guide for Information Technology Systems, June 2001

[2] National Institute of Standards and Technology Special Publication 800-53, Recommended Security Controls for Federal Information Systems, December 2007

[3] National Institute of Standards and Technology Special Publication 800-53A, Guide for Assessing the Security Controls in Federal Information Systems, June 2008

[4] National Institute of Standards and Technology Special Publication 800-55, Performance Measurement Guide for Information Security, July 2008.

[5] Practical Software and Systems Measurement: Practical Measurement Framework for Software Assurance and Information Security, October 2008.

[6] ISO/IEC 15939, Software Engineering - Software Measurement Process

[7] The ISO 27001:2005 Information Security Management Systems (ISMS)

[8] ISO/IEC 27004, Information technology - Security techniques - Information security management measurement

[9] CMMI® (Capability Maturity Model Integration)

[10] https://buildsecurityin.us-cert.gov/daisy/bsi/home.html (as of June 8, 2009)

[11] http://measurablesecurity.mitre.org/ (as of June 10, 2009)

[12] Reijo M. Savola. Towards a Taxonomy for Information Security Metrics. Proceedings of the 2007 ACM workshop on Quality of protection.

[13] Reijo M. Savola. Towards a Security Metrics Taxonomy for the Information and Communication Technology Industry. International Conference on Software Engineering Advances (ICSEA 2007).

[14] Thomas Heyman et al. Using Security Patterns to Combine Security Metrics. The Third International Conference on Availability, Reliability and Security, IEEE 2008.

[15] A. Jaquith. Security metrics: replacing fear, uncertainty, and doubt. Addison-Wesley, 2007.

[16] O. Sami Saydjari. Is Risk a good security metric?. Proceedings of the 2nd ACM workshop on Quality of protection, October 2006.

[17] Andy Ju An Wang. Information Security Models and Metrics. Proceedings of the 43rd annual Southeast regional conference. March 2005.

[18] S. Chandra et al. Software Security Metric Identification Framework (SSM). Proceedings of the International Conference on Advances in Computing, Communication and Control. January 2009.

[19] National Institute of Standards and Technology Special Publication 800-64 Security Considerations in the System Development Life Cycle. October 2008.

[20] FIPS 199 Standard for Security Categorization of Federal Information and Information Systems.

[21] FIPS 200 Minimum Security Requirements for Federal Information and Information Systems.

[22] OWASP Application Security Verification Standard 2008.

[23] Peter Mell et al. Common Vulnerability Scoring System V.2., June 2007.

[24] Federal Information System Control Audit Manual (FISCAM), US Govt. Accountability Office, February 2009.

[25] Control Objectives for Information and related Technologies 4.1 (COBIT 4.1), IT Governance Institute.

[26] Payment Card Industry (PCI), Data Security Standard, Requirements and Security Assessment Procedures, October 2008.

[27] Pete Herzog, Open-Source Security Testing Methodology Manual (OSSTMM 2.1), Institute for Security and Open Methodologies.