Splunk 6.4 - NIAP CCEVS · 5.1 Assessment of the Splunk Test Environment ... cryptographic functions ... establishing all reference identifiers from the application-configured reference

1 of 42

Assurance Activities Report for a Target of Evaluation

Splunk 6.4.5

Security Target (Version 1.0)

Assurance Activities Report (AAR) Version 1.0

2/1/2017

Evaluated by:

Booz Allen Hamilton Common Criteria Test Laboratory NIAP Lab # 200423 304 Sentinel Drive, Annapolis Junction, MD 20701

Prepared for: National Information Assurance Partnership

Common Criteria Evaluation and Validation Scheme

2 of 42

The Developer of the TOE:

Splunk Inc., 250 Brannan Street

San Francisco, CA 94107

The Author of the Security Target: Booz Allen Hamilton,

304 Sentinel Drive, Annapolis Junction, MD 20701 USA

The TOE Evaluation was sponsored by:

Splunk Inc., 250 Brannan Street

San Francisco, CA 94107

Evaluation Personnel: Christopher Gugel, CC Technical Director

Herbert Markle Brad Isbell

Christopher Rakaczky

Applicable Common Criteria Version Common Criteria for Information Technology Security Evaluation, September 2012 Version 3.1 Revision 4

Common Evaluation Methodology Version

Common Criteria for Information Technology Security Evaluation, Evaluation Methodology, September 2012 Version 3.1 Revision 4

3 of 42

Table of Contents

1 Purpose ............................................................................................................................................... - 2 - 2 TOE Summary Specification Assurance Activities ............................................................................ - 2 - 3 Operational Guidance Assurance Activities ....................................................................................... - 6 - 4 Security Assurance Requirements .................................................................................................... - 10 - 5 Test Assurance Activities (Test Report) ........................................................................................... - 12 -

5.1 Assessment of the Splunk Test Environment .......................................................................... - 13 - 5.1.1 Physical Assessment ........................................................................................................... - 13 - 5.1.2 Logical Assessment ............................................................................................................ - 13 -

5.2 Test Cases ............................................................................................................................... - 13 - 5.2.1 Cryptographic Support........................................................................................................ - 14 - 5.2.2 Identification and Authentication ....................................................................................... - 24 - 5.2.3 User Data Protection ........................................................................................................... - 30 - 5.2.4 Security Management ......................................................................................................... - 31 - 5.2.5 Privacy ................................................................................................................................ - 33 - 5.2.6 Protection of the TSF .......................................................................................................... - 33 -

5.3 Vulnerability Testing .............................................................................................................. - 38 - 6 Conclusions ...................................................................................................................................... - 39 - 7 Glossary of Terms ............................................................................................................................ - 39 -

02/01/2017 CC TEST LAB #200423-0

Page - 1 -

02/01/2017 CC TEST LAB #200423-0

Page - 2 -

1 Purpose The purpose of this document is to serve as a non-proprietary attestation that this evaluation has satisfied all of the TSS, AGD, and ATE Assurance Activities required by the Protection Profiles/Extended Packages to which the TOE claims exact conformance.

2 TOE Summary Specification Assurance Activities The evaluation team completed the testing of the Security Target (ST) ‘Splunk Enterprise 6.4.5 Security Target’ and confirmed that the TOE Summary Specification (TSS) contains all Assurance Activities as specified by the ‘Protection Profile for Application Software, Version 1.2’. The evaluators were able to individually examine each SFR’s TSS statements and determine that they comprised sufficient information to address each SFR claimed by the TOE as well as meet the expectations of the APP PP Assurance Activities. Through the evaluation of ASE_TSS.1-1, described in the ETR, the evaluators were able to determine that each individual SFR was discussed in sufficient detail in the TSS to describe the SFR being met by the TSF in general. However, in some cases the Assurance Activities that are specified in the claimed source material instruct the evaluator to examine the TSS for a description of specific behavior to ensure that each SFR is described to an appropriate level of detail. The following is a list of each SFR, the TSS Assurance Activities specified for the SFR, and how the TSS meets the Assurance Activities. Additionally, each SFR is accompanied by the source material (App PP) that defines where the most up-to-date TSS Assurance Activity was defined. FCS_CKM_EXT.1 – The evaluator shall inspect the application and its developer documentation to determine if the application needs asymmetric key generation services. If not, the evaluator shall verify the generate no asymmetric cryptographic keys selection is present in the ST. No documentation was found that identified a need for asymmetric keys was found. This SFR selection is “generate no asymmetric cryptographic keys” selection. The ST has the correct selection. FCS_CKM.2 - The evaluator shall ensure that the supported key establishment schemes correspond to the key generation schemes identified in FCS_CKM.1.1 (if applicable). If the ST specifies more than one scheme, the evaluator shall examine the TSS to verify that it identifies the usage for each scheme. The TOE uses only one scheme and that is the mandated selection required for FCS_CKM.2 with no other schemes selected for additional schemes supported. Section 8.2.2 of the ST states that the TOE supports RSA key establishment schemes for establishment of TLS/HTTPS communications. RSA key establishment conforms to NIST SP 800-56B. This function is vendor-asserted under FIPS 140-2 IG D.4, Vendor Affirmation. FCS_COP.1(1) – This SFR does not contain any App PP TSS Assurance Activities. FCS_COP.1(2) – The evaluator shall check that the association of the hash function with other application cryptographic functions (for example, the digital signature verification function) is documented in the TSS. Section 8.2.4 of the ST states that the TOE states that the TOE performs cryptographic hashing in support of TLS/HTTPS and that both SHA-1 and SHA-256 are supported. FCS_COP.1(3) – This SFR does not contain any App PP TSS Assurance Activities. FCS_COP.1(4) – This SFR does not contain any App PP TSS Assurance Activities. FCS_HTTPS_EXT.1 – This SFR does not contain any App PP TSS Assurance Activities.

02/01/2017 CC TEST LAB #200423-0

Page - 3 -

FCS_RBG_EXT.1 – If “implement DRBG functionality” is selected, the evaluator shall ensure that additional FCS_RBG_EXT.2 elements are included in the ST. The evaluator verified that FCS_RBG_EXT.2 was included in the ST. FCS_RBG_EXT.2 – This SFR does not contain any App PP TSS Assurance Activities. FCS_STO_EXT.1.1 – The evaluator shall check the TSS to ensure that it lists all persistent credentials (secret keys, PKI private keys, or passwords) needed to meet the requirements in the ST. For each of these items, the evaluator shall confirm that the TSS lists for what purpose it is used, and how it is stored. The Section 8.2.9 of the ST provides a table of the credentials that the TOE uses that meet the requirements in the ST along with their purpose. The credentials are stored in the GNOME keyring. FCS_TLSC_EXT.1 – The evaluator shall check the description of the implementation of this protocol in the TSS to ensure that the cipher suites supported are specified. The evaluator shall check the TSS to ensure that the cipher suites specified include those listed for this component. Section 8.2.10 of the ST lists the following as the supported ciphersuites used in TLS 1.2:

TLS_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA256 TLS_RSA_WITH_AES_128_CBC_SHA256 FCS_TLSC_EXT.1.2 – The evaluator shall ensure that the TSS describes the client’s method of establishing all reference identifiers from the application-configured reference identifier, including which types of reference identifiers are supported (e.g. Common Name, DNS Name, URI Name, Service Name, or other application-specific Subject Alternative Names) and whether IP addresses and wildcards are supported. The evaluator shall ensure that this description identifies whether and the manner in which certificate pinning is supported or used by the TOE. Section 8.2.10 of the ST states The TOE will perform several TLS functions identically regardless of whether it is acting as a client or as a server. It will validate the peer certificate used for the connection. Mutual authentication is supported and can be enabled/disabled administratively. The TOE does not support the use of Wildcards or IP Addresses. The TOE can be configured within the .conf files to verify Common Name (CN) and/or Subject Alternative Names (SAN) reference identifiers. Section 8.4.1 of the ST describes the algorithm as to how a certificate is validated. FCS_TLSC_EXT.1.3 – This SFR does not contain any App PP TSS Assurance Activities.

FCS_TLSS_EXT.1 – The evaluator shall check the description of the implementation of this protocol in the TSS to ensure that the cipher suites supported are specified. The evaluator shall check the TSS to ensure that the cipher suites specified include those listed for this component. The evaluator shall also check the operational guidance to ensure that it contains instructions on configuring the TOE so that TLS conforms to the description in the TSS. Section 8.2.10 of the ST lists the following as the supported ciphersuites used in TLS 1.2:

TLS_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA256 TLS_RSA_WITH_AES_128_CBC_SHA256

02/01/2017 CC TEST LAB #200423-0

Page - 4 -

FCS_TLSS_EXT.1.2 – The evaluator shall verify that the TSS contains a description of the denial of old SSL and TLS versions. Section 8.2.10 of the ST states that only TLS 1.2 is supported. All other connection requests are rejected. FCS_TLSS_EXT.1.3 – The evaluator shall verify that the TSS describes the key agreement parameters of the server key exchange message. Section 8.2.10 of the ST states that when acting as a TLS server, the TSF will generate 2048-bit RSA key establishment parameters.

FCS_TLSS_EXT.1.4 – This SFR does not contain any App PP TSS Assurance Activities. FCS_TLSS_EXT.1.5 – The evaluator shall ensure that the TSS description required per FIA_X509_EXT.2.1 includes the use of clientside certificates for TLS mutual authentication. Section 8.2.10 of the ST states that for intra-TOE transfer, mutual authentication using client-side X.509v3 certificates is used to establish the TLS session. FCS_TLSS_EXT.1.6 – If the TOE implements mutual authentication, the evaluator shall verify that the TSS describes how the DN and SAN in the certificate is compared to the expected identifier. Section 8.2.10 of the ST states The TOE will perform several TLS functions identically regardless of whether it is acting as a client or as a server. It will validate the peer certificate used for the connection. Mutual authentication is supported and can be enabled/disabled administratively. The TOE does not support the use of Wildcards or IP Addresses. The TOE can be configured within the .conf files to verify Common Name (CN) and/or Subject Alternative Names (SAN) reference identifiers. Section 8.4.1 of the ST describes the algorithm as to how a certificate is validated.

FDP_DEC_EXT.1 – For Linux: The evaluator shall verify that either the application software or its documentation provides a list of the hardware resources it accesses. The evaluator shall verify that either the application software or its documentation provides a list of sensitive information repositories it accesses. Section 8.3.2 of the ST consistently identifies the SFR selection of “network connectivity”. Section 8.3.2 of the ST consistently identifies the SFR selection of “no sensitive information repositories”. The evaluator found that the ST clearly states that the TOE relies on its underlying platform to provide network connectivity. The supplemental administrative guidance, Securing Splunk Enterprise with Common Criteria 6.4.5 guidance, is consistent with the declaration in the TOE. There are no other devices being configured for use. This was confirmed during testing. FDP_NET_EXT.1 - This SFR does not contain any App PP TSS Assurance Activities. FIA_X509_EXT.1 – The evaluator shall ensure the TSS describes where the check of validity of the certificates takes place. The evaluator ensures the TSS also provides a description of the certificate path validation algorithm. Section 8.4.1 of the ST states that the TOE provides an internal mechanism to perform certificate validation and then describes the steps/checks in order for the TOE to validate a certificate.

02/01/2017 CC TEST LAB #200423-0

Page - 5 -

FIA_X509_EXT.2 – The evaluator shall check the TSS to ensure that it describes how the TOE chooses which certificates to use, and any necessary instructions in the administrative guidance for configuring the operating environment so that the TOE can use the certificates. The evaluator shall examine the TSS to confirm that it describes the behavior of the TOE when a connection cannot be established during the validity check of a certificate used in establishing a trusted channel. The evaluator shall verify that any distinctions between trusted channels are described. Section 8.4.2 of the ST states that the TOE uses X.509 certificates for TLS\HTTPS authentication. The TOE can determine which X.509 certificates and keys to use through the use of .conf files. The TSS also states that if a certificate with unknown revocation status (because the TSF is unable to read the CRL) is accepted. The administrator is warned that the revocation check could not be performed. FMT_CFG_EXT.1 – The evaluator shall check the TSS to determine if the application requires any type of credentials and if the application installs with default credentials. Section 8.5.1 of the ST states that the TOE requires credentials for remote administration via the Web GUI and that the initial installation of the TOE creates an account with default credentials that is used for initial login. Once these credentials have been provided, the administrator is prompted to change the credentials before any other administrative actions can be performed. FMT_MEC_EXT.1 – The evaluator shall review the TSS to identify the application's configuration options (e.g. settings) and determine whether these are stored and set using the mechanisms supported by the platform. At a minimum the TSS shall list settings related to any SFRs and any settings that are mandated in the operational guidance in response to an SFR. Section 8.5.2 of the ST states that the TOE is capable of using the underlying platform’s recommend methods for storing and setting configuration options. In the TOE’s evaluated configuration, all configuration information related to the Splunk application is stored in /etc/opt/splunk. FMT_SMF.1 – This SFR does not contain any App PP TSS Assurance Activities. FPR_ANO_EXT.1 – The evaluator shall inspect the TSS documentation toidentify functionality in the application where PII can be transmitted. Section 8.6.1 of the ST states that the TOE does not collect personally identifiable information (PII) for administrators or users, therefore there is no case in which the TOE will transmit this data over the network. FPT_AEX_EXT.1.1 – The evaluator shall ensure that the TSS describes the compiler flags used to enable ASLR when the application is compiled. Section 8.7.1 of the ST states that the TOE was compiled using the -fstack-protector-strong compilation flag. FPT_AEX_EXT.1.5 – The evaluator shall ensure that the TSS section of the ST describes the compiler flag used to enable stack-based buffer overflow protection in the application. Section 8.7.1 of the ST states that the TOE was compiled using the -fstack-protector-strong compilation flag. FPT_API_EXT.1 – The evaluator shall verify that the TSS lists the platform APIs used in the application. The evaluator shall then compare the list with the supported APIs (available through e.g. developer accounts, platform developer groups) and ensure that all APIs listed in the TSS are supported.

02/01/2017 CC TEST LAB #200423-0

Page - 6 -

Section 8.7.2 of the ST references Appendix A for the list of dynamic libraries and system calls (APIs). The Appendix explains that “Splunk Enterprise ships almost all of the libraries and scripting languages Splunk requires to operate and does not depend on the platform. The list of third-party libraries shipped with Splunk is captured in Table 8-3 (FPT_LIB_EXT.1). So, scripting languages like Python/Lua/JS are part of the TOE and are not platform APIs leveraged by TOE and have been verified through the FPT_LIB_EXT.1 assurance activities. The ST then provided a list and table that identified the only exceptions where Splunk leverages platform APIs and dynamically links to platform provided libraries. The Appendix A table was created by the vendor by first making a list of system calls from the Splunk source code. These system calls were then mapped to the correct Unix library (.so) and then mapped to the correct Unix Package that the library is contained. This final table mapping the library and Unix package is presented in the ST. The evaluator examined the original system call list and the Appendix A table. The evaluator verified that:

• the system calls were contained in the libraries (.so), • the libraries were contained in the packages listed, • the packages and libraries are Unix operating system supported APIs as claimed, and • the packages existed on the platform.

FPT_LIB_EXT.1 – This SFR does not contain any App PP TSS Assurance Activities. FPT_TUD_EXT.1.6 – The evaluator shall verify that the TSS identifies how the application installation package and updates to it are signed by an authorized source. The definition of an authorized source must be contained in the TSS. The evaluator shall also ensure that the TSS (or the operational guidance) describes how candidate updates are obtained. Section 8.7.4 of the ST states that Splunk automatically checks to see if an update is available. Splunk will notify the users via the login screen that there is an update available. Splunk does not download the update automatically. After selecting the update URL, the user will be redirected to the authorized Splunk customer portal site where the customer must authenticate prior to being able must manually download the RPM package to the underlying platform. This package must then be manually installed using the platform’s RPM application by someone with root privilege. RPM can also be used to show the current version of the TOE. Splunk provides a public key that is installed to RPM in the evaluated configuration. An administrator can then run “rpm -K” in order to verify the update against the installed public key prior to installation. FTP_DIT_EXT.1 – This SFR does not contain any App PP TSS Assurance Activities.

3 Operational Guidance Assurance Activities The evaluation team completed the testing of the Operational Guidance, which includes the review of the “Splunk® Enterprise: Securing Splunk Enterprise with Common Criteria 6.4.5 (AGD) document, and confirmed that the Operational Guidance contains all Assurance Activities as specified by the ‘Protection Profile for Application Software, Version 1.2’ (App PP). The evaluators reviewed the App PP to identify the security functionality that must be discussed for the operational guidance. This is prescribed by the Assurance Activities for each SFR and the AGD SARs. The evaluators have listed below each of the SFRs defined in the App PP that have been claimed by the TOE (some SFRs are conditional or optional) as well as the AGD SAR, along with a discussion of where in the operational guidance the associated Assurance Activities material can be found. The AGD includes references to other guidance documents that must be used to properly install, configure, and operate the TOE in its evaluated configuration. The AGD and the other Splunk 6.4.5 guidance documents were reviewed to assess the Operational Guidance Assurance Activities. The AGD contains references to these documents in Chapter 4 and these references can also be found below: If an SFR is not listed, one of the following conditions applies:

02/01/2017 CC TEST LAB #200423-0

Page - 7 -

• There is no Assurance Activity for the SFR. • The Assurance Activity for the SFR specifically indicates that it is simultaneously satisfied by

completing a different Assurance Activity (a testing Assurance Activity for the same SFR, a testing Assurance Activity for a different SFR, or a guidance Assurance Activity for another SFR).

• The Assurance Activity for the SFR does not specify any actions to review the operational guidance.

The following references are used in this section of the document:

[1] Splunk Enterprise: Installation Manual 6.4.5 – contains general instructions on installing Splunk on different OS platforms.

[2] Splunk Enterprise: Securing Splunk Enterprise 6.4.5 – provides information on securing the application and data to reduce attack surfaces and mitigate the risk and impact of vulnerabilities.

[3] Splunk Enterprise: Securing Splunk Enterprise with Common Criteria 6.4.5 guidance document – contains specific steps to configure Splunk into the Common Criteria evaluated configuration. Several steps are redundant to [2] but provide more specific settings.

[4] Splunk Enterprise: Admin Manual 6.4.5 – provides information on how to administer the operational product.

[5] Online help

FCS_CKM_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities.

FCS_CKM.2 - The evaluator shall verify that the AGD guidance instructs the administrator how to configure the TOE to use the selected key establishment scheme(s).

In the section entitled Update Splunk Configuration Files with CC-compliant Settings of [3] Securing Splunk Enterprise with Common Criteria guidance document provides the steps in how to correctly configure the TOE to use only the RSA key exchange ciphersuites for the server.conf, web.conf (GUI communication – server), alert_actions.conf (smtp client), inputs.conf (for indexer communications-server), and output.conf (forwarder communications-client). The TOE when in CC configuration does not support any other type of key exchange. These steps were verified as part of ATE_IND and were found complete.

FCS_COP.1(1) – The evaluator checks the AGD documents to determine that any configuration that is required to be done to configure the functionality for the required modes and key sizes is present.

Splunk’s key size is not configurable there is no configuration required. The [4] Admin Manual does specify that the certificate keys must be 2048 bits at a minimum and in PEM format.

FCS_COP.1(2) – This SFR does not contain any App PP AGD Assurance Activities.



FCS_HTTPS_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities.

FCS_RBG_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities.

FCS_RBG_EXT.2 – This SFR does not contain any App PP AGD Assurance Activities.

FCS_STO_EXT.1.1 – This SFR does not contain any App PP AGD Assurance Activities.

02/01/2017 CC TEST LAB #200423-0

Page - 8 -

FCS_TLSC_EXT.1 – The evaluator shall also check the operational guidance to ensure that it contains instructions on configuring the TOE so that TLS conforms to the description in the TSS.

In the section entitled Configure Splunk Enterprise for Common Criteria of [3] Securing Splunk Enterprise with Common Criteria guidance document provides the steps in how to correctly configure the TOE to use only the RSA key exchange ciphersuites for the server.conf, web.conf (GUI communication – server), alert_actions.conf (smtp client), inputs.conf (for indexer communications-server), and output.conf (forwarder communications-client). The TOE when in CC configuration does not support any other type of key exchange. These steps were verified as part of ATE_IND and were found complete.

FCS_TLSC_EXT.1.2 – The evaluator shall ensure that the AGD guidance includes instructions for setting the reference identifier to be used for the purposes of certificate validation in TLS. In the section entitled Configure Splunk Enterprise for Common Criteria of [3] Securing Splunk Enterprise with Common Criteria guidance document provides the steps in how to correctly configure the TOE to use only the RSA key exchange ciphersuites for the server.conf, web.conf (GUI communication – server), alert_actions.conf (smtp client), inputs.conf (for indexer communications-server), and output.conf (forwarder communications-client). The TOE when in CC configuration does not support any other type of key exchange. These steps were verified as part of ATE_IND and were found complete. FCS_TLSC_EXT.1.3 – This SFR does not contain any App PP AGD Assurance Activities.

FCS_TLSS_EXT.1 – The evaluator shall also check the operational guidance to ensure that it contains instructions on configuring the TOE so that TLS conforms to the description in the TSS.


FCS_TLSS_EXT.1.2 – The evaluator shall verify that any configuration necessary to meet the requirement must be conained in the AGD guidance.

In the section entitled Configure Splunk Enterprise for Common Criteria of [3] Securing Splunk Enterprise with Common Criteria guidance document provides the steps in how to correctly configure the TOE to use only the RSA key exchange ciphersuites for the server.conf, web.conf (GUI communication – server), alert_actions.conf (smtp client), inputs.conf (for indexer communications-server), and output.conf (forwarder communications-client). The TOE when in CC configuration does not support any other type of key exchange. These steps were verified as part of ATE_IND and were found complete. FCS_TLSS_EXT.1.3 – The evaluator shall verify that any configuration guidance necessary to meet the requirement must be contained in the AGD guidance. In the section entitled Configure Splunk Enterprise for Common Criteria of [3] Securing Splunk Enterprise with Common Criteria guidance document provides the steps in how to correctly configure the TOE to use only the RSA key exchange ciphersuites for the server.conf, web.conf (GUI communication – server), alert_actions.conf (smtp client), inputs.conf (for indexer communications-server), and output.conf (forwarder communications-client). The TOE when in CC configuration does not support any other type of key exchange. These steps were verified as part of ATE_IND and were found complete. FCS_TLSS_EXT.1.4 – This SFR does not contain any App PP AGD Assurance Activities. FCS_TLSS_EXT.1.5 – The evaluator shall verify that the AGD guidance required per FIA_X509_EXT.2.1 includes instructions for configuring the client-side certificates for TLS mutual

02/01/2017 CC TEST LAB #200423-0

Page - 9 -

authentication.


FCS_TLSS_EXT.1.6 –The evaluator shall ensure that the AGD guidance includes configuration of the expected identifier or the directory server for the connection.


FDP_DAR_EXT.1 – The Linux platform currently does not provide data-at-rest encryption services which depend upon invocation by application developers. The evaluator shall verify that the Operational User Guidance makes the need to activate platform encryption clear to the end user. In the section entitled Prerequisites of [3] requirement #4 specifically indicates the 2 LUKS encrypted partitions should be available ofr $SPLUNK_HOME (/opt/splunk) and $SPLUNK_ETC (/etc/opt/splunk). The instructions provide a specific link on how to properly configure the LUKS partitions.

The evaluator also found that the ST makes the requirement for LUKS partition clear in Section 8.3.1: "The TOE relies on the underlying platform to provide data-at-rest encryption. In addition to securely storing credential data in the GNOME keyring (see section 8.2.9 above), the private keys and filesystem objects that comprise the TOE itself can be stored on a drive partition that is secured using Linux Unified Key Setup (LUKS) encryption."

FDP_DEC_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities. FDP_NET_EXT.1 - This SFR does not contain any App PP AGD Assurance Activities. FIA_X509_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities.

FIA_X509_EXT.2 – The evaluator shall check that the AGD guidance includes any necessary instructions in the administrative guidance for configuring the operating environment so that the TOE can use the certificates. If the requirement that the administrator is able to specify the default action, then the evaluator shall ensure that the operational guidance contains instructions on how this configuration action is performed.

In the section entitled Prerequisites specifically addresses of [3] Securing Splunk Enterprise with Common Criteria guidance document, the OS and SELinux requirements of installing RHEL with the Subscription Manager, SELinux installation and settings, GNOME key ring, and the LUKS encrypted partitions. Then as part of the Installation procedures address how to use the environment to install the TOE into its evaluated configuration.

In the section entitled Generate/obtain cc-compliant certificates of [3] Securing Splunk Enterprise with Common Criteria guidance document, specifically states that the TOE does not generate any cryptographic

02/01/2017 CC TEST LAB #200423-0

Page - 10 -

keys or certificates. It also states that the certificates the customer supplies must be FIPS compliant and in the PEM format. There are instructions as to creating, initializing and adding secrets to the Secret Storage (GNOME keyring). Configuring the TOE to update CRL information.

In the section entitled Configure Splunk Enterprise for Common Criteria of [3] Securing Splunk Enterprise with Common Criteria guidance document, provides the steps in how to correctly configure the TOE to use only the RSA key exchange ciphersuites for the server.conf, web.conf (GUI communication – server), alert_actions.conf (smtp client), inputs.conf (for indexer communications-server), and output.conf (forwarder communications-client). The TOE when in CC configuration does not support any other type of key exchange. These steps were verified as part of ATE_IND and were found complete. FMT_CFG_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities. FMT_MEC_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities. FMT_SMF.1 – The evaluator shall verify that every management function mandated by the PP is described in the operational guidance and that the description contains the information required to perform the management duties associated with the management function. Enabling/disabling the transmission of any information describing the system’s hardware, software, or configuration, Enabling/disabling supported TLS ciphersuites and Enabling/disabling TLS mutual authentication is accomplished via modifying the .conf files as shown in the [3] Securing Splunk Enterprise with Common Criteria guidance document and mirrored information is contained within the [1] Securing Splunk Enterprise and [4] Admin manual Querying the version of the TOE is covered in the [5] online help documentation under Determine which version of Splunk Enterprise you’re running (Splunk Web under the HelpAbout or the CLI command splunk –version. FPR_ANO_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities. FPT_AEX_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities. FPT_API_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities. FPT_LIB_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities.

FPT_TUD_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities.

FTP_DIT_EXT.1 – This SFR does not contain any App PP AGD Assurance Activities.

4 Security Assurance Requirements The following additional information is also required. AGD_OPE.1 If cryptographic functions are provided by the TOE, the operational guidance shall contain instructions for configuring the cryptographic engine associated with the evaluated configuration of the TOE.

In the section entitled Configure Splunk Enterprise for Common Criteria of [3] Securing Splunk Enterprise with Common Criteria guidance document provides the steps in how to correctly configure the TOE to use only the RSA key exchange ciphersuites for the server.conf, web.conf (GUI communication – server), alert_actions.conf (smtp client), inputs.conf (for indexer communications-server), and output.conf (forwarder communications-client). The TOE when in CC configuration does

02/01/2017 CC TEST LAB #200423-0

Page - 11 -

not support any other type of key exchange. These steps were verified as part of ATE_IND and were found complete.

• It shall provide a warning to the administrator that use of other cryptographic engines was not

evaluated nor tested during the CC evaluation of the TOE.

[3] Securing Splunk Enterprise with Common Criteria guidance document has the declaration as the first item under the Common Criteria Evaluation section.

• The document must describe the process for verifying updates to the TOE by verifying a digital

signature – this may be done by the TOE or the underlying platform. The evaluator shall verify that this process includes the following steps:

o Instructions for obtaining the update itself. This should include instructions for making the update accessible to the TOE (e.g., placement in a specific directory).

o Instructions for initiating the update process, as well as discerning whether the process was successful or unsuccessful. This includes generation of the hash/digital signature.

[3] Securing Splunk Enterprise with Common Criteria guidance document has a section called Update Splunk that describes the automatic checking for updates, manual download process to obtain the update, how to verify the integrity of the download. How to install the product is covered under the Installation of the Splunk .rpm section.

• The TOE will likely contain security functionality that does not fall in the scope of evaluation

under this PP. The operational guidance shall make it clear to an administrator which security functionality is covered by the evaluation activities.

[3] Securing Splunk Enterprise with Common Criteria guidance document defines the administrative functions that are in scope of the evaluation.

ALC_CMC.1

• The evaluator shall check the ST to ensure that it contains an identifier (such as a product name/version number) that specifically identifies the version that meets the requirements of the ST. The ST clearly and consistently states the version as 6.4.5.

• Further, the evaluator shall check the AGD guidance and TOE samples received for testing to

ensure that the version number is consistent with that in the ST. The AGD documents clearly indicate the version as 6.4.5.

• If the vendor maintains a web site advertising the TOE, the evaluator shall examine the

information on the web site to ensure that the information in the ST is sufficient to distinguish the product.

Splunk’s website and support site clearly delineates between different versions for both obtaining the product download as well as for online documentation help where one needs to select the correct version.

ALC_CMS.1 The evaluator shall ensure that the developer has identified (in guidance documentation for application developers concerning the targeted platform) one or more development environments appropriate for use in developing applications for the developer’s platform.

02/01/2017 CC TEST LAB #200423-0

Page - 12 -

• For each of these development environments, the developer shall provide information on how to configure the environment to ensure that buffer overflow protection mechanisms in the environment(s) are invoked (e.g., compiler flags).

• The evaluator shall ensure that this documentation also includes an indication of whether such protections are on by default, or have to be specifically enabled.

• The evaluator shall ensure that the TSF is uniquely identified (with respect to other products from the TSF vendor), and that documentation provided by the developer in association with the requirements in the ST is associated with the TSF using this unique identification.

Splunk has a whole online documentation line that supports the development of Apps for each of the different versions including the specific version of the TOE 6.4.5. Splunk (the TOE) is the development framework for building apps for the TOE. The TOE provides the libraries for app development, structure requirements, and integration requirements. The developer creates an app in the Splunk Web (App) Framework and supports only Simple XML, Simple XML jS/CSS extensions, HTML. The code can be created outside of the framework but must be imported as part of the integration process. The overflow protection is automatic with Splunk framework as it is compiled with the –fstack-protect-strong compiler flag as documented in the ST. Additionally, the OS should be configured per Splunk® Enterprise Securing Splunk Enterprise with Common Criteria 6.4.5

ALC_TSU_EXT.1 The evaluator shall verify that the TSS contains a description of the timely security update process used by the developer to create and deploy security updates.

• The evaluator shall verify that this description addresses the entire application. • The evaluator shall also verify that, in addition to the TOE developer’s process, any third-party

processes are also addressed in the description. • The evaluator shall also verify that each mechanism for deployment of security updates is

described.

Section 8.1 of the ST identifies the corporate policy on Timely Security Updates that fulfills the requirement. Any feedback that necessitates a fix will result in a patch to Splunk itself so there is no third-party update process to consider when updating the TOE. The TOE contains a number of components, including third party components that Splunk does not have control over the implementation of. Any implementation flaws are expected to be addressed within 90 days of reporting. This process was verified during the course of the evaluation.

The evaluator shall verify that, for each deployment mechanism described for the update process:

• The TSS lists a time between public disclosure of a vulnerability and public availability of the security update to the TOE patching this vulnerability, to include any third-party or carrier delays in deployment. The evaluator shall verify that this time is expressed in a number or range of days.

• The evaluator shall verify that this description includes the publicly available mechanisms (including either an email address or website) for reporting security issues related to the TOE.

• The evaluator shall verify that the description of this mechanism includes a method for protecting the report either using a public key for encrypting email or a trusted channel for a website.

In Section 8.1 of the ST identifies that implementation flaws are expected to be addressed within 90 days of reporting. Any feedback that necessitates a fix will result in a patch to Splunk itself so there is no third-party update process to consider when updating the TOE. Splunk provides customers with a support section on splunk.com where they have the ability to submit support issues. Splunk’s customer support is an HTTPS site that requires users to authentication prior to use.

5 Test Assurance Activities (Test Report) The following sections demonstrate that all ATE Assurance Activities for the TOE have been met. This evidence has been presented in a manner that is consistent with the “Reporting for Evaluations Against NIAP-Approved Protection Profiles” guidance that has been provided by NIAP. Specific test steps and

02/01/2017 CC TEST LAB #200423-0

Page - 13 -

associated detailed results are not included in this report in order for it to remain non-proprietary. The test report is a summarized version of the test activities that were performed as part of creating the Evaluation Technical Report (ETR).

5.1 Assessment of the Splunk Test Environment

5.1.1 Physical Assessment Splunk Headquarters located in San Francisco, CA is the physical location for the Splunk Enterprise test environment. Booz Allen reviewed the physical security controls of the test environment and interviewed Splunk employees to ensure that the Splunk Enterprise testing environment was secure. Booz Allen has found that Splunk Headquarters has similar access controls to Booz Allen’s CCTL. The Splunk location requires a person to be a Splunk employee to enter the building or be escorted as a visitor by a Splunk employee. The building is primarily controlled by a badge access system for employees whereas visitors must sign in and wear a temporary visitor nameplate. The laboratory where the Splunk devices are installed is a secured internal room located in the Splunk Headquarters location. Thus, physical access to the test devices would require a person to pass through the badge access control by being a Splunk employee or a visitor being escorted by a Splunk employee, and have a badge with the appropriate access privilege to the internal locked room where the servers are located. The evaluator had to sign in daily, be escorted, and be provided access into the server room. Then the evaluator conducted a daily inspection of the space and equipment for any signs of tampering of the space or equipment and found no such evidence of malicious tampering. Booz Allen finds that these physical access controls are satisfactory to protect the environment from unwanted physical access.

5.1.2 Logical Assessment The functional testing was not executed remotely from the physical test environment. The only way to access the test platform devices was to connect to the local test network that the TOE resides on and was built for common criteria functional testing specifically. At the end of each work day, the evaluators saved any configuration that was performed according to the AGD and shutdown the devices. A file listing of the TOE directory structure was created with a hash of each file and compared to the previous day to determine if any changes had been made. At times during the testing, Splunk personnel performed changes to the configuration of the test environment to support testing. Any configuration performed by Splunk personnel during the functional testing timeframe was conducted using the AGD as guidance and under the supervision of the evaluators. Booz Allen finds these logical access controls are satisfactory to protect the environment from unwanted logical access.

5.2 Test Cases The evaluation team completed the functional testing activities within the vendor’s test environment. The evaluation team conducted a set of testing that includes all ATE Assurance Activities as specified by the ‘Protection Profile for Application Software, version 1.2 [App PP]. The evaluators reviewed the App PP to identify the security functionality that must be verified through functional testing. This is prescribed by the Assurance Activities for each SFR. If an SFR is not listed, one of the following conditions applies:

• The Assurance Activity for the SFR specifically indicates that it is simultaneously satisfied by completing a test Assurance Activity for a different SFR.

• The Assurance Activity for the SFR does not specify any actions related to ATE activities which might be based on an SFR selection or assignment.

Note that some SFRs do not have Testing Assurance Activities associated with them at the element level (e.g. FPT_API_EXT.1.1). In such cases, testing for the SFR is considered to be satisfied by completion of all Assurance Activities at the component level.

02/01/2017 CC TEST LAB #200423-0

Page - 14 -

The following lists for each ATE Assurance Activity, the test objective, test instructions, test steps, and test results. Note that unless otherwise specified, the test configuration is to be in the evaluated configuration as defined by the AGD. For example, some tests require the TOE to be brought out of the evaluated configuration to temporarily disable cryptography to prove that the context of transmitted data is accurate. As part of the cleanup for each test, the TOE is returned to the evaluated configuration.

5.2.1 Cryptographic Support Test cases for FCS_CKM.1, FCS_CKM.2, FCS_COP.1(1), FCS_COP.1(2), FCS_COP.1(3), FCS_COP.1(4), and FCS_RBG_EXT.2 are not included within this section. This is because the ATE Assurance Activities have been satisfied by the vendor having the TOE's algorithms assessed under Cryptographic Algorithm Validation Program (CAVP). As part of CAVP validation the TOE’s cryptographic algorithms went through CAVS testing which directly maps to these SFRs’ ATE Assurance Activities. Refer to the results of the CAVP validation for the certificates listed within the Security Target. Test Case Number 002 SFR FCS_STO_EXT.1 – Storage of Secrets Test Objective For Linux: The evaluator shall verify that all keys are stored using Linux keyrings. Test Instructions Execute this test per the test steps. Test Steps 1. Log into OS of Indexer (001 inputs.conf is used).

2. Obtain root privilege. 3. Access the GNOME keyring on the underlying operating. 4. Compare output to Table 8-1 in the ST.

Test Results Pass Execution Method Manual Test Case Number 003 SFR FCS_TLSC_EXT.1.1 – TLS Client Protocol– Test 1 Test Objective The evaluator shall also perform the following tests:

Test 1: The evaluator shall establish a TLS connection using each of the cipher suites specified by the requirement. This connection may be established as part of the establishment of a higher-level protocol, e.g., as part of an EAP session. It is sufficient to observe the successful negotiation of a cipher suite to satisfy the intent of the test; it is not necessary to examine the characteristics of the encrypted traffic in an attempt to discern the cipher suite being used (for example, that the cryptographic algorithm is 128-bit AES and not 256-bit AES).

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TOE Client to use TLS_RSA_WITH_AES_128_CBC_SHA.

2. Start traffic capture between TOE Client and TLS server. 3. Execute functionality that would stimulate traffic between TLS server and

TOE. 4. Stop capture. Observe that the cipher suite that was negotiated by the TOE is

consistent with Step 1. 5. Repeat steps 1-4 for each ciphers selected in ST: AES128-SHA256,

AES256-SHA256 6. Repeat entire test for each interface required: TOE to SMTP Server and TOE to

TLS server. Test Results Pass Execution Method Manual Test Case Number 004 SFR FCS_TLSC_EXT.1.1 – TLS Client Protocol – Test 2

02/01/2017 CC TEST LAB #200423-0

Page - 15 -

Test Objective Test 2: The evaluator shall attempt to establish the connection using a server with a server certificate that contains the Server Authentication purpose in the extendedKeyUsage field and verify that a connection is established. The evaluator will then verify that the client rejects an otherwise valid server certificate that lacks the Server Authentication purpose in the extendedKeyUsage field and a connection is not established. Ideally, the two certificates should be identical except for the extendedKeyUsage field.

Test Instructions Execute this test per the test steps. Test Steps 1. Load a valid server certificate containing the Server Authentication purpose in

the extendedKeyUsage field on the TLS server. 2. Initiate a connection from the TOE Client to the TLS server. 3. Validate that the connection was successful.

4. Repeat steps 1-4 with a valid server certificate that does not contain the Server

Authentication purpose correctly set in the extendedKeyUsage field. 5. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 005 SFR FCS_TLSC_EXT.1.1 – TLS Client Protocol – Test 3 Test Objective Test 3: The evaluator shall send a server certificate in the TLS connection that does

not match the server-selected cipher suite (for example, send a ECDSA certificate while using the TLS_RSA_WITH_AES_128_CBC_SHA cipher suite or send a RSA certificate while using one of the ECDSA cipher suites.) The evaluator shall verify that the TOE disconnects after receiving the server’s Certificate handshake message.

Test Instructions Execute this test per the test steps.

Test Steps 1. Load a certificate with unsupported cipher on TLS server. 2. Initiate a connection from the TOE Client to the TLS server. 3. Validate that the connection fails

Test Results Pass Execution Method Manual Test Case Number 006 SFR FCS_TLSC_EXT.1.1 – TLS Client Protocol – Test 4 Test Objective Test 4: The evaluator shall configure the server to select the

TLS_NULL_WITH_NULL_NULL cipher suite and verify that the client denies the connection.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure the TLS server to only support the “aNULL, eNULL” cipher suite.

2. Initiate a connection from the TOE Client to the TLS server. 3. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 007 SFR FCS_TLSC_EXT.1.1 – TLS Client Protocol – Test 5 Test Objective Test 5: The evaluator shall perform the following modifications to the traffic:

Test 5.1 (a): Change the TLS version selected by the server in the Server Hello to a non-supported TLS version (for example 1.3 represented by the two bytes 03 04) and verify that the client rejects the connection.

02/01/2017 CC TEST LAB #200423-0

Page - 16 -

Test 5.2 (b): Modify at least one byte in the server’s nonce in the Server Hello handshake message, and verify that the client rejects the Server Key Exchange handshake message (if using a DHE or ECDHE cipher suite) or that the server denies the client’s Finished handshake message. Test 5.3 (c): Modify the server’s selected cipher suite in the Server Hello handshake message to be a cipher suite not presented in the Client Hello handshake message. The evaluator shall verify that the client rejects the connection after receiving the Server Hello. Test 5.4 (d)*: Modify the signature block in the Server’s Key Exchange handshake message, and verify that the client rejects the connection after receiving the Server Key Exchange message. Test 5.5 (e): Modify a byte in the Server Finished handshake message, and verify that the client sends a fatal alert upon receipt and does not send any application data. Test 5.6 (f): Send a garbled message from the Server after the Server has issued the ChangeCipherSpec message and verify that the client denies the connection.

Test Instructions Execute this test per the test steps. Test Steps 1. Begin capturing packets between the TOE Client and the TLS server.

2. Run the test program created during the setup on the test system. 3. Initiate a connection from the TOE Client to the TLS server. 4. Stop capturing packets between the TOE Client and the TLS server. 5. Validate that the connection fails. 6. Repeat steps 1-5 for each of the required modifications specified in App PP. 7. Repeat entire test for each interface required: TOE Client to SMTP Server and

TOE Client to Trusted TLS server. *NOTE: The lab received a TRRT response to Technical Query 214 that Test 5.4 should have been conditional depending on if a ciphersuite was claimed with DH. The ST does not claim a DH ciphersuite. Therefore, this test did not need to be performed.

Test Results Pass Execution Method Manual Test Case Number 008 SFR FCS_TLSC_EXT.1.2 – TLS Client Protocol – Test 1 Test Objective The evaluator shall configure the reference identifier according to the AGD

guidance and perform the following tests during a TLS connection: Test 1: The evaluator shall present a server certificate that does not contain an identifier in either the Subject Alternative Name (SAN) or Common Name (CN) that matches the reference identifier. The evaluator shall verify that the connection fails.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure the TOE Client to check for a CN and SAN with entries that do not

match the Trusted TLS server’s certificate. 2. Initiate a connection from the TOE Client to the Trusted TLS server. 3. Validate that the connection fails.

Test Results Pass

02/01/2017 CC TEST LAB #200423-0

Page - 17 -

Execution Method Manual Test Case Number 009 SFR FCS_TLSC_EXT.1.2 – TLS Client Protocol – Test 2 Test Objective The evaluator shall configure the reference identifier according to the AGD

guidance and perform the following tests during a TLS connection: Test 2: The evaluator shall present a server certificate that contains a CN that matches the reference identifier, contains the SAN extension, but does not contain an identifier in the SAN that matches the reference identifier. The evaluator shall verify that the connection fails. The evaluator shall repeat this test for each supported SAN type.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure the TOE Client to check for a CN and SAN with a CN entry that

matches and a SAN entry that does not match the Trusted TLS server’s certificate.

2. Initiate a connection from the TOE Client to the Trusted TLS server. 3. Validate that the connection fails.


guidance and perform the following tests during a TLS connection: Test 3: The evaluator shall present a server certificate that contains a CN that matches the reference identifier and does not contain the SAN extension. The evaluator shall verify that the connection succeeds.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure the TOE Client to only check for a CN with a CN entry that matches

the Trusted TLS server’s certificate. 2. Initiate a connection from the TOE Client to the Trusted TLS server. 3. Validate that the connection succeeds.


guidance and perform the following tests during a TLS connection: Test 4: The evaluator shall present a server certificate that contains a CN that does not match the reference identifier but does contain an identifier in the SAN that matches. The evaluator shall verify that the connection succeeds.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure the TOE Client to check for a CN that will not match and a SAN

that will match the Trusted TLS server’s certificate. 2. Initiate a connection from the TOE Client to the Trusted TLS server. 3. Validate that the connection succeeds.

Test Results Pass Execution Method Manual Test Case Number 012 SFR FCS_TLSC_EXT.1.2 – TLS Client Protocol – Test 5

02/01/2017 CC TEST LAB #200423-0

Page - 18 -

Test Objective The evaluator shall configure the reference identifier according to the AGD guidance and perform the following tests during a TLS connection: Test 5: The evaluator shall perform the following wildcard tests with each supported type of reference identifier: Test 5.1: The evaluator shall present a server certificate containing a wildcard that is not in the left-most label of the presented identifier (e.g. foo.*.example.com) and verify that the connection fails. Test 5.2: The evaluator shall present a server certificate containing a wildcard in the left-most label but not preceding the public suffix (e.g. *.example.com). The evaluator shall configure the reference identifier with a single left-most label (e.g. foo.example.com) and verify that the connection succeeds. The evaluator shall configure the reference identifier without a left-most label as in the certificate (e.g. example.com) and verify that the connection fails. The evaluator shall configure the reference identifier with two left-most labels (e.g. bar.foo.example.com) and verify that the connection fails. Test 5.3: The evaluator shall present a server certificate containing a wildcard in the left-most label immediately preceding the public suffix (e.g. *.com). The evaluator shall configure the reference identifier with a single left-most label (e.g. foo.com) and verify that the connection fails. The evaluator shall configure the reference identifier with two left-most labels (e.g. bar.foo.com) and verify that the connection fails.

Test Instructions Execute this test per the test steps. Test Steps The TOE claims no support for the use of wildcards. Therefore, all connection

attempts should fail.

1. Configure the Trusted TLS server’s certificate with a * in left-most label of the presented identifier (e.g. foo.*.example.com).

2. Configure the TOE Client to validate CN 3. Initiate a connection from the TOE Client to the Trusted TLS server. 4. Validate that the connection fails. 5. Repeat steps 1-4 but configure the Trusted TLS server’s certificate with a * in

left-most label but not preceding the public suffix (e.g. *.example.com). 6. Repeat steps 1-4 but configure the Trusted TLS server’s certificate with a * in

the left-most label immediately preceding the public suffix (e.g. *.com). 7. Repeat steps 1-6 but configure the TOE Client to validate the SAN instead.

Test Results Pass Execution Method Manual Test 13 URI or Service name and Test 14 pinned certificates are not supported by the TOE. Therefore, these two tests are not included. Test Case Number 015 SFR FCS_TLSC_EXT.1.3 – TLS Client Protocol Test Objective The evaluator shall use TLS as a function to verify that the validation rules in

FIA_X509_EXT.1.1 are adhered to and shall perform the following additional test: Test 1: The evaluator shall demonstrate that a peer using a certificate without a valid certification path results in an authenticate failure. Using the administrative guidance, the evaluator shall then load the trusted CA certificate(s) needed to validate the peer’s certificate, and demonstrate that the connection succeeds. The

02/01/2017 CC TEST LAB #200423-0

Page - 19 -

evaluator then shall delete one of the CA certificates, and show that the connection fails.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure mutual authentication between TOE and Trusted TLS server:

a. Configure TOE Client to use the 4 certificate chain. b. Configure the Trusted TLS server to use the 4 certificate chain.

2. Initiate a connection from the TOE Client to the Trusted TLS server. 3. Validate that the connection succeeds. 4. Delete intermediate certificate so Trusted TLS server only sends 3 of the

certificates. 5. Initiate a connection from the TOE Client to the Trusted TLS server. 6. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 016 SFR FCS_TLSS_EXT.1.1 – TLS Server Protocol– Test 1 Test Objective The evaluator shall also perform the following tests:

Test 1: The evaluator shall establish a TLS connection using each of the cipher suites specified by the requirement. This connection may be established as part of the establishment of a higher-level protocol, e.g., as part of an EAP session. It is sufficient to observe the successful negotiation of a cipher suite to satisfy the intent of the test; it is not necessary to examine the characteristics of the encrypted traffic in an attempt to discern the cipher suite being used (for example, that the cryptographic algorithm is 128-bit AES and not 256-bit AES).

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TOE Server to only accept TLS_RSA_WITH_AES_128_CBC_SHA.

2. Start traffic capture between TOE and TLS client. 3. Initiate a connection from the TLS client to TOE Server. 4. Stop capture. Observe that the cipher suite that was negotiated by the TOE is

consistent with Step 1.

5. Repeat steps 1-4 for each ciphers selected in ST: AES128-SHA256, AES256-SHA256.

6. Repeat entire test for each interface required: TOE Server to Trusted TLS client and TOE to Web Browser Client.

Test Results Pass Execution Method Manual Test Case Number 017 SFR FCS_TLSS_EXT.1.1 – TLS Server Protocol – Test 2 Test Objective The evaluator shall also perform the following tests:

Test 2: The evaluator shall send a Client Hello to the server with a list of cipher suites that does not contain any of the cipher suites in the server’s ST and verify that the server denies the connection. Additionally, the evaluator shall send a Client Hello to the server containing only the TLS_NULL_WITH_NULL_NULL cipher suite and verify that the server denies the connection.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure the TOE Server to accept only “aNULL, eNULL” cipher suite.

2. Initiate a connection from the TLS client to TOE Server. 3. Validate that the connection fails.

Test Results Pass

02/01/2017 CC TEST LAB #200423-0

Page - 20 -

Execution Method Manual Test Case Number 018 SFR FCS_TLSS_EXT.1.1 – TLS Server Protocol – Test 3 Test Objective The evaluator shall also perform the following tests:

Test 3: The evaluator shall use a client to send a key exchange message in the TLS connection that does not match the server-selected cipher suite (for example, send an ECDHE key exchange while using the TLS_RSA_WITH_AES_128_CBC_SHA cipher suite or send a RSA key exchange while using one of the DSA cipher suites.) The evaluator shall verify that the application disconnects after receiving the key exchange message.

Test Instructions Execute this test per the test steps. Test Steps 1. Load a certificate with unsupported cipher on TLS client.

2. Initiate a connection from the TLS client to TOE Server. 3. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 019 SFR FCS_TLSS_EXT.1.1 – TLS Server Protocol – Test 4 Test Objective The evaluator shall also perform the following tests:

Test 4: The evaluator shall perform the following modifications to the traffic: Test 4.1: Change the TLS version selected by the server in the Server Hello to a non-supported TLS version (for example 1.3 represented by the two bytes 03 04) and verify that the client rejects the connection. Test 4.2: Modify at least one byte in the client’s nonce in the Client Hello handshake message, and verify that the server rejects the client’s Certificate Verify handshake message (if using mutual authentication) or that the server denies the client’s Finished handshake message. Test 4.3: Modify the signature block in the Client’s Key Exchange handshake message, and verify that the server rejects the client’s Certificate Verify handshake message (if using mutual authentication) or that the server denies the client’s Finished handshake message. Test 4.4: Modify a byte in the Client Finished handshake message, and verify that the server rejects the connection and does not send any application data. Test 4.5*: After generating a fatal alert by sending a Finished message from the client before the client send a ChangeCipherSpec message, send a Client Hello with the session identifier from the previous test, and verify that the server denies the connection. Test 4.6: Send a garbled message from the client after the client has issued the ChangeCipherSpec message and verify that the Server denies the connection.

Test Instructions Execute this test per the test steps. Test Steps 1. Begin capturing packets between the TOE Server and the TLS client.

2. Run the test program created during the setup on the test system. 3. Initiate a connection from the TLS client to the TOE Server. 4. Stop capturing packets between the TOE Server and the TLS client. 5. Validate that the connection fails.

02/01/2017 CC TEST LAB #200423-0

Page - 21 -

6. Repeat steps 1-5 for each of the required modifications specified in App PP. 7. Repeat entire test for each interface required: TOE Server to Trusted TLS

client and TOE Server to Web Browser TLS client. *Note: The TOE uses session tickets not session IDs. This Test 4.5 cannot be performed. See TD 131.

Test Results Pass Execution Method Manual Test Case Number 020 SFR FCS_TLSS_EXT.1.2 – TLS Server Protocol Test Objective Test 1: The evaluator shall send a Client Hello requesting a connection with version

SSL 2.0 and verify that the server denies the connection. The evaluator shall repeat this test with SSL 3.0, TLS 1.0, TLS 1.1, and TLS 1.2 if it was selected.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TLS client to send SSL2.0 as the protocol version.

2. Initiate a connection from the TLS client to the TOE Server. 3. Validate that the connection fails.

4. Repeat steps 1-3 for each of the required protocol version: SSL2.0, SSL3.0,

TLS1.0, TLS 1.1

5. Validate that each connection fails. 6. Repeat step steps 1-3 for protocol version 1.2. 7. Validate that the connection is successful.

Test Results Pass Execution Method Manual server Test 21 FCS_TLSS_EXT.1.3 is not applicable as the TOE does to support ECDHE. Test Case Number 022 SFR FCS_TLSS_EXT.1.5 – TLS Server Protocol– Test 1 Test Objective Test 1: The evaluator shall configure the server to send a certificate request to the

client and shall attempt a connection without sending a certificate from the client. The evaluator shall verify that the connection is denied.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure mutual authentication between TOE Server and Trusted TLS client:

a. Configure the TOE Server for mutual authentication between TOE and Trusted TLS.

b. Configure Trusted TLS to not require mutual authentication (won’t send certificate).

2. Initiate a connection from the Trusted TLS client to the TOE Server. 3. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 023 SFR FCS_TLSS_EXT.1.5 – TLS Server Protocol– Test 2 Test Objective Test 2: The evaluator shall configure the server to send a certificate request to the

client without the supported_signature_algorithm used by the client’s certificate. The evaluator shall attempt a connection using the client certificate and verify that the connection is denied.

Test Instructions Execute this test per the test steps.

02/01/2017 CC TEST LAB #200423-0

Page - 22 -

Test Steps 1. Configure mutual authentication between TOE Server and Trusted TLS client: a. Configure TOE Server to use a correct certificate. b. Configure Trusted TLS client to send a certificate without the

supported signature algorithm. 2. Initiate a connection from the Trusted TLS client to the TOE Server. 3. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 024 SFR FCS_TLSS_EXT.1.5 – TLS Server Protocol– Test 3 Test Objective Test 3: The evaluator shall demonstrate that using a certificate without a valid

certification path results in the function failing. Using the administrative guidance, the evaluator shall then load a certificate or certificates needed to validate the certificate to be used in the function, and demonstrate that the function succeeds. The evaluator then shall delete one of the certificates, and show that the function fails.


a. Configure TOE Server to use the 4 certificate chain. b. Configure the Trusted TLS client to use the 4 certificate chain.

2. Initiate a connection from the Trusted TLS client to the TOE Server. 3. Validate that the connection succeeds. 4. Delete intermediate certificate so Trusted TLS client only sends 3 of the

certificates. 5. Initiate a connection from the Trusted TLS client to the TOE Server. 6. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 025 SFR FCS_TLSS_EXT.1.5 – TLS Server Protocol– Test 4 Test Objective Test 4: The evaluator shall configure the client to send a certificate that does not

chain to one of the Certificate Authorities (either a Root or Intermediate CA) in the server’s Certificate Request message. The evaluator shall verify that the attempted connection is denied.


a. Configure TOE Server to use the 4 certificate chain. b. Configure the Trusted TLS client to use the 4 certificate chain.

2. Swap a valid intermediate certificate with another intermediate certificate that is not part of the 4 cert chain.


Test Results Pass Execution Method Manual Test Case Number 026 SFR FCS_TLSS_EXT.1.5 – TLS Server Protocol – Test 5 Test Objective Test 5: The evaluator shall configure the client to send a certificate with the Client

Authentication purpose in the extendedKeyUsage field and verify that the server accepts the attempted connection. The evaluator shall repeat this test without the Client Authentication purpose and shall verify that the server denies the connection.

02/01/2017 CC TEST LAB #200423-0

Page - 23 -

Ideally, the two certificates should be identical except for the Client Authentication purpose.


a. Configure TOE Server to use the valid certificate. b. Configure the Trusted TLS client to use the certificate containing a valid

Server Authentication purpose in the extendedKeyUsage field on the TLS client.

2. Initiate a connection from the Trusted TLS client to the TOE Server. 3. Validate that the connection was successful.

4. Repeat steps 1-3 with a Trusted TLS client certificate that does not contain a

valid the Server Authentication purpose set in the extendedKeyUsage field 5. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 027 SFR FCS_TLSS_EXT.1.5 – TLS Server Protocol – Test 6 Test Objective Test 6: The evaluator shall perform the following modifications to the traffic:

a) Configure the server to require mutual authentication and then modify a byte in the client’s certificate. The evaluator shall verify that the server rejects the connection. b) Configure the server to require mutual authentication and then modify a byte in the client’s Certificate Verify handshake message. The evaluator shall verify that the server rejects the connection.

Test Instructions Execute this test per the test steps. Test Steps 1. Begin capturing packets between the TOE Server and the Trusted TLS client.

2. Run the test program created during the Setup on the test system to corrupt client’s certificate.

3. Initiate a connection from the Trusted TLS client and the TOE Server. 4. Stop capturing packets between the TOE Server and the Trusted TLS client. 5. Validate that the connection fails.

6. Repeat steps 1-5 and corrupt the client’s Certificate Verify handshake message.

Test Results Pass Execution Method Manual Test Case Number 028 SFR FCS_TLSS_EXT.1.6 – TLS Server Protocol– Test 1 Test Objective Test 1: The evaluator shall send a client certificate with an identifier that does not

match an expected identifier and verify that the server denies the connection. Test Instructions Execute this test per the test steps. Test Steps 1. Configure the TOE Server to check for a CN and SAN with a CN entry that

matches and a SAN entry that does not match the Trusted TLS client’s certificate.


Test Results Pass Execution Method Manual

02/01/2017 CC TEST LAB #200423-0

Page - 24 -

Test Case Number 029 SFR FCS_HTTPS_EXT.1.1 – HTTPS Protocol Test Objective The evaluator shall attempt to establish an HTTPS connection with a webserver,

observe the traffic with a packet analyzer, and verify that the connection succeeds and that the traffic is identified as TLS or HTTPS.

Test Instructions Execute this test per the test steps. Test Steps 1. Begin capturing traffic between the Web Browser Client to the TOE Server.

2. Initiate a connection from the Web Browser client to the TOE Server. 3. Stop capture. 4. Verify that the connection succeeds and data is encrypted.

Test Results Pass Execution Method Manual Test 30 is not included as the App PP states that FCS_TLSC_EXT.1.2 satisfies the testing. Test Case Number 031 SFR FCS_HTTPS_EXT.1.3 – HTTPS Protocol Test Objective Certificate validity shall be tested in accordance with testing performed for

FIA_X509_EXT.1, and the evaluator shall perform the following test: Test 1: The evaluator shall demonstrate that using a certificate without a valid certification path results in an application notification. Using the administrative guidance, the evaluator shall then load a certificate or certificates to the Trust Anchor Database needed to validate the certificate to be used in the function, and demonstrate that the function succeeds. The evaluator then shall delete one of the certificates, and show that the application is notified of the validation failure.

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TOE Server to use the 4 certificate chain.

2. Initiate a connection from the Web Browser client to the TOE Server. 3. Validate that the connection succeeds. 4. Delete intermediate certificate so TOE Server only sends 3 of the certificates. 5. Initiate a connection from the Web Browser client to the TOE Server. 6. Validate that the connection fails. Mutual authentication path (TOE Server received, TOE Client received) was tested FCS_TLSC_EXT.1.3 and FCS_TLSS_EXT.1.5


5.2.2 Identification and Authentication Test Case Number 032 SFR FIA_X509_EXT.1.1 – X.509 Certificate Validation – Test 1 Test Objective The tests described must be performed in conjunction with the other certificate

services assurance activities, including the functions in FIA_X509_EXT.2.1. The tests for the extendedKeyUsage rules are performed in conjunction with the uses that require those rules. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 1: The evaluator shall demonstrate that validating a certificate without a valid

02/01/2017 CC TEST LAB #200423-0

Page - 25 -

certification path results in the function failing. The evaluator shall then load a certificate or certificates as trusted CAs needed to validate the certificate to be used in the function, and demonstrate that the function succeeds. The evaluator shall then delete one of the certificates, and show that the function fails.

Test Instructions Execute this test per the test steps. Test Steps This is the same test that has been accomplished in FCS_HTTPS_EXT.1.3,

FCS_TLSC_EXT.1.3, and FCS_TLSS_EXT.1.5 Test Results Pass Execution Method Manual Test Case Number 033 SFR FIA_X509_EXT.1.1 – X.509 Certificate Validation – Test 2 Test Objective The tests described must be performed in conjunction with the other certificate

services assurance activities, including the functions in FIA_X509_EXT.2.1. The tests for the extendedKeyUsage rules are performed in conjunction with the uses that require those rules. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 2: The evaluator shall demonstrate that validating an expired certificate results in the function failing.

Test Instructions Execute this test per the test steps. Test Steps 1. Install an expired certificate on the TLS server.

2. Initiate a connection from the TOE Client to the TLS server. 3. Validate that the connection fails.

Test Results Pass Execution Method Manual Test Case Number 034 SFR FIA_X509_EXT.1.1 – X.509 Certificate Validation – Test 3 Test Objective The tests described must be performed in conjunction with the other certificate

services assurance activities, including the functions in FIA_X509_EXT.2.1. The tests for the extendedKeyUsage rules are performed in conjunction with the uses that require those rules. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 3: The evaluator shall test that the TOE can properly handle revoked certificates-–conditional on whether CRL, OCSP, or OCSP Stapling is selected; if multiple methods are selected, then the following tests shall be performed for each method: ◦ The evaluator shall test revocation of the node certificate. The evaluator shall also test revocation of an intermediate CA certificate (i.e. the intermediate CA certificate should be revoked by the root CA), if intermediate CA certificates are supported. The evaluator shall ensure that a valid certificate is used, and that the validation function succeeds. The evaluator then attempts the test with a certificate that has been revoked (for each method chosen in the selection) to ensure when the certificate is no longer valid that the validation function fails.

02/01/2017 CC TEST LAB #200423-0

Page - 26 -

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TOE Server to use the 4 certificate chain (with revoked certificate)

2. Configure the TLS client to use the 4 certificate chain. 3. Initiate a connection from the TLS client to the TOE Server. 4. Validate that the connection fails. 5. Swap the revoked intermediate certificate chain on the TLS server. 6. Initiate a connection from the TLS client to the TOE Server. 7. Validate that the connection fails.


services assurance activities, including the functions in FIA_X509_EXT.2.1. The tests for the extendedKeyUsage rules are performed in conjunction with the uses that require those rules. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 4: If OCSP is selected, the evaluator shall configure the OCSP server or use a man-in-the-middle tool to present a certificate that does not have the OCSP signing purpose and verify that validation of the OCSP response fails. If CRL is selected, the evaluator shall configure the CA to sign a CRL with a certificate that does not have the CRLsign key usage bit set, and verify that validation of the CRL fails.


2. Configure the TLS client to use the 4 certificate chain. 3. Load CRL signed by Intermediate02 on TOE Server 4. Swap the intermediate 2 certificate with NoCRLsign key usage bit set on the

TLS client. 5. Initiate a connection from the TLS client to the TOE Server. 6. Validate that the connection fails.


services assurance activities, including the functions in FIA_X509_EXT.2.1. The tests for the extendedKeyUsage rules are performed in conjunction with the uses that require those rules. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 5: The evaluator shall modify any byte in the first eight bytes of the certificate and demonstrate that the certificate fails to validate. (The certificate will fail to parse correctly.)

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TOE Server to use the 4 certificate chain where server certificate has

had one of the first 8 bytes modified.

02/01/2017 CC TEST LAB #200423-0

Page - 27 -

2. Start TOE. 3. View audit log and verify HTTPServer failure

4. Configure the TLS Server to use the 4 certificate chain all good. 5. Begin capturing packets between the TOE Client and the TLS server. 6. Run the test program for MiTM attack created during the setup on the test

system that will modify certificate in first 8 bytes. 7. Initiate a connection from the TOE Client to the TLS server. 8. Stop capturing packets between the TOE Client and the TLS server. 9. Validate that the connection fails.


services assurance activities, including the functions in FIA_X509_EXT.2.1. The tests for the extendedKeyUsage rules are performed in conjunction with the uses that require those rules. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 6: The evaluator shall modify any byte in the last eight byte of the certificate and demonstrate that the certificate fails to validate. (The signature on the certificate will not validate.)

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TOE Server to use the 4 certificate chain where server certificate

has had one of the last 8 bytes modified. 2. Start TOE. 3. View audit log and verify HTTPServer failure


system that will modify certificate in last 8 bytes. 7. Initiate a connection from the TOE Client to the TLS server. 8. Stop capturing packets between the TOE Client and the TLS server. 9. Validate that the connection fails.


services assurance activities, including the functions in FIA_X509_EXT.2.1. The tests for the extendedKeyUsage rules are performed in conjunction with the uses that require those rules. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 7: The evaluator shall modify any byte in the public key of the certificate and

02/01/2017 CC TEST LAB #200423-0

Page - 28 -

demonstrate that the certificate fails to validate. (The signature on the certificate will not validate.)

Test Instructions Execute this test per the test steps. Test Steps 1. Configure TOE Server to use the 4 certificate chain where server certificate

has had one of the bytes of public key modified. 2. Start TOE. 3. View audit log and verify HTTPServer failure


system that will modify public key. 7. Initiate a connection from the TOE Client to the TLS server. 8. Stop capturing packets between the TOE Client and the TLS server. 9. Validate that the connection fails.


services assurance activities, including the functions in FIA_X509_EXT.2.1. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 1: The evaluator shall construct a certificate path, such that the certificate of the CA issuing the TOE's certificate does not contain the basicConstraints extension. The validation of the certificate path fails.


2. Configure the TLS client to use the 4 certificate chain that has the basicConstraints extension missing.



services assurance activities, including the functions in FIA_X509_EXT.2.1. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 2: The evaluator shall construct a certificate path, such that the certificate of the CA issuing the TOE's certificate has the CA flag in the basicConstraints extension not set. The validation of the certificate path fails.


2. Configure the TLS client to use the 4 certificate chain that does not have the basicConstraints extension set.

02/01/2017 CC TEST LAB #200423-0

Page - 29 -



services assurance activities, including the functions in FIA_X509_EXT.2.1. If the application supports chains of length four or greater, the evaluator shall create a chain of at least four certificates: the node certificate to be tested, two Intermediate CAs, and the self-signed Root CA. If the application supports a maximum trust depth of two, then a chain with no Intermediate CA should instead be created. Test 3: The evaluator shall construct a certificate path, such that the certificate of the CA issuing the TOE’s certificate has the CA flag in the basicConstraints extension set to TRUE. The validation of the certificate path succeeds.


2. Configure the TLS client to use the 4 certificate chain that has the basicConstraints extension set.


Test Results Pass Execution Method Manual Test Case Number 042 SFR FIA_X509_EXT.2.2 – X.509 Certificate Validation – Test 1 Test Objective The evaluator shall perform the following test for each trusted channel:

Test 1: The evaluator shall demonstrate that using a valid certificate that requires certificate validation checking to be performed in at least some part by communicating with a non-TOE IT entity. The evaluator shall then manipulate the environment so that the TOE is unable to verify the validity of the certificate, and observe that the action selected in FIA_X509_EXT.2.2 is performed. If the selected action is administrator-configurable, then the evaluator shall follow the operational guidance to determine that all supported administrator-configurable options behave in their documented manner.

Test Instructions None Test Steps 1. Load the revoked certificate on the TLS client.

2. Initiate a connection from the TLS client to the TOE Server. 3. Verify certificate is revoked. 4. Change environment so that TOE is unable to verify with CRL.

5. Initiate a connection from the TLS client to the TOE Server. 6. Verify that the connection was successful.

7. Go to Test 43 and continue Test Results Pass Execution Method Manual Test Case Number 043 SFR FIA_X509_EXT.2.2 – X.509 Certificate Validation – Test 2 Test Objective The evaluator shall perform the following test for each trusted channel:

Test 2: The evaluator shall demonstrate that an invalid certificate that requires

02/01/2017 CC TEST LAB #200423-0

Page - 30 -

certificate validation checking to be performed in at least some part by communicating with a non-TOE IT entity cannot be accepted.

Test Instructions None Test Steps 1. Re-establish CRL

2. Configure Indexer to check for CN and SAN = invalidname 3. Initiate a connection from the TLS client to the TOE Server. 4. Verify connection fails.


5.2.3 User Data Protection Test Case Number 044 SFR FDP_DEC_EXT.1.1 – Access to Platform Resources Test Objective The evaluator shall perform the platform-specific actions below and inspect user

documentation to determine the application's access to hardware resources. The evaluator shall ensure that this is consistent with the selections indicated. The evaluator shall review documentation provided by the application developer and for each resource which it accesses, identify the justification as to why access is required. The evaluator shall verify that either the application software or its documentation provides a list of the hardware resources it accesses.

Test Instructions None Test Steps Based on the selection of Linux as the OS there are no testing activities. All

activities for Linux is operational guidance and is reported in the Assurance Activites for Operation Guidance section.

Test Results Pass Execution Method Manual Test Case Number 045 SFR FDP_DEC_EXT.1.2 – Access to Platform Resources Test Objective The evaluator shall perform the platform-specific actions below and inspect user

documentation to determine the application's access to sensitive information repositories. The evaluator shall ensure that this is consistent with the selections indicated. The evaluator shall review documentation provided by the application developer and for each sensitive information repository which it accesses, identify the justification as to why access is required. The evaluator shall verify that either the application software or its documentation provides a list of sensitive information repositories it accesses.

Test Instructions None Test Steps 1. This test is N/A because the ST selects “no sensitive information repositories”

for this SFR. Test Results Pass Execution Method Manual Test Case Number 046 SFR FDP_NET_EXT.1.1 – Network Communications Test Objective The evaluator shall perform the following tests:

Test 1: The evaluator shall run the application. While the application is running, the

02/01/2017 CC TEST LAB #200423-0

Page - 31 -

evaluator shall sniff network traffic ignoring all non-application associated traffic and verify that any network communications witnessed are documented in the TSS or are user-initiated. Test 2: The evaluator shall run the application. After the application initializes, the evaluator shall run network port scans to verify that any ports opened by the application have been captured in the ST for the third selection and its assignment. This includes connection-based protocols (e.g. TCP, DCCP) as well as connectionless protocols (e.g. UDP).

Test Instructions None Test Steps 1. Start traffic capture on the network where the TOE, SMTP Server, Web

Browser are attached. 2. Initiate a connection from the Web Browser client to the TOE Server. 3. Initiate a connection from the TLS client to the TOE Server. 4. Initiate a connection from the TOE Client to the SMTP Server. 5. Stop capture. Filter .pcap to only show traffic to and from the TOE. 6. Verify all traffic discovered coming to and from the TOE is documented in

TSS. 7. From a remote machine run a portscan against the TOE platform searching for

all TCP ports and UDP. 8. Verify that any ports opened by the application have been captured in the ST

selection and assignment. Test Results Pass Execution Method Manual Test Case Number 047 SFR FDP_DAR_EXT.1.1 - Encryption Of Sensitive Application Data Test Objective The evaluator shall inventory the filesystem locations where the application may

write data. The evaluator shall run the application and attempt to store sensitive data. The evaluator shall then inspect those areas of the filesystem to note where data was stored (if any), and determine whether it has been encrypted.

Test Instructions None Test Steps 1. Verify LUKS was applied to host machine by issuing df –l command

2. Verify /etc/opt and /etc directory structures are encypted with LUKS by looking for LUKS “secfsda” nomenclature associated with partitions.

3. Verify LUKS partition is active and in user by issuing the cryptsetup –v status <partition id>


5.2.4 Security Management Test Case Number 048 SFR FMT_MEC_EXT.1 – Supported Configuration Mechanism Test Objective The evaluator shall run the application while monitoring it with the utility strace.

The evaluator shall make security-related changes to its configuration. The evaluator shall verify that strace logs corresponding changes to configuration files that reside in /etc (for system-specific configuration) or in the user's home directory (for user-specific configuration).

Test Instructions None Test Steps 1. Run strace utility on splunkd.

2. Make changes security related fields (server name, mail server ip, add user).

02/01/2017 CC TEST LAB #200423-0

Page - 32 -

3. Stop strace. 4. Inspect strace logs and verify that the TOE writes files accordingly.

Test Results Pass Execution Method Manual Test Case Number 049 SFR FMT_CFG_EXT.1.1 – Secure by Default Configuration – Test 1 Test Objective If the application uses any default credentials the evaluator shall run the following

tests. Test 1: The evaluator shall install and run the application without generating or loading new credentials and verify that only the minimal application functionality required to set new credentials is available.

Test Instructions None Test Steps 1. Install TOE according to installation instructions.

2. Launch TOE. 3. Connect to TOE with web browser and authenticate with default credentials. 4. A change password web page should be served with no bypass mechanism.

Test Results Pass Execution Method Manual Test Case Number 050 and 051 SFR FMT_CFG_EXT.1.1 – Secure by Default Configuration – Test 2 – Test 3 Test Objective If the application uses any default credentials the evaluator shall run the following

tests. Test 2: The evaluator shall attempt to clear all credentials and verify that only the minimal application functionality required to set new credentials is available. Test 3: The evaluator shall run the application, establish new credentials and verify that the original default credentials no longer provide access to the application.

Test Instructions None Test Steps 1. A change password web page should be served with no bypass. Attempt to

clear all credentials – TOE won’t allow the removal of all users. 2. Create a new user with admin role. 3. Connect to TOE Web UI. 4. Login with new credentials and remove admin role. 5. Logout and attempt to use default credential to login – This should fail. 6. Login with new credentials and recreate the admin user. 7. Logout and attempt to login with default credentials – This should fail. 8. Logout of Web UI. 9. Log in to the OS. 10. Delete TOE’s password file (forces TOE to reinitialize user accounts). 11. Logout of SSH connection. 12. Connect to TOE with web browser and authenticate using default credentials. 13. A change password web page should be served with no bypass.

Test Results Pass Execution Method Manual Test Case Number 052 SFR FMT_CFG_EXT.1.2 – Secure by Default Configuration Test Objective The evaluator shall install and run the application. The evaluator shall inspect the

filesystem of the platform (to the extent possible) for any files created by the application and ensure that their permissions are adequate to protect them. The

02/01/2017 CC TEST LAB #200423-0

Page - 33 -

method of doing so varies per platform. The evaluator shall run the command find . -perm /007 inside the application's data directories to ensure that all files are not world-accessible (either read, write, or execute). The command should not print any files.

Test Instructions None Test Steps 1. Login to the OS.

2. As ‘root’ user cd to the application’s data directories. 3. Run the following command: find . -perm /007 4. Repeat if more than one directory structure is created by the installation of the

TOE. 5. Verify that there are no results reported.

Test Results Pass Execution Method Manual Test Case Number 053 SFR FMT_SMF.1.1 – Specification of Management Functions Test Objective The evaluator shall test the application's ability to provide the management

functions by configuring the application and testing each option selected from above. The evaluator is expected to test these functions in all the ways in which the ST and guidance documentation state the configuration can be managed.

Test Instructions None Test Steps 1. Disable/Enable supported TLS cipher suites satisfied with

FCS_TLSC_EXT.1.1 and FCS_TLSS_EXT.1.1 tests. Configured as OS user. 2. Disable/Enable TLS mutual authentication satisfied with FCS_TLSC_EXT &

FCS_TLSS_EXT involving certificates. Configured as OS user. 3. Query of version of the TOE as OS level splunk user: splunk version 4. Connect to TOE Web UI. 5. Login and query the version of the TOE from WebUI user: HelpAbout 6. Enable/disable the transmission of any information describing the system’s

hardware, software, or configuration which is specifically described in the TSS section as done by configuring email alerts. Configuring the email alerts is part of FCS_TLSC_EXT.1.1 – TLS Client Protocol – Test 1b (Indexer Client to SMTP Server) and FMT_MEC_EXT.1 – Supported Configuration Mechanism.


5.2.5 Privacy Test 54 FPR_ANO_EXT.1.1 – User Consent for Transmission of Personally Identifiable Information (PII) is not applicable as the TOE does not store or transmit PII.

5.2.6 Protection of the TSF Test Case Number 055 SFR FPT_AEX_EXT.1.1 – Anti-Exploitation Capabilities Test Objective The evaluator shall perform either a static or dynamic analysis to determine that no

memory mappings are placed at an explicit and consistent address. The method of doing so varies per platform. The evaluator shall run the same application on two different Linux systems. The evaluator shall then compare their memory maps using pmap -x PID to ensure the

02/01/2017 CC TEST LAB #200423-0

Page - 34 -

two different instances share no mapping locations. Test Instructions Execute this test per the test steps. Test Steps 1. Install the TOE on 2 different machines.

2. Launch both TOE instantiations. 3. Execute the following command on both platforms as ‘root’ user:

pmap -x [PID] 4. Examine the memory maps from both systems and ensure that there is no

sharing of the same memory locations with the exception of any program listed in the exception assignment.

Test Results Pass Execution Method Manual Test Case Number 056 SFR FPT_AEX_EXT.1.2 – Anti-Exploitation Capabilities Test Objective The evaluator shall verify that no memory mapping requests are made with write

and execute permissions. The method of doing so varies per platform. The evaluator shall perform static analysis on the application to verify that both • mmap is never be invoked with both the PROT_WRITE and PROT_EXEC permissions, and • mprotect is never invoked with the PROT_EXEC permission.

Test Instructions Execute this test per the test steps. Test Steps 1. Have vendor provide access to source code and provide the following:

a. A list from a search for “mmap” in all of the files. b. A list for a search for “mprotect” in all of the files.

2. Search for PROT_EXEC within the “mmap” search results. 3. Verify all results can be traced back to the source library that is part of the

exceptions assignment of FPT_AEX_EXT.1.2. 4. Search for PROT_EXEC in the “mprotect” search results. 5. Verify all results can be traced back to the source library that is part of the

exceptions assignment of FPT_AEX_EXT.1.2. Test Results Pass Execution Method Manual Test Case Number 057 SFR FPT_AEX_EXT.1.3 – Anti-Exploitation Capabilities Test Objective The evaluator shall configure the platform in the ascribed manner and carry out one

of the prescribed tests: The evaluator shall ensure that the application can successfully run on a system with SELinux enabled and enforcing.

Test Instructions Execute this test per the test steps. Test Steps 1. As ‘root’ user issue the command: Ensure that SELinux is enabled and

enforcing by issuing the sestatus command. 2. Verify the results show that SELinux is enabled and its current mode is

enforcing. Test Results Pass Execution Method Manual Test Case Number 058 SFR FPT_AEX_EXT.1.4 – Anti-Exploitation Capabilities Test Objective The evaluator shall run the application and determine where it writes its files. For

02/01/2017 CC TEST LAB #200423-0

Page - 35 -

files where the user does not choose the destination, the evaluator shall check whether the destination directory contains executable files. This varies per platform: The evaluator shall run the program, mimicking normal usage, and note where all files are written. The evaluator shall ensure that there are no executable files stored in the same directories to which the application wrote.

Test Instructions Execute this test per the test steps. Test Steps 1. As ‘root’ create a filesystem listing before installation of TOE.

2. After TOE is installed and configured create another filesystem listing after installation to compare to the first list.

3. Determine TOE directory structure. 4. Login to the TOE via the Web UI and make modifications under the

administrator menu. 5. As ‘root’ create a filesystem listing after running the product to compare to the

second list. 6. Determine if any files were written to directories were executables are stored.

Test Results Pass Execution Method Manual Test Case Number 059 SFR FPT_AEX_EXT.1.5 – Anti-Exploitation Capabilities Test Objective The evaluator shall perform a static analysis to verify that stack-based buffer

overflow protection is present. The method of doing so varies per platform: If the application is compiled using GCC, the evaluator shall ensure that the -fstack-protector-strong or -fstackprotector-all flags are used. The -fstack-protector-all flag is preferred but -fstack-protector-strong is acceptable.

Test Instructions Execute this test per the test steps. Test Steps 1. Have vendor show compiler script.

2. Verify that the TOE is compiled using either the -fstack-protector-strong or -fstack-protector-all flag.

Test Results Pass Execution Method Manual Test Case Number 060 SFR FPT_TUD_EXT.1.1 – Integrity for Installation and Update Test Objective The evaluator shall check for an update using procedures described in the

documentation and verify that the application does not issue an error. If it is updated or if it reports that no update is available this requirement is considered to be met.

Test Instructions Execute this test per the test steps. Test Steps 1. Establish a connection to TOE WebUI.

2. Observe the login screen to see if a notice and the download button is displayed.

3. Verify that the “download” button launches the browser to the intended download page and does not issue an error.

Test Results Pass Execution Method Manual Test Case Number 061 SFR FPT_TUD_EXT.1.2 – Integrity for Installation and Update

02/01/2017 CC TEST LAB #200423-0

Page - 36 -

Test Objective The evaluator shall verify that application updates are distributed in the format supported by the platform. This varies per platform: The evaluator shall ensure that the application is packaged in the format of the package management infrastructure of the chosen distribution. For example, applications running on Red Hat and Red Hat derivatives should be packaged in RPM format. Applications running on Debian and Debian derivatives should be packaged in deb format.

Test Instructions Execute this test per the test steps. Test Steps 1. Establish a connection to TOE WebUI.

2. Observe the login screen to see if a notice and the download button is displayed.

3. Press the “download” button to launch the browser to the intended download page and does not issue an error.

4. Observe download format and ensure that there is one for Red Hat Linux (.rpm)

Test Results Pass Execution Method Manual Test Case Number 062 SFR FPT_TUD_EXT.1.3 – Integrity for Installation and Update Test Objective The evaluator shall record the path of every file on the entire filesystem prior to

installation of the application, and then install and run the application. Afterwards, the evaluator shall then uninstall the application, and compare the resulting filesystem to the initial record to verify that no files, other than configuration, output, and audit/log files, have been added to the filesystem.

Test Instructions Execute this test per the test steps. Test Steps 1. Record every path of every file on the entire filesystem.

2. Install and configure the TOE application following guidance. 3. Launch the TOE application and perform some activities within the TOE

application. 4. Uninstall the TOE application following guidance. 5. Record every path of every file on the entire filesystem in a second list. 6. Create a difference file to show what is left after the install compared to the

original list. 7. Create a filtered list removing all of the splunk directory items to determine

what was modified or added to OS during install. 8. Verify that no files, other than configuration, output, and audit/log files, are left

on the filesystem. Test Results Pass Execution Method Manual Test Case Number 063 SFR FPT_TUD_EXT.1.4 – Integrity for Installation and Update Test Objective The evaluator shall verify that the application's executable files are not changed by

the application. The evaluator shall complete the following test: Test 1: The evaluator shall install the application and then locate all of its executable files. The evaluator shall then, for each file, save off either a hash of the file or a copy of the file itself. The evaluator shall then run the application and exercise all features of the application as described in the TSS. The evaluator shall then compare each executable file with the either the saved hash or the saved copy of the files. The evaluator shall verify that these are identical.

Test Instructions Execute this test per the test steps. Test Steps 1. Install the TOE application to the system.

02/01/2017 CC TEST LAB #200423-0

Page - 37 -

2. Calculate and record the SHA-1 hash of all TOE executable files. 3. Continue testing for the day. 4. Calculate and record the SHA-1 hash of all TOE executable files. 5. Compare the two SHA listings. 6. Verify that there are no differences.

Test Results Pass Execution Method Manual Test Case Number 064 SFR FPT_TUD_EXT.1.5 – Integrity for Installation and Update Test Objective The purpose of this test is to verify the versioning of the TOE and documentation. Test Instructions Execute this test per the test steps. Test Steps 1. Query of version of the TOE as OS user: rpm –qi splunk

2. Query of version of the TOE as splunk user: splunk version 3. Connect to TOE Web UI. 4. Login and query the version of the TOE from WebUI user: HelpAbout 5. Verify the version displayed in all three cases matches the TOE identifier.

Test Results Pass Execution Method Manual Test Case Number 065 SFR FPT_LIB_EXT.1 – Use of Third Party Libraries Test Objective The evaluator shall install the application and survey its installation directory for

dynamic libraries. The evaluator shall verify that libraries found to be packaged with or employed by the application are limited to those in the assignment.

Test Instructions Execute this test per the test steps. Test Steps 1. Install the TOE application to the system.

2. Verify that the third party dynamic libraries are limited to those in the SFR assignment.

3. Search for the libraries defined in the ST in the TOE’s installation directory structure. These libraries could be single files or packages that contain multiple files such as Python.

4. Verify that the libraries defined in the ST are contained in the TOE file structure.

5. Verify that all libraries in TOE file structure are contained in the libraries defined in the ST.

Test Results Pass Execution Method Manual Test Case Number 066 SFR FTP_DIT_EXT.1 – Protection of Data in Transit Test Objective The evaluator shall perform the following tests.

Test 1: The evaluator shall exercise the application (attempting to transmit data; for example by connecting to remote systems or websites) while capturing packets from the application. The evaluator shall verify from the packet capture that the traffic is encrypted with HTTPS, TLS or DTLS in accordance with the selection in the ST. Test 2: The evaluator shall exercise the application (attempting to transmit data; for example by connecting to remote systems or websites) while capturing packets from the application. The evaluator shall review the packet capture and verify that no sensitive data is transmitted in the clear. Test 3: The evaluator shall inspect the TSS to determine if user credentials are

02/01/2017 CC TEST LAB #200423-0

Page - 38 -

transmitted. If credentials are transmitted the evaluator shall set the credential to a known value. The evaluator shall capture packets from the application while causing credentials to be transmitted as described in the TSS. The evaluator shall perform a string search of the captured network packets and verify that the plaintext credential previously set by the evaluator is not found.

Test Instructions Execute this test per the test steps. Test Steps

Encryption of traffic to and from the TOE is tested as part of FCS_HTTPS_EXT.1, FCS_TLSC_EXT.1, and FCS_TLSS_EXT.1. Test 2 and Test 3 are combined into a single test because the only sensitive data transmitted to/from the TOE are administrative credentials during login to Splunk Web.

1. Begin capturing packets using Wireshark 2. Login to TOE as administrator. 3. Navigate to the Splunk administration page. 4. Change the password to “catchme123” for current user 5. Create a new user the credentials of :“testuser” and “changeme123” 6. Edit another existing user changing the password for that user:

“changeme123” 7. Stop capturing packets using Wireshark 8. Inspect the packet capture for known plaintext credential values:

chamgeme123, testuser Test Results Pass Execution Method Manual

5.3 Vulnerability Testing The evaluation team created a set of vulnerability tests to attempt to subvert the security of the TOE. These tests were created based upon the evaluation team's review of the vulnerability analysis evidence and independent research. The evaluation team conducted searches for public vulnerabilities related to the TOE. A few notable resources consulted include securityfocus.com, the cve.mitre.org, and the nvd.nist.gov. Upon the completion of the vulnerability analysis research, the team had identified several generic vulnerabilities upon which to build a test suite. These tests were created specifically with the intent of exploiting these vulnerabilities within the TOE or its configuration. The team tested the following areas:

• Port Scan: Port scanning was redone to ensure nothing has changed in the configuration while testing for the PP assurance measures.

• The Web Interface vulnerability scan: This scan was the primary emphasis on the penetration testing for both automated scanning and manual attempts and was meant to look for OWASP Top 10 vulnerabilities. Attempted to scan all open ports found by the port scan.

• Weak SSH support: Attempt to force SSHv1 on the host OS platform. • Virus scan: A virus scan was accomplished on software as required by the Assurance Activities

for the Application Software Protection Profile. The TOE, Splunk Enterprise v6.4.5, successfully prevented the attempts of subverting its security. Verdict: The evaluation team has completed testing of this component, resulting in a verdict of PASS.

02/01/2017 CC TEST LAB #200423-0

Page - 39 -

6 Conclusions The TOE was evaluated against the ST and has been found by this evaluation team to be conformant with the ST. The overall verdict for this evaluation is: Pass.

7 Glossary of Terms

Acronym Definition AES Advanced Encryption Standard CAVP Cryptographic Algorithm Validation Program CBC Cipher Block Chaining CLI Command Line Interface CSP Critical Security Parameter DHE Diffie-Hellman DRBG Deterministic Random Bit Generator HMAC Hashed Message Authentication Code MCLI Management Command Line Interface MPLS Multiprotocol Label Switching APP PP Network Device Protection Profile NTP Network Time Protocol OSI Open Systems Interconnection OTN Optical Transport Network RSA Rivest Shamir Adelman (encryption algorithm) SDH Synchronous Digital Hierarchy SFTP Secure File Transfer Protocol SHA Secure Hash Algorithm SHS Secure Hash Standard SONET Synchronous Optical Networking SSH Secure Shell TL1 Transaction Language One

Table 6-1: Acronyms

Terminology Definition

Authorized Administrator Any user which has been assigned to a privilege level that is permitted to perform all TSF-related functions.

Role An assigned role gives a user varying access to the management of the TOE.

Security Administrator Synonymous with Authorized Administrator for the purposes of this evaluation.

User Any entity (human user or external IT entity) outside the TOE that interacts with the TOE.

Table 6-2: Terminology

Documents

Splunk 6.4 - NIAP CCEVS · 5.1 Assessment of the Splunk Test Environment ... cryptographic functions ... establishing all reference identifiers from the application-configured reference