Splunk Enterprise on Dell EMC VxFlex integrated …...10 Splunk Enterprise on Dell EMC VxFlex integrated rack with Isilon | 000054 2.6 Splunk Enterprise Splunk Enterprise is a software

000054

White Paper

Splunk Enterprise on Dell EMC VxFlex integrated rack with Isilon

Abstract This white paper highlights the design considerations of Splunk Enterprise

deployment on VxFlex integrated rack with Isilon

November 2019

Revisions

2 Splunk Enterprise on Dell EMC VxFlex integrated rack with Isilon | 000054

Revisions

Date Description

November 2019 Initial release

Acknowledgements

This paper was produced by the following:

Author: Swathi Nagaram

Support: Nataraj Naikar

Other: Shalini G

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind with respect to the information in this

publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, Dell EMC and other trademarks are trademarks of Dell Inc. or its

subsidiaries. Other trademarks may be trademarks of their respective owners. [11/11/2019] [White Paper] [000054]

Table of contents


Table of contents

Revisions............................................................................................................................................................................. 2

Acknowledgements ............................................................................................................................................................. 2

Table of contents ................................................................................................................................................................ 3

Executive summary ............................................................................................................................................................. 4

1 Introduction ................................................................................................................................................................... 5

1.1 Objective ............................................................................................................................................................. 5

1.2 Audience ............................................................................................................................................................. 5

1.3 Terminology ........................................................................................................................................................ 5

2 Product overview .......................................................................................................................................................... 7

2.1 Dell EMC VxFlex integrated rack ........................................................................................................................ 7

2.2 VxFlex OS overview ........................................................................................................................................... 8

2.3 VxFlex Manager.................................................................................................................................................. 9

2.4 VMware vSphere ................................................................................................................................................ 9

2.5 Isilon storage ...................................................................................................................................................... 9

2.6 Splunk Enterprise ............................................................................................................................................. 10

2.6.1 Data ageing in Splunk .............................................................................................................................. 10

3 Splunk distributed clustered deployment ................................................................................................................... 11

3.1 Virtualization design ......................................................................................................................................... 13

3.2 Compute design................................................................................................................................................ 13

3.3 Storage design.................................................................................................................................................. 13

3.4 Isilon Storage design ........................................................................................................................................ 14

3.5 Network architecture ......................................................................................................................................... 14

3.6 Splunk Enterprise clustered deployment design .............................................................................................. 16

4 Validation .................................................................................................................................................................... 17

4.1 Validation procedure ......................................................................................................................................... 17

5 Conclusion .................................................................................................................................................................. 19

A Appendix ..................................................................................................................................................................... 20

A.1 Hardware and Software components ............................................................................................................... 20

A.2 Best Practices ................................................................................................................................................... 20

A.3 Configure Isilon NFS for the VxFlex integrated rack ........................................................................................ 21

B Technical support and resources ............................................................................................................................... 23

B.1 Related resources............................................................................................................................................. 23

B.2 Additional resources ......................................................................................................................................... 23

Executive summary


Executive summary

Splunk Enterprise software enables collection, indexing and visualization of machine-generated data gathered

from different sources in the IT infrastructure. These sources can include applications, networking devices,

host and application logs, mobile devices and more. Splunk turns silos of data into operational insights and

provides visibility across the IT infrastructure to enable faster problem solving and informed, data-driven

decisions.

Together, Dell EMC and Splunk enable you to harness the power of machine data analytics with simplified

deployment and scalability by lowering the cost of IT operations and delivering end-to-end operational

intelligence.

This white paper covers the Splunk Enterprise distributed clustered deployment for 50 GB ingestion/day with

30-day hot/warm retention on VxFlex integrated rack with four nodes using Isilon for Splunk cold buckets to

help customers gain high data availability, simplified scalability, and large capacity data retention needs.

Usage of Isilon storage for cold bucket storage needs is optional. A general recommendation is to add Isilon

storage when the cold bucket data is larger than 60 TB.

This approach can be extended to various volume ingestion requirements (based on the SVA guidelines) by

scaling the required number nodes on the VxFlex integrated rack and leveraging the Dell EMC Isilon scale-

out NAS storage platform for cold bucket storage needs.

Introduction


1 Introduction Machine data is the largest and fastest-growing section of data. Every second of every day, hundreds to

thousands of devices record what is happening in your business, with records coming in an array of

unpredictable formats. Many organizations find that once they use Splunk for one use case, they want to add

more. In addition, data sets keep growing exponentially, with no end in sight.

Dell EMC VxFlex integrated rack for Splunk addresses your current and future needs by offering flexible

solutions that allow you to scale compute and storage independently, or hyperconverged system.

1.1 Objective This white paper demonstrates the Splunk Enterprise distributed clustered deployment on VxFlex integrated

rack and delivers the solution for high performance and large capacity data retention needs using Isilon.

1.2 Audience This document is intended for decision makers, business leaders, architects, Splunk administrators,hyper

converged infrastructure administrators, and technical administrator of IT environments responsible for

deployment of Splunk on Dell EMC VxFlex integrated rack with ESXi hypervisor.

The reader of this document must have a working knowledge of Dell EMC VxFlex integrated rack, VMware

vSphere technologies, Isilon, Splunk Enterprise, and should have a basic familiarity with storage, compute,

and network technologies and topologies.

1.3 Terminology The following table defines acronyms and terms that are used throughout this document:

Terms and definitions

Term Definition

MDM Meta Data Manager

SDS Storage Data Server

SDC Storage Data Client

SVM Storage Virtual Machine

OS Operating System

RCM Release Certification Matrix

SSD Solid-state drive

IaaS Infrastructure as a Service

PaaS Platform as a Service

NAS Network-attached storage

Introduction


Term Definition

C1/C11 Distributed Clustered Deployment-Single Site (Topology category code as per SVA)

SH Search Head

CM Cluster Master

LM License Master

SVA Splunk Validated Architectures

Product overview


2 Product overview

2.1 Dell EMC VxFlex integrated rack The Dell EMC VxFlex integrated rack is an engineered system that provides the ultimate performance,

reliability, scalability, agility, and flexibility for modern data center workloads, IaaS, and PaaS cloud

infrastructure initiatives. The system is powered by Dell EMC VxFlex OS software-defined storage and based

on industry-leading enterprise-class Dell EMC PowerEdge servers. It is a rack scale hyperconverged system

that comes with a proprietary intelligent physical infrastructure (IPI) cabinet, offers integrated networking and

dedicated system management control plane.

VxFlex integrated rack benefits

Product overview


The modular design of VxFlex integrated rack enables you to add standardized units of infrastructure to the

environment. With this scalable model, it is all about expanding the infrastructure in small increments, as

required by applications, eliminating the over-provisioning that is experienced with other approaches. The

following figure shows the overall VxFlex OS Architecture:

VxFlex integrated rack scalability

Each cabinet is equipped with redundant access switches (Cisco 93180YC-EX). A pair of aggregation

switches is installed in the first cabinet and configured in access/aggregation network topology. If more than

one cabinet exists, the aggregation switches can be spread across or installed in other cabinets.

The entire system is built and configured at the Dell EMC factory according to the proven and tested best

practices. In addition to the unmatched performance, scalability and performance, customers also enjoy one-

call support for all components and end-to-end life-cycle management through a proven automated Release

Certification Matrix (RCM) for all components including software and firmware.

2.2 VxFlex OS overview VxFlex OS is a software defined block storage that uses the server’s local disks and network to create an IP-

based virtual SAN that has all the benefits of external storage without the cost and complexity of fiber-

channel. The software is purpose-built to deliver the ultimate performance, data reliability, and scalability

expected of enterprise storage. The multiple-deployment options, on-demand scale capability, multi

hypervisor support, and resilience make it suitable for virtually all type of workloads.

Use the following options for deploying VxFlex OS:

• Two-layer

• Hyperconverged

• Hybrid (One part of the system is deployed in two-layer and other part is in hyperconverged)

This paper discusses about hyperconverged option only.

Product overview


2.3 VxFlex Manager VxFlex Manager is the VxFlex integrated rack management and Orchestration (M&O) tool that provides a

single pane of glass for provisioning, managing, monitoring, alerting, life-cycle management, and reporting. It

increases efficiency by reducing time-consuming manual operations that are required to implement, provision,

and manage operations for your VxFlex integrated rack. Through automation, you can deploy and manage

operations for your VxFlex integrated rack.

VxFlex Manager brings together multiple management consoles, workflow automation, and an intuitive

interface that allows customers to monitor, manage, deploy, and maintain physical and virtual resources with

the click of a button. Key tenets of the VxFlex Manager architecture include:

• System assurance: Compliancy and non-disruptive remediation

• Insights: Monitoring, alerting, and health checks

• Implementation simplification: Simplified and automated system deployment and workflows

• Node serviceability: Single-button take node out of and back into service

• Hypervisor update: Single-button upgrade of hypervisor

2.4 VMware vSphere The vSphere virtualization layer decouples the application from the underlying physical resources. This

decoupling enables greater flexibility in the application layer by eliminating hardware downtime for

maintenance and changes to the physical system without affecting the hosted applications. In a server

virtualization use case, this layer enables multiple independent virtual machines (VM) to share the same

physical hardware.

vSphere is a complete and robust virtualization platform, virtualizing business-critical applications with

dynamic resource pools for flexibility and reliability. It transforms physical resources of a computer by

virtualizing the CPU, RAM, hard disk, and network controller. This transformation creates fully functional VMs

that run isolated and encapsulated operating systems and applications.

2.5 Isilon storage Isilon hybrid storage is highly flexible and strikes the balance between large capacity and high-performance

storage to provide support for a broad range of enterprise file workloads.

It uses intelligent software to scale data across large number of hardware units, enabling explosive growth in

performance and capacity. The OneFS™ operating system, the revolutionary storage architecture,offers a

single clustered file system.

OneFS provides value by incorporating parallelism at a deep level in the operating system. Virtually, the

system is distributed across multiple hardware units. This parallelism allows OneFS to scale in every

dimension as the infrastructure is expanded. By providing multiple redundancy levels, the system has no

single point of failure. As a result, it can grow to a multi-petabyte scale while providing greater reliability than

traditional systems.

OneFS runs on Isilon scale-NAS hardware, ensuring that Isilon benefits from the ever-improving cost and

efficiency curves of hardware. It allows you to add hardware to or remove hardware from the cluster at any

time. The data is protected from hardware changes. This feature alleviates the cost and burden of data

migrations and hardware refreshes.

Product overview


2.6 Splunk Enterprise Splunk Enterprise is a software platform that enables you to collect, index, and visualize machine-generated

data gathered from different sources in your IT infrastructure. These sources can include applications,

networking devices, host and application logs, mobile devices, and more.

It gives you real-time insight and understanding into what is happening and provides end to-end visibility

across your IT infrastructure to enable informed, data-driven decisions.

For more information about Splunk enterprise, see Splunk Enterprise Overview.

2.6.1 Data ageing in Splunk Upon receiving the data from forwarder, the indexer parses the raw data into distinct events based on the

timestamp of the event and writes them to the appropriate index. Splunk implements storage tiering of hot,

warm and cold buckets to optimize performance for newly indexed data and provide an option to keep older

data for longer periods on higher capacity storage.

In this solution, hot and warm buckets reside on the SSD storage pool of VxFlex integrated rack and cold

buckets are configured on Isilon storage.

For more information about data ageing, see Managing Indexers and Clusters of Indexers.

Splunk storage tiering

https://docs.splunk.com/Documentation/Splunk/7.3.1/Overview/AboutSplunkEnterprise

https://docs.splunk.com/Documentation/Splunk/7.3.1/Indexer/HowSplunkstoresindexes

Splunk distributed clustered deployment


3 Splunk distributed clustered deployment

The Splunk solution is deployed on a hyperconverged deployment with VxFlex integrated rack. As per Splunk

validated architectures (SVA) document, for 50 GB/day ingestion volume, single instance deployment is

recommended. In this solution, to demonstrate the Splunk enterprise distributed clustered deployment, a

multi-instance setup with replication and search factor of 2 is used. For more details about SVA, see Splunk

Validated Architectures.

The following table provides the configuration details used in Splunk deployment:

Configuration details

Sizing 50 GB/day clustered

Retention(hot/warm) 30 days

Number of VxFlex Nodes 4

Compute Power Edge R640 Servers

CPU 2 x Intel Xeon (24C,2.70 GHz)

Memory 384 GB

Storage 10 x 3.84 TB SSDs

Network 25 GbE Cisco Nexus

Hot/warm Storage 1 TB

Cold Storage Configurable

Isilon series A200

For detailed configuration information about Splunk Clustered deployment on VxFlex integrated rack, see

Hardware and Software components.

The following figure provides an overview of the logical architecture of Splunk enterprise distributed clustered

deployment on VxFlex integrated rack for 50 GB/day ingestion volume with 30-day hot/warm and configurable

cold data retention using Isilon:

https://www.splunk.com/pdfs/technical-briefs/splunk-validated-architectures.pdf




Logical Architecture of Splunk Enterprise distributed clustered deployment-single site

The VxFlex integrated rack HCI is configured with ESXi hypervisor for compute, network, and VxFlex OS for

software-defined storage. Hyperconverged infrastructure configuration has SDC and SDS role that is

configured on each node of the cluster. SDC provides the compute capabilities while SDS provides backend

storage.

The Storage Data Server (SDS) aggregates the raw local storage in a node and serves it up as VxFlex OS

storage. A single protection domain is carved out of SSD drives on these four SDS nodes. A single storage

pool is configured, and multiple volumes were carved out to meet the Splunk requirements. These volumes

are mapped to the ESXi cluster and added as a datastore and later mapped as disks drive to Splunk virtual

machine using VMware Paravirtual SCSI (PVSCSI) adapters.

Each VxFlex integrated rack node has a Dell EMC Storage Virtual Machine (SVM) running on it, providing

both storage clustering and storage services.

In this solution, 50 GB log data has forwarded using the universal forwarder to indexer cluster, where the log

data gets indexed and resides in hot bucket.



The search head helps you to search indexed data available in the hot/warm bucket. Once the retention

exceeds the 30-day period, the data moves to Isilon cold bucket.

3.1 Virtualization design VxFlex integrated rack delivers virtualization, compute, and storage in a scalable, easy to manage,

hyperconverged infrastructure appliance. It deeply integrates VMware vSphere virtualization software that

delivers an industry-leading virtualization platform to provide application virtualization with a highly available,

resilient, efficient on-demand infrastructure.

3.2 Compute design Following table shows the details of compute design for Splunk components.

Compute design consideration

Instance role Quantity Physical cores Memory (GB)

Search Head 1 16 96

Indexer 2 24 96

Cluster Master 1 16 96

Forwarder 1 16 96

Note 1: Dell EMC recommends to provisioning minimum of 12 cores to SVM for Splunk enterprise

deployments.

Note 2: Splunk Enterprise is a resource-intensive. For best performance, do not overcommit vCPU or

memory for Splunk instances. When hyper-threading is enabled, allocate the equivalent number of physical

cores.

3.3 Storage design Following table describes the VxFlex OS storage design. Multiple storage volumes have been carved out on

the VxFlex OS and Isilon, these are mapped to Splunk Virtual machines.

Storage design consideration

Instance role Quantity Operating System storage (GB)

Hot/warm bucket storage (TB)

Cold bucket storage (TB)

Search Head 1 300 0 0

Indexer 2 300 1 3

Cluster master 1 250 0 0

Forwarder 1 250 0 0

Note 1: For this configuration, Splunk sizer recommends 750 GB per indexer for hot/warm data retention, but

for this solution, 1 TB has been provisioned per indexer.

Note 2: For this solution, Isilon cold data buckets have been carved out for four months retention period. But

customers can configure it based on their needs and allocate storage appropriately.



3.4 Isilon Storage design A four-node Isilon A200 cluster has been used for validation. The following table provides detailed

configuration of Isilon:

Isilon node configuration

CPU RAM SSD capacity HDD capacity Network

2 x Intel Pentium Processors @D1508

16 GB 400 GB 40 TB 2 x 10 GbE

Note: It is recommended using Isilon H500 for enterprise Splunk deployments for cold data storage.

3.5 Network architecture The following figure demonstrates high-level network architecture and design of Splunk enterprise distributed

clustered deployment on VxFlex integrated rack:

Network architecture of VxFlex integrated rack



VxFlex networking details

Components Description

Cisco Nexus 93180YC-EX 10 Gbps & 25 Gbps TOR switches

Cisco Nexus 9332PQ 40 Gbps Aggregation switches

Cisco Nexus 3172TQ 1 Gbps & 10 Gbps Management switches

Data Domain 2 x 10 Gbps links

Application traffic 2 x 25 Gbps links

VxFlex storage traffic 2 x 25 Gbps links

Architecture flow:

• VxFlex integrated rack uses a pair of Cisco Nexus 93180YC-EX as TOR switches

• VxFlex integrated rack uses a pair of Cisco Nexus 9332PQ as Aggregation switches

• Cisco Virtual Port-Channel is configured between the TOR and Aggregation switches

• Cisco Nexus 3172TQ switch is used for OOB traffic with 1 GbE dedicated network

• Each node consists of four 25 GbE ports, two ports connected to each TOR (Cisco Nexus 93180YC-

EX) switch

• Both TOR (Cisco Nexus 93180YC-EX) switches have uplinks to aggregation (Cisco Nexus 9332PQ)

switches for redundancy and network bandwidth aggregation

For an overview of VxFlex integrated rack tech extension with Isilon, see Dell EMC VxFlex integrated rack

Technology Extension with Isilon storage

https://cpsdocs.dellemc.com/bundle/P_PG_IR_ISILON/pdf/enus/0




3.6 Splunk Enterprise clustered deployment design A Splunk indexer cluster offers benefits such as high availability, simplified scaling and disaster recovery. The

following figure shows the Splunk Enterprise clustered infrastructure deployment - Single Site (C1/C11) with

single search head, one cluster master and two indexer peers:

Splunk Enterprise clustered infrastructure for 50 GB/day data indexing volume with 30-day hot/warm retention.

Search head to search the data in the cluster.

Cluster Master or Master node manages the indexing tiering and is responsible for coordination and

enforcement of the configured data replication policy. The same cluster master has been configured as

license master.

Indexer peer nodes perform the indexing of ingested data.

Replication factor defines the number of copies of raw data that the Splunk cluster maintains. For more

details, see Splunk replication factor.

Search factor defines how many searchable copies of the indexed data needs to be maintained. For more

details, see Splunk search factor.

In addition, one universal forwarder (UF) was configured to send the log data to the cluster.

Note: Splunk recommended (and default) replication factor is 3, while a replication factor of 2 provides

minimal protection against a single indexer node failure.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Indexer/Thereplicationfactor

https://docs.splunk.com/Documentation/Splunk/7.3.1/Indexer/Thesearchfactor

Validation


4 Validation Splunk Enterprise has been deployed and validated on VxFlex integrated rack with following design:

• Four VxFlex integrated rack nodes

• Distributed Clustered Deployment-single site (C1 /C11) topology

• 50 GB ingestion per day volume with 30-day hot/warm storage retention

• Five VMs have been created, each VM is installed with Splunk binaries and assigned one search

head, two indexers, one forwarder, and one cluster master

Optimized the design for both high performance and data retention capability using VxFlex integrated rack for

storage of hot/warm and Isilon is integrated to provide configurable cold data bucket retention

For this validation, a 50 GB log data is generated and forwarded it via universal forwarder, the data gets

indexed and stores the events in hot bucket, where the events can be searched using search head. Once the

warm bucket volume size exceeds the specified limit, data is rolled into Isilon cold bucket.

4.1 Validation procedure To validate the data aging and searching on VxFlex integrated rack, following steps have been performed:

1. Generated 50 GB of log data and forwarded it using universal forwarder to Splunk cluster. The following

figure illustrates the size of the log data:

Size of log data

2. Data is moved to indexer from the universal forwarder. The processed data is stored in the hot bucket

path of indexer/data/splunk/homedb as shown in the following figure:

Homedb path

Validation


3. Validated the indexer events present in the indexer by running a search query on hot bucket data using

search head (GUI) as shown in the following figure:

Validation result of search

4. After 30-days hot/warm retention, observed that data moved from warm bucket to cold bucket path

/data/splunk/isilon_colddb as shown in the following figure:

Validation of cold data to Isilon path

The cold bucket path /data/splunk/isilon_colddb is mounted to Indexer VM using Isilon NFS. For

more information about Isilon NFS configuration, see A.3 Configure Isilon NFS for the VxFlex.

Conclusion


5 Conclusion This whitepaper provides detailed information on how to deploy Splunk enterprise with specific ingestion

volume per day with Isilon. This approach can be extended to other ingestion volume requirements (based on

the SVA guidelines) by scaling the required number of VxFlex integrated rack nodes and optionally leveraging

the Dell EMC Isilon scale-out NAS storage platform for cold bucket storage needs. The VxFlex integrated rack

system provides standardized hardware and software configurations enabling Splunk customers with non-

disruptive scalability to meet their performance and storage needs.

With Dell EMC VxFlex integrated rack, Splunk deployment gains a scalable, flexible and cost-effective

operational intelligence platform that leverages VxFlex OS software-defined storage and VMware for

virtualizing the core Splunk components, while Isilon provides optional scale-out storage for cold data.

Conclusion


A Appendix

A.1 Hardware and Software components

Component Definition

VxFlex integrated rack 4 x VxFlex Nodes (R640 servers):

• VxFlex OS version: R2_6.1

• ESXi version: 6.5

• CPU: 2 x Intel(R) Xeon(R) 2.70 GHz processor

• Memory: 384-GB RAM (12 x 32-GB DIMMs)

• Storage: 10 x 3.84 TB SSD

Network • 2 NIC cards, each having 2 ports 25 GbE connection

Splunk VM configuration 5 x Linux VMs:

• Operating system version: Red Hat Enterprise Linux Server release 7.5 (Maipo)

SVM configuration 4 x Linux VMs:

• Operating system version: SUSE Linux Enterprise 12

• Cores: 12

• Memory:10 GB

Splunk enterprise software Splunk universal forwarder

• Version: 7.3.1

• Version: 7.3.1.1

Isilon OneFS software • Version: 8.1.0.4

A.2 Best Practices

This solution implements the following Dell EMC and VMware best practices to provide optimal performance

for all Splunk Enterprise virtual machines running on the VxFlex integrated rack.

• Create a vSphere HA cluster to provide a virtualized, high-availability Splunk Enterprise environment

that is easy to use and cost-effective.

• Use a VMware VMXNET3 network adapter to optimize network performance.

• Splunk Enterprise is resource-intensive. For best performance, do not overcommit vCPU or memory

for Splunk instances.

• Use thick provisioned eager zeroed disk to optimize virtual disk performance. Use a single virtual

socket for each virtual machine. With virtual Non-Uniform Memory Access (NUMA) topology, a single

virtual socket that has fewer virtual CPU cores than the physical CPU cores of a socket in the

physical ESXi host is recommended.

• Use a VMware Paravirtual SCSI controller to increase throughput with significant CPU utilization

reduction in the SAN environment.

• Install VMware tools in the guest Operating System to improve virtual machine (VM) performance.

• Set the VM advance parameters numa.vcpu.preferHT to “true” for enabling hyperthreading with

NUMA in ESXi.

Conclusion


• It is strongly recommended to follow the VxFlex integrated rack standard best practices guide for cluster

formation and network configuration.

Isilon Best Practices

• Enable Smart Pools settings across all Isilon nodes and use an SSD as L3 cache for random read

acceleration

• Enable SmartConnect to provide automatic client connection load balancing and failover capabilities

• Enable SmartCache for write performance

• Use Optimization for concurrent data access pattern

• Use 10 Gb/s external network for data connection

• Increase network MTU to 9000 (Jumbo Frames)

A.3 Configure Isilon NFS for the VxFlex integrated rack

1. Create a subnet.

a. Log in to the Isilon web service using root account.

b. Go to Cluster Management > Network Configuration.

c. Click More > Add Subnet of groupnet0 to create a subnet.

2. Create an access zone.

a. Go to Access > Access Zones.

b. Click Create an access zone to create an access zone for Splunk.

3. Create an IP address pool.

a. Go to Cluster Management > Network Configuration.

b. Click More > Add Pool of subnet-10g to create an IP address pool.

4. Create NFS Export.

a. Go to Protocols > UNIX Sharing (NFS) > NFS Exports.

b. Select the Current Access Zone as Splunk.

c. Click Create Export and provide required details.

5. Add Isilon NFS storage to VxFlex integrated rack.

a. Log in to the vCenter client using the administrator account.

b. Go to Home > Inventory > Hosts and Clusters > ESXi server > Configure > Storage >

Datastores.

c. Click Add data store and provide required details.

6. Add Isilon cold Storage to each indexer VM and follow the procedure.

a. Log in to the vCenter client using the administrator account.

b. Click Indexer VM and Edit settings.

c. Click New Hard Disk and Add.

d. Provide the necessary details.

Conclusion


7. Prepare Splunk Cold buckets using Isilon disks on VMs.

a. Log in to the indexer using SSH.

b. Make a partition on the newly provisioned Isilon virtual disk:

fdisk /dev/sdd

c. Make a file system on the partition:

mkfs.xfs -f /dev/sdd1

d. Create a directory and mount the Isilon virtual disk to a separate mount point

mkdir -p /data/splunk/isilon_colddb

chmod -R 750 /data/splunk/isilon_colddb

chown -R splunk:splunk /data/splunk/isilon_colddb

mount /dev/sdd1 /data/splunk/isilon_colddb

vi /etc/fstab

/dev/sdd1 /data/splunk/isilon_colddb xfs defaults 0 0

8. Create a new configuration file indexes.conf in each indexer VM.

a. Go to the path $SPLUNK_HOME/etc/system/local/indexes and add the following

configuration for 30-day hot/warm retention and required configurable cold data retention

#volume definitions

[volume:hotwarm]

path = /data/splunk/homedb

[volume:cold]

path = /data/splunk/Isilon_colddb

# index definition (calculation is based on a single index)

[main]

homePath = volume:hotwarm/defaultdb/db

coldPath = volume:cold/defaultdb/colddb

homePath.maxDataSizeMB = 768000

coldPath.maxDataSizeMB = [configurable]

b. /opt/splunk/bin/splunk restart

Note: The configuration can be modified based on retention needs and any number of parameters can be

added. For more information about indexes configuration details, see indexes.conf.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Admin/Indexesconf

Conclusion


B Technical support and resources

B.1 Related resources

Note: Few links might require registration to access.

• Managing Indexers and Clusters of Indexers

• Hyperconverged Infrastructure Dell EMC VxFlex Family Overview

• Dell EMC VxFlex integrated rack Technology Extension with Isilon storage

• Splunk Enterprise Overview

• Splunk Validated Architectures

• Splunk replication factor

• Splunk search factor

• Splunk indexers configuration (indexes.conf)

B.2 Additional resources

• Dell EMC Online Support site (registration required)

• Dell EMC Hyperconverged Infrastructure

• Dell EMC VxFlex integrated systems

• Storage technical documents and videos


https://www.dellemc.com/en-us/hyperconverged-infrastructure/vxflex.htm

https://www.dellemc.com/th-th/collaterals/unauth/white-papers/products/converged-infrastructure/vxflex_overview_whitepaper.pdf


https://docs.splunk.com/Documentation/Splunk/7.3.1/Overview/AboutSplunkEnterprise


https://docs.splunk.com/Documentation/Splunk/7.3.1/Indexer/Thereplicationfactor


https://docs.splunk.com/Documentation/Splunk/7.3.1/Admin/Indexesconf

http://www.dellemc.com/hci

http://www.dellemc.com/vxflex

http://www.dell.com/storageresources

Documents

Splunk Enterprise on Dell EMC VxFlex integrated …...10 Splunk Enterprise on Dell EMC VxFlex integrated rack with Isilon | 000054 2.6 Splunk Enterprise Splunk Enterprise is a software