NimbleOS 5.3.0.0 Release Notes - Hewlett Packard Enterprise€¦ · KB-000246 MPIO Timeout Parameters for MSDSM and NimbleDSM in Windows 2012 R2 • For VMware, refer to the Common

NimbleOS 5.3.0.0 Release Notes

Version 5.3.0.0

Published October, 2020

Legal Notices

© Copyright 2020 Hewlett Packard Enterprise Development LP. All rights reserved worldwide.

Notices

The information contained in herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise productsand services are set forth in the express warranty statements accompanying such products and services. Nothing herein should beconstrued as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors oromissions contained herein.

Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying. Consistentwith FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Com-mercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no controlover and is not responsible for information outside the Hewlett Packard Enterprise website.

Acknowledgments

Intel®, Itanium®, Pentium®, Intel Inside®, and the Intel Inside logo are trademarks of Intel Corporation in the United States and othercountries.

Microsoft® and Windows® are either registered trademarks or trademarks of Microsoft Corporation in the United States and/orother countries.

Adobe® and Acrobat® are trademarks of Adobe Systems Incorporated. Java® and Oracle® are registered trademarks of Oracle and/orits affiliates.

UNIX® is a registered trademark of The Open Group.

Publication Date

Tuesday October 6, 2020 12:19:40

Document ID

old1592810548757

Support

All documentation and knowledge base articles are available on HPE InfoSight at https://infosight.hpe.com. To register forHPE InfoSight, click the Create Account link on the main page.

Email: [email protected]

For all other general support contact information, go to https://www.nimblestorage.com/customer-support/.

2Copyright © 2020 by Hewlett Packard Enterprise Development LP. All rights reserved.

Legal Notices

https://infosight.hpe.com

mailto:[email protected]

https://www.nimblestorage.com/customer-support/

Contents

NimbleOS 5.3.0.0......................................................................................................................4Important Update Note............................................................................................................................................................................................................4

Special Notes..................................................................................................................................................................................................................................4

New Features in 5.3.0.0...........................................................................................................................................................................................................7

Documentation.............................................................................................................................................................................................................................8

Verified Update Paths..............................................................................................................................................................................................................9

Known Critical Issues..............................................................................................................................................................................................................12

Resolved Critical Issues.........................................................................................................................................................................................................19

Resolved Issues.........................................................................................................................................................................................................................21

Known Issues..............................................................................................................................................................................................................................24

Copyright © 2020 by Hewlett Packard Enterprise Development LP. All rights reserved.

NOS

NimbleOS 5.3.0.0

5.3.0.0Version:

Tuesday October 6, 2020 12:19:40Revision:

The release notes describe the major changes, fixes, and known issues for this release of the NimbleOS. They do not includeall individual fixes and internal changes.

For technical support, contact HPE Nimble Storage Support at:

mailto:[email protected] (877-364-6253), option 2.

Important Update NoteUpdating NimbleOS can involve an update to component firmware on the standby controller. This can cause an email alertand automated case indicating "Standby Controller Not Available" when the firmware update process takes longer than fiveminutes. This is expected behavior and does not affect data services. At the end of the software update, you can check statusof both controllers in the Web UI under Manage > Hardware. One controller will be ACTIVE and the other STANDBY undernormal operating conditions following a successful software update.

All third-party software notices can be found on HPE InfoSight (https://infosight.hpe.com) on the Resources > Documentationpage:

https://infosight.hpe.com/resources/nimble/docs

The Documentation page also includes the General Terms and Conditions document. You can display this document byperforming the following steps:

1 In the navigation pane on the HPE InfoSight Documentation page, scroll through the Document Type list and selectSupport Policy.

2 In the page that appears, select General Terms and Conditions. This document opens in a browser tab.

Special Notes

DescriptionNote

HPE Nimble Storage continues to qualify configurations between releases. TheValidated Configuration Matrix provides information about validated configurationsand is updated frequently. It is a good practice to check your system configurationagainst this online tool. The Validated Configuration Matrix tool is available onHPE InfoSight:

https://infosight.hpe.com/resources/nimble/validated-configuration-matrix

CRITICAL

Arrays must be running NimbleOS 5.0.6.0 or later to update to NimbleOS 5.3.0.0.CRITICAL

An extended data services outage may occur with MS iSCSI initiator and Intel NICsusing the built-in Windows driver e1q60x64.sys (version 11.0.5.21/11.0.5.22).

If you encounter this problem, please update your system to use the latest Windowsdriver.

CRITICAL


NimbleOS 5.3.0.0 Important Update Note

mailto:[email protected]

https://infosight.hpe.com



DescriptionNote

A service outage may occur on Windows 2012 R2 hosts using Emulex or BroadcomFibre Channel HBAs with firmware/driver prior to 11.2. Update the Emulex orBroadcom firmware/driver to 11.2 or later

CRITICAL

Due to a known Red Hat Enterprise Linux bug 1002727, while running virtualizedin VMware ESX, manually rebooting the active controller in presence of heavy IOsusing the reboot --controller command on a Fibre Channel array may trigger anincorrect retry initiated by RHEL guests running the following kernel versions:

• 6.4 and earlier• 6.5 without the patch• 7.0 without the patch

This incorrect retry logic may lead to unexpected application behavior. In theseenvironments, we recommend the failover command instead.

CRITICAL

Due to a known Red Hat Enterprise Linux bug 3550561, unexpected applicationbehavior may occur on RHEL 7.5 hosts with kernel-3.10.0-862.3.2.el7 or derivativesusing Emulex FC FCoE HBAs (lpfc driver) and raw devices. To avoid this issue:

• If running RHEL 7.6, update to kernel-3.10.0-957.el7 or later.• If running RHEL 7.5z, update to kernel-3.10.0-862.25.3.el7 or later.

CRITICAL

As outlined in the current Validated Configuration Matrix, HPE Nimble Storagefully supports Windows guest operating systems on Microsoft Hyper-V, includingVirtual Fibre Channel (VFC) connectivity and multipathing with HPE Nimble StorageDSM and VSS support. However, Linux guest operating systems running in Hyper-V VFC configurations are not qualified.

Running Red Hat Linux guest operating systems with the “Linux Integration Ser-vices” kit installed, or with hv_storvsc drivers in such configurations can lead toRed Hat bug 1364282, which can cause an unexpected service outage.

CRITICAL

Starting with NimbleOS 5.1.1.0, the size of the software package now exceeds 2GB, which may lead to lengthier software download times. Previously, the sizes ofthe NimbleOS 5.0.x download packages were approximately 1.6 GB, and NimbleOS4.x packages were approximately 900 MB.

Important

After completing the NimbleOS update for array groups configured for SynchronousReplication, download the corresponding version of the Synchronous ReplicationWitness software, and update the witness host.

Important

Microsoft Offload Data Transfer (ODX) is not supported if the destination volumehas synchronous replication enabled.

Important

As of vSphere 7.0, VMware has discontinued the flex client. Consequently, the HPENimble Storage vCenter Plugin no longer supports the flex plugin for vCenter 7.0.

Important


NimbleOS 5.3.0.0 Special Notes

https://infosight.hpe.com/org/urn%3Ainfosight%3A3d132a16-b52d-4040-a655-b3a9c3647505/resources/nimble/software

https://infosight.hpe.com/org/urn%3Ainfosight%3A3d132a16-b52d-4040-a655-b3a9c3647505/resources/nimble/software

DescriptionNote

You can enable deduplication for CS1000, CS3000, CS5000, CS7000, CS700, andCS500 arrays on a volume only if the corresponding storage pool has a Flash toDisk Ratio (FDR) greater than 4%. To calculate the FDR, obtain the "Total arraycapacity (MiB)" and "Total array cache capacity (MiB)" values by using the HPENimble Storage CLI command pool_name. This command returns the Pool capac-ity (MiB), which is the "Total array capacity (MiB)", and the Pool cache capacity(MiB), which is the "Total array cache capacity (MIB)".

Then perform the following calculation:

FDR = "Total array cache capacity (MiB)"/"Total array capacity (MiB)" * 100

If the array has sufficient capability for deduplication, the pool --info commandwill also show a value for dedupe capacity (MiB).

Note On the HF20H, HF20, HF40, and HF60 platforms, pool --info displays "N/A"as the value for dedupe capacity (MiB). This because you can enable deduplicationfor the entire array.

Important

For connections to the NimbleOS GUI, you must have port 5392 open for the GroupManagement IP address and both diagnostic IP addresses.

Important

Numerous host integration toolkits are supported in NimbleOS 5.3.0.0. It isstrongly recommended that they be installed on all Windows, Linux, and VMwarehosts. For more information about supported toolkits, refer to the Validated Con-figuration Matrix, which is available on HPE Nimble Storage InfoSight:


Important

HPE Nimble Storage recommends that you update to HPE Nimble Storage WindowsToolkit (NWT) 7.0.1 or later if you are using Microsoft VSS Synchronization andNimbleOS 5.1.4.200 or later.

Using application consistent snapshots with earlier versions of NWT and NimbleOS5.1.4.100 may result in the following error messages:

• In the host's VSS requestor log (C:\ProgramData\Nimble Storage\Logs\VssRe-questor.log):

PID:1996 TID:5752 ERR reqcommon. cpp:683 Request-Status=QueryStatus(), Function=pAsync->QuerySta-tus(), Error=VSS_E_PROVIDER_VETO, rc=SystemError,ca=ContactSupport

• In the Windows event viewer:

event id 4100: EndPrepareSnapshots method: failedto find LUN s/n <SERIAL_NUMBER> on connected ar-rays. Make sure that the Nimble array version iscompatible with this version of Nimble WindowsToolkit.

event id 4170: Nimble VSS provider is not compati-ble with the current version of the Nimble arraysoftware(). Install appropriate version of theNimble VSS provider.

NWT 7.0.1 resolves this issue.

Important


NimbleOS 5.3.0.0 Special Notes


DescriptionNote

HPE Nimble Storage Connection Manager (NCM) for VMware 7.0 is signed byVMware for ESXi 7.x. It can be installed through the VMware Update Manager oresxcli command without the --no-sig-check flag.

See the NCM for VMware Release Notes 7.0 or later and the latest VMware Inte-gration Guide for further details.

To locate the latest version of the guide, log in to HPE InfoSight. Choose Re-sources > Nimble Storage Documentation. In the left pane, click IntegrationGuide, then click Connection Manager (NCM) for VMware. From the list displayed,choose the version of the guide that you want.

Important

Various timeout values affect HPE Nimble Storage targets from Windows/Linuxhosts. Before you update the NimbleOS, install the HPE Nimble Storage WindowsToolkit (NWT) or HPE Nimble Storage Linux Toolkit (NLT) on the host or tune thetimeout values. Timeout details for various operating systems can be found onHPE InfoSight under Resources > Documentation. From the HPE Nimble StorageDocumentation page, locate the article you want.

The following Knowledge Base articles and Integration Guides explain how toconfigure and verify host timeout settings for the major supported operating sys-tems (OS):

• For Windows, refer to KB-000052: Windows Host Disk Timeout Values.

In the context of Microsoft Windows, the following article should also be con-sidered:

KB-000246 MPIO Timeout Parameters for MSDSM and NimbleDSM in Windows2012 R2

• For VMware, refer to the Common Tasks and Best Practices > Host TimeoutValues section of the VMware Integration Guide.

• For Linux, refer to KB-000304: Linux Host Disk Timeout Values.

Important

vVol VMs cannot be claimed after they are deleted from the downstream array.

A vVol VM can be protected and may subsequently be replicated to a downstreamarray (as configured in the storage policy). In the case where this vVol VM wasdeleted, a supported “claim” workflow allows us to claim this vVol VM on thedownstream array.

Due to validation failures in the vCenter, this workflow is not supported if it isperformed in an environment where the vCenter version is 6.5 or later.

See the following VMware DCPN Ticket Reference:

https://dcpn.force.com/TechnicalRequestCaseRedesignPart-ner?Id=5000H00001JRKhf

Important

New Features in 5.3.0.0NimbleOS 5.3.0.0 introduces support for the following feature:

External KMIP Integration

NimbleOS now supports using an external key manager to manage the master encryption key. External key management isdone via the industry standard KMIP protocol, and any KMIP supporting key manager may be used.


NimbleOS 5.3.0.0 New Features in 5.3.0.0



https://infosight.hpe.com/InfoSight/media/kb/active/htr1455131698674.whz/index.html

https://infosight.nimblestorage.com/InfoSight/media/kb/active/htr1455131726154.whz

https://infosight.nimblestorage.com/InfoSight/media/kb/active/htr1455131726154.whz

https://infosight.hpe.com/InfoSight/media/kb/active/htr1455131700640.whz/index.html

https://dcpn.force.com/TechnicalRequestCaseRedesignPartner?Id=5000H00001JRKhf

https://dcpn.force.com/TechnicalRequestCaseRedesignPartner?Id=5000H00001JRKhf

DocumentationThese Release Notes and other user documentation are available on HPE InfoSight:


You can manually reach the documentation page by logging onto HPE InfoSight and selecting Resources > Nimble Storage >Documentation.

Document Search Interface

There are several methods you can use to locate the documents you need.

The Nimble Storage Documentation page provides a search interface that allows you to search for information across alldocumentation, including support and knowledge base articles, best practices, solutions and integration guides, productdocumentation, and configuration matrices.

To go directly to a document, use the navigation pane on the left side of the Nimble Storage Documentation page. Thenavigation pane organizes documents into categories, including:

• Document Type• Nimble Software and Solutions• Software Version• Integration• Platform

You can use the page scroll bar to move up and down the navigation pane.

Third-Party Software Notices

All third-part software notices can be found in the Documentation Portal on HPE InfoSight.

Here are the steps to manually access the third-party software notices.

1 Log in to HPE InfoSight (https://infosight.hpe.com) .

2 From the menu, select Resources Nimble Documentation .

3 In the left navigation pane of the Documentation Portal, scroll through the Document Type section and selectSupport Policy.

4 From the list of documents, select General Terms and Conditions. The document opens in a new browser tab.

Core User Documentation

The following is the core user documentation for NimbleOS:

• GUI Administration Guide• CLI Administration Guide• SNMP Reference• Command Reference• REST API Reference

If you are using an HPE Nimble Storage dHCI-enabled array, you should also check the dHCI Deployment Guides and GettingStarted Guide.

Workflow Documents

There are several workflow guides that contain procedures you can perform using either the CLI or the GUI. Each workflowguide covers a specific, frequently performed task related to HPE Nimble Storage products. Each task described by a workflowdocument is explained in detail in the GUI Administration Guide and the CLI Administration Guide.


NimbleOS 5.3.0.0 Documentation


Hardware

Documentation for all hardware components is available on HPE InfoSight. Click the Hardware Guide link in the DocumentType category. Hardware documentation includes array and expansion shelf installation quick start guides, installation,upgrade, and replacement guides, and comprehensive hardware guides.

Host Integration Guides

Host Integration Guides are available from HPE InfoSight. To locate these documents on the HPE InfoSight Documentationpage, scroll down the navigation pane to the section called Integration Guide.

Note A single Host Integration Guide supports multiple version of NimbleOS and the companion Integration Toolkit softwarepackages. The version number listed on the guide might be different from the version numbers of the NimbleOS and Toolkitsoftware packages that it supports.

Verified Update Paths

Table 1: From Versions 5.x

From Versions 5.x

To VersionFrom Version

5.3.0.05.2.1.300

5.3.0.05.2.1.200

5.3.0.05.2.1.100

5.3.0.05.2.1.0

5.3.0.05.1.4.200

5.3.0.05.1.4.100

5.3.0.05.1.4.0

5.3.0.05.1.3.100

5.3.0.05.1.3.0

5.3.0.05.1.2.100

5.3.0.05.1.2.0

5.3.0.05.1.1.0

5.3.0.05.0.10.0

5.3.0.05.0.9.100

5.3.0.05.0.9.0

5.3.0.05.0.8.100

5.3.0.05.0.8.0

5.3.0.05.0.7.300

5.3.0.05.0.7.200

5.3.0.05.0.7.100

5.3.0.05.0.7.0


NimbleOS 5.3.0.0 Verified Update Paths

From Versions 5.x


5.3.0.05.0.6.0

5.0.10.05.0.5.300

5.0.10.05.0.5.200

5.0.10.05.0.5.0

5.0.10.05.0.4.0

5.0.10.05.0.3.100

5.0.10.05.0.3.0

5.0.10.05.0.2.0

5.0.10.05.0.1.100

5.0.10.05.0.1.0


From Versions 4.x


5.1.4.2004.5.6.0

5.1.4.2004.5.5.0

5.1.4.2004.5.4.0

5.0.10.04.5.3.0

5.0.10.04.5.2.0

5.0.10.04.5.1.0

5.0.10.04.5.0.0

5.0.10.04.4.1.0

5.0.10.04.4.0.0

5.0.10.04.3.1.0

5.0.10.04.3.0.0

5.0.10.04.2.1.0

5.0.10.04.2.0.0

5.0.10.04.1.0.0


From 3.x Versions


5.0.10.03.9.3.0

5.0.10.03.9.2.0



From 3.x Versions


5.0.10.03.9.1.0

5.0.10.03.9.0.0

5.0.10.03.8.1.0

5.0.10.03.8.0.0

5.0.10.03.7.0.0

5.0.10.03.6.2.0

5.0.10.03.6.1.0

5.0.10.03.6.0.0

5.0.10.03.5.4.0

5.0.10.03.5.3.0

5.0.10.03.5.2.0

5.0.10.03.5.0.0

5.0.10.03.4.1.0

5.0.10.03.4.0.0

5.0.10.03.3.0.0

5.0.10.03.2.1.0

5.0.10.03.1.0.0


From 2.0.x VersionsFrom 2.1.x VersionsFrom 2.2.x, 2.3.x Versions

To VersionFrom VersionTo VersionFrom VersionTo VersionFrom Version

2.1.9.12.0.8.02.3.18.02.1.9.14.5.6.02.3.18.0

2.1.9.12.0.7.02.3.18.02.1.9.04.5.6.02.3.16.0

2.1.9.12.0.6.*2.3.18.02.1.8.04.5.6.02.3.15.0

2.1.9.12.0.5.02.2.9.02.1.7.04.5.6.02.3.14.0

2.1.9.12.0.4.02.2.9.02.1.6.04.5.6.02.3.12.*

2.2.9.02.1.5.04.5.6.02.3.9.*

2.2.9.02.1.4.04.5.6.02.3.8.0

2.2.9.02.1.3.04.5.6.02.3.7.0

2.2.9.02.1.2.04.5.6.02.3.6.0

2.1.9.12.1.1.04.5.6.02.3.4.0

2.1.9.12.1.0.04.5.6.02.3.3.0

4.5.6.02.3.2.1

4.5.6.02.3.2.0



From 2.0.x VersionsFrom 2.1.x VersionsFrom 2.2.x, 2.3.x Versions


4.5.6.02.3.1.0

3.9.3.02.2.11.0

3.9.3.02.2.10.0

3.9.3.02.2.9.0

3.9.3.02.2.7.*

3.9.3.02.2.6.0

3.9.3.02.2.5.*

2.2.11.02.2.3.*

2.2.11.02.2.2.0

2.2.11.02.2.1.0

2.2.11.02.2.0.0


From 1.0.x VersionsFrom 1.3, 1.2, 1.1 VersionsFrom 1.4.x Versions


Contact Support1.0.7.*1.4.6.01.3.*.*2.1.9.11.4.12.0

Contact Support1.0.6.*1.4.6.01.2.*.*2.1.9.11.4.11.0

1.2.2.01.1.*.*2.1.9.11.4.10.0

2.1.9.11.4.9.0

2.1.9.11.4.8.0

2.1.9.11.4.7.0

1.4.12.01.4.*.*

Known Critical Issues

Known Critical Issues in NimbleOS version 5.3.0.0

WorkaroundDescriptionTitleComponentID

When planning to remove amember array from group,schedule a planned maintenancewindow and place all ESX hostsinto maintenance mode to mini-mize impact to availability. ESXtypically resumes connection tovVol datastores, and reconnectsto VMs, after a period of 15-30minutes automatically withouta manual intervention.

Scaled vVol environments with500 vVol VDI VMs or more than5000 Nimble vVol volumes mayexperience IO disruption whenremoving a member array fromgroup. Symptom of problemwould appear as vVol datastoresbeing (inaccessible). VirtualMachine status would also ap-pear as (inaccessible).

Removing memberarray from multi-array group maycause IO disruptionto scaled vVol envi-ronments

Data ServiceAS-77607


NimbleOS 5.3.0.0 Known Critical Issues



Not applicableVolume moves transfer datafrom one Nimble array to anoth-er. During this move, if the hostsends I/O to the incorrect array,the I/O needs to be forwardedto the correct array owning thedata. This results in higher thanusual I/O latency. To avoid thisissue in VMware environments,the Nimble Connection Managerfor VMware needs to be in-stalled on all hosts accessing thevolume.

Volume move mayresult in latency ifNimble ConnectionManager is not in-stalled


Contact HPE Nimble StorageSupport

When the Data Service detectsa metadata inconsistency, theservice may restart repeatedlyand hosts could experience un-expected application behavior.

Data Servicerestarts when de-tecting metadatainconsistency


Contact HPE Nimble StorageSupport to identify and replacethe faulty SSD.

If the array encounters a faultySSD with multiple bad seg-ments, the Log-structured FileSystem (LFS) attempts to re-solve the errors. If the array isunder high load and LFS has ahigh humber of block errors toresolve it may exhaust the IORWbuffers causing the Data Serviceto restart repeatedly.

Data service mayrestart repeatedlydue to faulty SSD


Not applicableWhile committing internaltransactions, Data Service mayhit a rare race condition. To re-cover from this Data Servicemight restart

Data service mayrestart due to arace condition


Not applicableA disruption in network connec-tions can cause Data Service torestart unexpectedly.

Data Service mayunexpectedlyrestart


Review network devices toidentify and reduce networkchecksum errors.

In environments with frequentnetwork checksum errors, inter-ruptions of Replication and DataService restarts may occur.

Replication inter-ruptions and DataService restartsmay occur due tonetwork errors


Not applicableDuring snapshot replication ofa dedupe-enabled volume, thedownstream array file systemmay restart due to an out-of-memory condition.

Snapshot replica-tion of deduplica-tion-enabled vol-umes may lead toFile System restart






Not applicableData Services may unexpectedlyrestart due to out of memorycondition while running snapreplication on dedupe enabledvolumes with high compressionratios.

Data Service mayrestart unexpected-ly


Not applicableThe Data service may restartwhen Bin Migration is going on.This can happen when followingactivities happen together: 1.Bin migration is occurring for avolume. 2. Group Managementservice restart (because of anyreason). 3. Group Managementservice unable to re-sync withData service after &nb-sp; restart.This can lead to Data servicerestart but it will not impact binmigration as after Data servicerestart bin migration will re-sumed automatically.

Rare race conditionbetween Data ser-vice and GroupManagement ser-vice cause Dataservice restart


Not applicableData Service may restart whenoperations of volume migrationare stuck waiting for ownershiptransfer completion.

Data Service mayrestart unexpected-ly during volumemigration


Contact HPE Nimble StorageSupport.

Counters used for space account-ing may cause validation to fail.The array may see a blip inspace usage because of whichwrites may be denied and inworst case, volume may go of-fline.

Writes may be de-nied due to coun-ters in space ac-counting


Not applicableOn new array installations, whencreating the first vVol datastoreon the host, there is a possibilitythat the datastore is inaccessi-ble initially. However, it will be-come accessible within 5 min-utes.

Delay for first vVoldatastore becom-ing accessible

Host IntegrationAS-100561

If after 20 minutes the controllersensors do not report goodstate, please contact HPE Nim-ble Storage Support for assis-tance.

During boot up due to a knownIntel defect the controller sen-sors may report missing for aperiod of time in the array alerts.After about 15-20 minutes, itreturns to a valid state and thesensors should report validreadings again.

Controller sensorsmissing forAFxx/HFxx arrays

PlatformAS-86764






HPE Nimble Storage has identi-fied a rare firmware defect in asubset of drives which can, un-der certain write intensiveworkloads, cause the array tounder-perform.

Performance affect-ing firmware defectin a subset of 6TBdrives.

PlatformAS-94961

Not applicableDuring internal file operations,processes may be waiting for alock to be released. If the waittime exceeds 30 seconds, a ser-vice health check may restartthe Data service to recover.

Data service mayrestart duringwhen file operationtimeout is exceed-ed

PlatformAS-86099

Not applicableThe Data Service on the arraymay restart when timeout forinternal communication betweenarray controllers has been ex-ceeded. The service restarts torestore the communication.

Data Service mayrestart due tohealth check failure

PlatformAS-104517

Contact HPE Nimble StorageSupport to review disks for re-placement.

In rare instances, the Data Ser-vice may restart if the array hasmultiple bad drives which makeIO handling very slow.

Data Service mayrestart if the arrayhas multiple baddrives

PlatformAS-108793

Not applicableNimble Drive Error Recovery(NDER) is activated for drivesfailing I/O in an attempt to recov-er the drive. In rare instances,the process may surpass iSCSIhost timeout values, causinghost I/O inaccessibility.

NDER process maylead to host recon-nects

PlatformAS-96053

Not applicableThe current thermal policy hasa limitation where individualPCIe components are not moni-tored on card-by-card basis. Asa result, the high level tempera-ture policy that is current imple-mented, is sometimes incapableof regulating the temperaturesof individual cards.

Limited thermalmonitoring policyfor PCIe compo-nents

PlatformAS-107489

Contact HPE Nimble Support forassistance in restoring array toActive/Standby status.

In rare instances the intercon-nect between controllers maybe down following a controllerreseat.

Array remains insolo/stale aftercontroller reseat.

PlatformAS-51053





Verify that the software updatepackage has been downloadedto the system and then attemptthe software update again usingthe software --resume_updatecommand from the CLI.

During software update fromNimbleOS 3.x versions to laterversion, a failure to extract thesoftware update package mayincorrectly return the errormessage /tmp is out of space.even if there is still space in thedirectory.

Incorrect softwareupdate error mes-sage /tmp is out ofspace.

PlatformAS-93456

Not applicableGroup Management service mayrestart due to a race conditionbetween threads when a volumeaccess control list is removed.The service restarts to recoverfrom the condition.

Group Manage-ment service mayrestart due to racecondition

System Manage-ment

AS-110030

A Manual Group Leader Failoverwill be required to restore FibreChannel connectivity to thehosts.

An Automatic Failover (AFO) ofthe Group Management Serviceswill not be initiated if all FibreChannel (FC) interfaces on theGroup Leader array fail on bothcontrollers.

No AutomaticFailover in theevent the host los-es all FC connectiv-ity to an array

System Manage-ment

AS-94737

Not applicableThe Automatic Switchover Ser-vice internally creates and clos-es threads each time duringAutomatic Failover (AFO) quo-rum setup and tear down. Thismay cause the service to eventu-ally crash after reaching themaximum thread limit. The sys-tem recovers automaticallywhen the Automatic SwitchoverService restarts.

AutomaticSwitchover Servicerestarts due tothread limitations

System Manage-ment

AS-89701

Please contact HPE NimbleStorage Support

The Automatic Switchover Ser-vice may restart unexpectedlywhen the witness is removed orAutomatic Switchover is dis-abled on an array group. In rareinstances, this may lead to anissue with a database entrywithin the array is not clearedsuccessfully. If this array resetupis attempted on this array, theoperation will fail when the Ar-ray Management Service encoun-ters this stale entry.

Array resetup failsdue to previouscomplications withArray SwitchoverService

System Manage-ment

AS-107367





A failover can be initiated in or-der to restart the Group Manage-ment Service. You may alsocontact HPE Nimble StorageSupport to restart the servicemanually.

When performing a controllerupgrade to a high-end model,the object limits will still showthe lower limits if the GroupManagement Service is notrestarted.

Group Manage-ment Service mustbe restarted to un-lock additional vol-ume limits aftercontroller upgrade

System Manage-ment

AS-65615

Please contact HPE NimbleStorage Support.

When there are network commu-nication issues between theGroup Leader and BackupGroup Leader, the system goesinto an out-of-sync condition. Inrare circumstances, even afternetwork connection is restored,the array group may still remainout-of-sync.

Array group re-mains out-of-syncfollowing networkrecovery.

System Manage-ment

AS-106276

Not applicableThe Group Management servicemay restart during an arrayshutdown while processingREST request. No user opera-tions are impacted because thearray is already in the middle ofa shutdown. The shutdown pro-ceeds normally.

Group Manage-ment service mayrestart during arrayshutdown

System Manage-ment

AS-61614

Use unique name when creatingsnapshot collection.

If an attempt is made to createa snapshot collection with samethe name of an existing snap-shot collection, an error indicat-ing object already exists will bereported and the Group Manage-ment service may restart. Theservice will recover upon restart.

Group Manage-ment service mayrestart due to dupli-cate snapshot col-lection name

System Manage-ment

AS-94113

Not applicableA system management processcan restart when the system isunder heavy load. The systemrecovers automatically. The Da-ta service is not affected.

Group Manage-ment Servicerestarts underheavy load

System Manage-ment

AS-100254

Not applicableThe Array Management servicemay restart due to a race condi-tion encountered during serviceshutdown. The service will recov-er after the restart.

Array Managementservice restart dur-ing service shut-down

System Manage-ment

AS-98124

Contact HPE Nimble StorageSupport for assistance in deter-mining the cause of the failure.

If a software update precheckfails, in some cases it will returnonly the failure status withoutproviding additional informationabout the cause of the failure.

Software precheckfailures returngeneric error mes-sage

System Manage-ment

AS-87736





Not applicableGroup Management may be-come unavailable temporarilywhen a large number of restoresare performed on a volume.

Group Manage-ment Services un-available temporar-ily when perform-ing volume restore

System Manage-ment

AS-98194

Not applicableIn rare occurrences, a customer-initiated reboot may cause akernel reboot on the active con-troller. This will cause a longerreboot cycle.

Graceful shutdowntakes longer thanexpected

System Manage-ment

AS-95169

There is no workaround for thisissue. Controller failover willmaintain data availability.

The arrays Data Service mayrestart under certain conditionsin order to maintain data avail-ability. In some cases, the restartof this process takes longer thanexpected, resulting in an unex-pected timeout and a controllerfailover. The controller experi-encing the timeout will rebootand become the standby con-troller. This event is non-disrup-tive.

Controller failovermay be triggeredby delayed shut-down of an internalarray process

System Manage-ment

AS-105612

Not applicableController may reboot unexpect-edly if there is high memory uti-lization for the java and/or jettyprocesses on the array.

Controller may un-expectedly restartdue to high memo-ry utilization

System Manage-ment

AS-84499

Use volume collections to checksyncRep volumes, or use otherfilters to meet the needs

There is currently no way to fil-ter volumes using SynchronousReplication within the array GUI.

Unable to Filtervolumes using Syn-chronous Replica-tion

System Manage-ment

AS-92379

Refresh the page to restorebutton functionality.

After performing a bulk updateoperation from the Manage >Data Storage > Volumes >volumename > Data Protec-tion tab in the GUI, the pagefooter may not update and pre-vious button may be unavail-able.

Page footer in GUImay fail to updateafter bulk updateoperation

System Manage-ment

AS-97968



Resolved Critical Issues

Resolved Critical Issues in NimbleOS version 5.3.0.0


Not applicableWhen a blocks fingerprint is re-quested and the correspondingdomains Fingerprint Index isbeing merged in parallel, DataService may restart due to arace between Fingerprint Indexslookup and merge operations.

Data Service mayrestart during fin-gerprint lookup


Not applicableUnder certain conditions, theData service may restart duringarray internal index processingwithin a short time span. Trans-actions during the processingmay take too long to completewithin the defined time span,which causes the service torestart.

Data Service mayrestart unexpected-ly due to healthcheck failure.



In rare cases during index cre-ation, when a 16 TiB volume isfully unmapped, the resultingindex structure fails verificationand brings down the Data Ser-vice leading to an outage.

Index verificationfails if a 16 TiB vol-ume is completelyunmapped causingData Service to godown



During data block allocationstatus application programminginterface (API) execution the filesystem service may restart. Thishappens when data block alloca-tion start offset is chunk-size-bytes aligned but data block al-location end offset is not alignedwith chunk-size-bytes.

File system servicemay restart unex-pectedly during da-ta block allocationstatus API execu-tion


Not applicable, on restart DataService would behave normally.

Generation delete operationsand NVRAM to disk data flushoperation can cause Data Ser-vice to restart due to healthcheck failure, as it can holdcheckpoint for a long time.

Data Service mayrestart due to vol-ume managerhealth check failure



NimbleOS 5.3.0.0 Resolved Critical Issues



Not applicableWhen the Group Managementservice restarts for unrelatedreasons, the host sends setcontext (creates a new session)because its VASA sessions havebeen invalidated with the GroupManagement service goingdown. However, when the newsession call is made, we createa new session, then realizeGroup Management is down,and send back an error to thehost. Unfortunately, the newsession is created (object inmemory), but is not truly a validsession. These sessions couldaccumulate to a large numberover time, which may cause anout of memory condition forJetty. Once the service runs outof memory, it can also restartunexpectedly.

VASA Sessions cre-ated and cachedeven though GroupManagement ser-vice is unavailable.


Please contact HPE NimbleStorage Support if you en-counter this issue.

Under specific circumstances,services may fail to start on areplacement controller.

Services may fail tostart on a replace-ment controller

PlatformAS-106391

Contact HPE Nimble StorageSupport if there are multiplerestarts to work around the is-sue.

In rare cases while committinglarge internal transactions, theprocess may timeout. As a re-sult, the Data Service mayrestart to recover the condition.

Data Service mayrestart while com-mitting large inter-nal transactions

PlatformAS-103129

In order to add more 4 or moreProliant Servers, theworkaround is to add up to 3servers at a time from the plug-in.

Currently, if customer plans toadd 4 or more Proliants serversin their dHCI deployment via theplugin, the operation fails.

Plugin: Cannot add4 or more serversin the dHCI deploy-ment

dHCIAS-104924

Create a pool on the BackupGroup Leader array, then acti-vate the shelf.

Group management service mayrestart during shelf activationon Backup Group Leader (BGL)array. This occurs if a user triesto activate a shelf for BGL arraywhich is not associated with anypool. The Group managementservice restarts because of anempty pool.

Group manage-ment servicerestart during shelfactivation

System Manage-ment

AS-103976


NimbleOS 5.3.0.0 Resolved Critical Issues



Disable Automatic Switchoverusing the steps below: &nb-sp; 1. Loginto the array GUI &nb-sp; 2. Select Admin-istration > Availability &nb-sp; 3.Uncheck the Enable checkboxfor Automatic Switchover &nb-sp; 4. ClickSave Use Manual Failover forSynchronous Replication Vol-umes in place of ASO.

In rare circumstances, especiallyin deployments without networkredundancy, if the group leaderarray loses connectivity to boththe backup group leader and thewitness, this will result in Auto-matic Switchover (ASO). Thereis a small risk that not all writeswill have been mirrored to thepartner array making ASO anunsafe operation.

Unsafe AutomaticSwitchover (ASO)in the event of con-nectivity loss.

System Manage-ment

AS-105878

Not applicableThere is a possibility of sporadic,transient, active directory au-thentication failures. In thesecases, the system will recoveron its own, requiring no user in-teraction.

Intermittent loginfailures due to Ac-tive directorylookups failures

System Manage-ment

AS-92465

Resolved Issues

Resolved Issues in NimbleOS version 5.3.0.0


Not applicableWhen a dedupe domain deletionis in progress during softwareupdate, internal metadata needsto be cleaned up in order for thedeletion to make progress. Dur-ing this process, the Data Ser-vice may restart in an attemptto automatically recover.

Data Servicerestart duringdeduplication do-main deletion andsoftware update


Remove the IP with link statusdown from configuration

This scenario is rare to happenas timeout values for most ofservices would prevent this sce-nario to be triggered. Eventuallyan IP with link up status will bepicked up within the timeoutperiod of the service.

Services that relyon TCP connec-tions might be inter-rupted if IPs areconfigured but linkstatus is down


NoneData Service may restart whengathering garbage collectiontelemetry if cache subsystem isnot fully initialized.

Data Service mayrestart when gath-ering garbage col-lection telemetry



NimbleOS 5.3.0.0 Resolved Issues



Contact HPE Nimble StorageSupport for assistance with plu-gin registration.

When array software is updatedusing the GUI, the new vCenterplugin is not getting installedproperly this is because there isa mismatch of plugin versionspecified in the plugin-pack-age.xml verses the actual pluginversion.

vCenter plugin notgetting deployedproperly becauseof version mis-match


After the controller reboots,BMC firmware is restarted andis functional again automatically.

In rare cases, out of order com-mands being sent to the Base-board Management Controller(BMC) may return out of orderresponses that are not handledin the correct order by Intelli-gent Platform Management In-terface (IPMI) software. In thisinstance, the IPMI messagequeue loses track of messageorder. The IPMI message queuenot being able to return IPMIWatchdog messages to thewatchdog thread causes thewatchdog thread to timeoutleading to an automatic reboot.While this BMC Watchdog time-out issue is specific to theAFx/HFx systems, this is not ahardware issue. Therefore,hardware replacement is unnec-essary.

IPMI software maynot handle com-mand exchangecorrectly with BMCleading to unex-pected reboots ofAFx/HFx con-trollers

PlatformAS-90455

Before an interface is in state,gracefully remove VLAN tag-ging from the Quad Port inter-face. After an interface is instate, perform controller failoverto recover the condition. Howev-er, with VLAN tagging still en-abled, subsequent events canoccur. Therefore, consider theaforementioned workaround tominimize chance of a re-occur-rence.

When a subnet on the array isconfigured to use VLAN tag-ging, Quad Port interfaces inthat subnet may become unre-sponsive. As a result, hosts con-nected to an unresponsive inter-face, on that VLAN, may experi-ence communication interrup-tion. Over time, this conditionmay occur on every interfacewithin that subnet, causing insta-bility with that subnet.

Interfaces on QuadPort 10GbE BaseTand 10GbE SFPcards using VLANtagging may be-come unresponsive

PlatformAS-91379





Not ApplicableThis scenario is rare to happenbut presently not handledgracefully leading to a File Sys-tem restart for recovery. The is-sue is triggered when a writerequest with valid length is re-ceived in SCSI Command De-scriptor Block but invalid (zero)value in Data-Out Buffer.

File System servicemay restart whenan invalid write re-quest is received

SANAS-94761

Rename the folders to beunique. If the Event Manage-ment service continues torestart or remains unavailable,contact HPE Nimble StorageSupport.

When an alert is raised on oneof two folders having the samename but are located in twoseparate pools, the Event Man-agement service may restart re-peatedly.

Event Managementservice restartwhen two folders indifferent poolshave the samename

System Manage-ment

AS-101273

Not applicableNimbleOS uses a defined statemachine for the replicationworkflow. At the end of the exe-cution of each step defined inthe state machine, it moves tothe next step. If it leads to anyunexpected step throughout theworkflow then it will lead to as-sertion failure which results inGroup Management servicerestart.

Group manage-ment service mayrestart due to asser-tion failure

System Manage-ment

AS-96143

Not applicableInternal workflow processingrelated to the replication partnerobject may cause the process todeadlock, resulting in Groupmanagement service restart.

Group manage-ment service mayrestart unexpected-ly

System Manage-ment

AS-105064

Contact HPE Nimble StorageSupport to identify ACLs inforced-delete/create-retry state,verify these ACLs are associatedwith snapshots that no longerexist, and delete these ACLsfrom the internal NimbleOSdatabase.

In certain scenarios, a volumedeletion will not complete dueto the existence of a stale ACLassociated with it. This staleACL is associated with a snap-shot of the volume that waspreviously deleted. These vol-umes will not show up in theCLI/GUI as they are in a hiddenstate.

Deletion of a vol-ume is not complet-ed due to the pres-ence of stale ACLsassociated with it

System Manage-ment

AS-105432

Not applicableIn rare cases the Data Servicemay restart due to a systemhealth check failure caused bylocked index processing threads.The service will recover after therestart.


System InternalsAS-106168



Known Issues

Known Issues in NimbleOS version 5.3.0.0


Contact HPE Nimble StorageSupport to increase the healthcheck timeout on the array

After updating the array toNimbleOS 4.x.x.x or later, a pro-cess runs to convert an on-diskdata-structure specific to Nim-bleOS by initiating an operationthat walks the existing tree ofthis data-structure. In somecases, this operation keeps run-ning even after the conversionhas completed successfully,leading to health check timeout.This results in a Data Servicerestart.

Data Servicerestart while con-verting internal da-ta-structures inNimbleOS


If RAID is degraded for an ex-tended period and Data Servicerestarts occur, contact HPENimble Storage to assess adjust-ing allocated buffer resources.

When RAID is degraded, IOneeds to be reconstructed byreading from multiple disks, andan internal buffer may exhaustits allocated resources. In rarecases when multiple disks aredegraded, the Data Service mayrestart unexpectedly.

Data Service mayrestart unexpected-ly when RAID is indegraded mode


Reducing the load on the arrayis the only known workaroundfor this issue.

On CS2XX or CS3XX, the arraymay run out of data pages whenunder a heavy load. This will re-sult in an unexpected Data Ser-vice restart.

Data Service mayrestart unexpected-ly for CS2XX andCS3XX arrays whenunder a heavy load


Not applicableIn rare instances, the Data Ser-vice may restart when internaldatabase communication be-tween services is not available.The service restarts to restorethe communication betweenservices.

Data Service mayrestart unexpected-ly due to internaldatabase communi-cation


Not applicableA lost wake-up to an operationmay cause the operation block-ing the checkpoint which resultsin Data Service restart.



Not applicableThis can occur when the memo-ry footprint used by internal in-dexes exhaust the default alloca-tion of scratch pages in the sys-tem after several volumes havebeen deleted.

Data Service mayrestart when arrayruns out of scratchpages



NimbleOS 5.3.0.0 Known Issues



Data service will be available af-ter restart.

During NimbleOS metadatasync, in rare instances, the dataservice may restart unexpected-ly. The metadata sync operationitself wont be affected and therestart will reset the race condi-tion; the data service will stabi-lize after the restart.

Data ServiceRestart due to arace condition dur-ing metadata sync


Not applicableAs part of the file systemcheckpoint process, internal in-dexes are required to merge todisk. The health check processuses a heartbeat mechanism todetermine if the merge is mak-ing progress. Under some cir-cumstances, the process per-forming the merge of a batch ofupdates to the index fails toheartbeat within a deadline.When that happens, the DataService may restart with ahealth check failure.

Data Service mayrestart unexpected-ly with healthcheck failure wheninternal index datastructures take toolong to merge


Not applicableUnder certain conditions, theData Service may restart whena large number of internal index-es merge within a short time-span and when some of theseindexes need to merge multipletimes in the same checkpoint. Inthis case, there can be lock con-tention between operations re-sponsible for picking what tomerge and operations process-ing the merge, leading to healthcheck timeout.

Data Service mayrestart unexpected-ly with VM healthcheck failure due tolock contention.


The array group will recover it-self following the Data Servicerestart.

The Data Service may restartunexpectedly on an array groupwith Storage Class Memory andSynchronous Replication config-ured, during an automaticfailover (AFO) event. This oc-curs when invalidating indexdata in memory exceeds thetimeout of 30 seconds.

Data Service mayrestart duringmetadata invalida-tion






Not applicableIn one of the code paths for theprocess that handles fingerprintmapping, the merge status isnot properly set. This may leadto an unexpected Data Servicerestart.

Data Servicerestarts when oneof the processes in-correctly sets itsreturn status.


Not Applicable, the Data Servicewill resume normal operationafter restart.

Generation delete loads a largenumber of ondisk metadatablocks which may prevent blockindex operation checkpoint fromfinishing. This causes the vol-ume manager health check tofail which results in Data Servicerestart.

Data Service mayrestart due to vol-ume managerhealth check failureduring generationdeletion


Not applicableWhen a read op finds partialdata in-core, it issues a mediaread to get remaining data. Bythe time, media read returns, thein-core data is synced, and treeis reopened for deletion. Theread does not expect tree to bein delete state and causes theData Service to restart.

Data Service mayrestart unexpected-ly due to race condi-tion


Contact Nimble Storage Sup-port.

When flash cache Garbage Col-lection copies forward live dataof a fragmented segment, itcould consume more buffersthan provisioned and cause theData Service to restart to recov-er.

Data Service mayrestart when run-ning out of buffers


Not applicable.Due to issues in communicatingwith the partner array duringreplication, there are few caseswhere the operation is not ableto make progress. As a result,the file system may restart tocorrect this condition.

File system restartto recover fromstalled replication


Not applicableWhile committing internaltransactions, Data Service mayhit a rare race condition. To re-cover from this Data Servicemay restart.

Data service mayrestart due to arace condition


Not applicableIf IO to disk are slow or fail, theData Service may restart to tryto recover the condition.

Data Servicerestart due to slowdisk IO or disk IOfailure






Not applicableIn rare cases, Data Service mayrestart during snapshot replica-tion due to the failure of check-sum algorithms to detect allnetwork errors.

Data Service mayrestart due to net-work errors


Not applicableDue to a race condition, the Da-ta Service may restart during agraceful shutdown causing unex-pected Data Services restartmessages to be generated. Thisshould not cause any I/O impactbecause the Data Service is al-ready in the process of shuttingdown.

Data Service canrestart unexpected-ly during shutdownprocess


Enabling encryption will resolvethe issue

Disabling encryption mightcause NVRAM data to fill upwith data waiting for masterpassphrase, which can causeSynchronous Replicated vol-umes to go out of sync.

Disabling encryp-tion may causeSynchronous Repli-cated volumes toremain out of sync



A network issue between up-stream and downstream couldabruptly stop operations run-ning downstream causing themto exit prematurely, resulting inData Service restart.

Data Service mayrestart if networkissue is encoun-tered betweenGroup Leader andBackup GroupLeader arrays


Remove downstream replica us-ing the correct steps orderedbelow. 1. Claim thereplica volume. 2. Delete all ofthe snapshots for the replicavolume. 3. Delete the replicavolume.

The service may restart whenremoving the downstreamreplica using the steps below. 1. Deletion of all snap-shots for the replica volume. 2.Claim the replica volume 3.Delete the replica volume

Very rare race be-tween Vol claim(with all snapshotsmarked for dele-tion) and space re-calculation onreplica downstreamvolume


The array will continue to shut-down after the Data servicerestart.

Volume manager does not resetinternal callbacks during theshutdown phase causing theData service to restart.

Data service mayrestart during arrayshutdown


Not applicableThe volume manager willdecrement bin and child volumefolder entries during volumedeletion. In rare occurrences,another thread may decrementthe child volume entry and notthe bin entry, causing the Dataservice to restart.

Data Service mayrestart on down-stream array due torace condition dur-ing volume deletion






Retry operation after a fewminutes to reassign array topool.

Assigning an array to a pool im-mediately after unassigning itfrom the same pool will fail withthe following error - Failed toassign arrays to the pool: A ser-vice is not running or is notreachable

Unassigning andreassigning arrayto a pool within 5minutes will fail


Not applicableAggressive volume creation anddeletion may cause one of thetree data structures to hit maxi-mum children it can have. Thislimit can occur on volume cre-ation because volume deletiondestroys the children asyn-chronously. The Data Servicewill restart when this limit hasbeen reached.

Aggressive Volumecreation and dele-tion may result inData Servicerestart


Contact HPE Nimble StorageSupport to increase the memorylimit for the service.

In some instances the cimserverservice will exceed its memorylimit and no longer be able tostart.

Cimserver stopswhen memory limitis exceeded.


Contact Nimble Storage Supportto identify the reason for thepartition filling up and to cleanup the log partition.

Log files generated by process-es in the Nimble Operating sys-tems are rotated and archivedusing certain parameters. Undersome conditions, if processeslog more than normal, this canresult in the log partitions beingfilled up, leading to other issueswith critical processes.

Log partition mayfill up

PlatformAS-50033

1. Reseat one of the power sup-plies to clear the hang condition.2. Or update NimbleOS to ver-sion 3.6.0.0 or later to minimizethe number of internal com-mands that can trigger this is-sue.

Under certain circumstances,internal commands may causethe integrated circuit to hangwhich causes both power sup-plies of the head shelf to appearas missing. The problem doesnot impact the power supplysability to deliver power to thearray.

Both power sup-plies showing up asmissing

PlatformAS-53621


In rare incidents, controllers donot power on following powercycle.

Controller does notpower on followinga power cycle.

PlatformAS-100088

Referencing the correspondingExpansion Shelf Quick StartGuide, correct the hardware ca-bling issue.

If a shelf is cabled incorrectlyand added to an array, the rawcapacity calculation could failleading to multiple restarts ofthe Data Service.

Incorrect shelf ca-bling can cause aseries of unexpect-ed Data Servicerestarts

PlatformAS-67242





The controller reboot shouldrestore SAS HBA to normalstate. HPE Nimble Support maycontact customer to collect addi-tional diagnostics if required.

When the SAS HBA detectsfaulty states, to recover, the ar-ray needs to reset the SASHBA’s firmware. The SAS HBAfirmware reset can block diskI/Os significantly longer thanour High Availability monitoringtimeouts allow. Instead, a con-troller reboot is triggered imme-diately if this state is detected,resulting in an unexpectedtakeover event.

Unexpected con-troller takeover dueto incorrect state ofthe SAS HBA

PlatformAS-33725

Not applicableThe Enclosure Managementservice may unexpectedlyrestart due to a race conditionwhich is resulting from drivestatus which has already beenmarked removed by the DataService. The restart of the ser-vice clears the condition.

Enclosure Manage-ment service mayunexpectedlyrestart due to drivestatus race condi-tion

PlatformAS-95294

Not applicableThe Data Service may restartunexpectedly when it detectsan internal check error. Therestart of the service clears thecondition.

Data Service mayunexpectedlyrestart

PlatformAS-90850

Not applicableThe Data Service may restartdue to a transient resource allo-cation failure. This happenswhen the service cannot com-plete a disk IO due to transientmemory allocation failure. Thisdoes not cause a service outageas Data service continues nor-mally after a restart.

Data Servicerestart due to re-source allocationfailure.

PlatformAS-103802

The restart of the service willclear the low memory condition,no further action is needed.

Data Service may restart in therare case when a controller islow on memory.

Data Service mayrestart if a con-troller is low onmemory

PlatformAS-99567

Contact HPE Nimble StorageSupport to identify the faultydrive for replacement.

Data Service may become latentand restart when the array hasa faulty drive with a lot of medi-um read and IO timeout errors.

Data Service mayrestart if the arrayhas an faulty drivewith a high numberof medium read er-rors

PlatformAS-110036





Not applicableOn very rare occasions of HBAerror handling, interrupts couldbe turned off long enough forthe NVRAM driver to timeoutwhile trying to complete directmemory access operations. Thecontroller recovers after a re-boot is initiated by the kernel.

Kernel panics whileprocessing hostbus adapter errors

PlatformAS-46629

No workaround is required. Thearray will recover itself automat-ically by restarting the DataService.

On rare occasions, the arraygroups Data Service may fail toinitialize if a shelf state changeoccurs simultaneously.

Delay with DataService startingduring shelf statechange

PlatformAS-101570

Add the disk from the CLI usingthe disk add command and out-put from disk list: 1. Run disk --list 2. Note the slot number, andshelf location for the disk la-beled foreign. 3. Add the disk:disk --add <slot number> --array <arrayname> --shelf_location <shelf loca-tion> Note: the --force optionmay be required Contact HPENimble Storage Support if thedisk does not move to resynchro-nizing state after completing thecommands.

Disk will report foreign for diskstate in the GUI Hardware pageor in the output of disk --listfrom the CLI. This typically oc-curs if diagnostic data may nothave been removed after test-ing.

Replacement diskreports foreign fordisk state inGUI/CLI

PlatformAS-99428

Review the array configurationmatrix for the array model: <ahref=https://infos-ight.hpe.com/InfoSight/media/lo-cal/active/34/CSxxx%20Con-fig%20Matrix.pdf>https://infos-ight.hpe.com/InfoSight/media/lo-cal/active/34/CSxxx%20Con-fig%20Matrix.pdf</a> Removeany additional cache from theexpansion shelf that exceedsthe max cache limit based onarray model.

ES2 and AFS2 expansionshelves contain additional slotsfor upgrading cache capacity ofthe array. Older array modelshave a maximum cache limitthat can be handled by the ar-ray. If ES2 or AFS2 expansionshelves are added to an arrayand the cache exceeds the maxcache limit for the array type,the data service may restart dueto running out of data pages.

Data service mayrestart if maximumcache exceeded forCS215, CS235,CS300, CS500,CS700 arrays

PlatformAS-93296

Not applicableThe Enclosure Managementservice may unexpectedlyrestart when it detects an inter-nal check error. The servicerestart clears the condition.

Enclosure Manage-ment Service mayrestart unexpected-ly

PlatformAS-105053





Not applicableIn rare cases, an SSD may reachits endurance limit but contin-ues to pass Nimble Drive ErrorRecovery algorithm. This causesa never-ending process of off-lining and on-lining the drive.This may occur for Intel andToshiba SSDs.

SSD has reached itsendurance limit(wear leveling) butthe disk is notmarked failed.

PlatformAS-91522

Not applicableThe Group Management Servicemay restart unexpectedly dueto receive buffer exhaustion onthe management network inter-face. No visible impact has beenreported because of this issue.

Unexpected GroupManagement Ser-vice restart due toreceive buffer ex-haustion

PlatformAS-98885

Please contact HPE NimbleStorage Support to resolve thisissue. This issue is resolved inNimbleOS version 5.0.2.0 andlater.

An attempt to create new VMsin a VMWare vVol environmentfails after updating to NimbleOS4.x.x.x. This occurs when a sys-tem partition within the array isrunning out of space.

VM creation fails inVMWare VVol envi-ronment after up-date to NimbleOS4.x.x.x

PlatformAS-76174

Free up /var disk space bydeleting unwanted log files(/var/log/EMU/mili/mili2d.log)and rebooting the host.

As part of the dHCI updateworkflow which involves theESXi server update, some com-mands need to be run on theserver. Running these com-mands fails with error Error: Ageneral system error occurred:Internal error. if the /var is full.This is a known issue with 6.7builds running with Emulexdriver which has been fixed aspart of ESX 6.7 U2. <ahref=https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vsphere-esxi-67u2-release-notes.html#re-solvedis-sues>https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vsphere-esxi-67u2-release-notes.html#re-solvedissues</a>

dHCI update work-flow can fail if /varmountpoint on thehost is full.

dHCIAS-103315

The only way to resume the up-date would be to failover to theoriginal group leader array andthen resuming the updatethrough the dHCI vCenter plug-in.

When the dHCI unified updatefeature is used to update thedHCI stack, the update will failif a Group Leader Failover oc-curs during the process.

An in-progess dHCIupdate fails if thegroup leader failsover

dHCIAS-103247





After confirming the array hasreturned to Active/Standby sta-tus, stop the hung task and runthe Nimble Add ProStack servertask again.

If an array failover occurs whileNimble Add ProStack servertask is running from vCenter,the process may hang and notcomplete.

Nimble AddProStack servertask may hang ifarray failover oc-curs

dHCIAS-97685

Not applicableWhen an ESXi server is addedto dHCI cluster, the update pagedoes not get updated to includethe newly added ESXi server.This refresh happens every 4hours. After the next refresh, thenew ESXi servers version will beincluded and accounted for onthe update page.

The update pageon the dHCI plugintakes 4 hours to re-fresh

dHCIAS-103769

A valid ESXi license must be as-signed to the server.

When adding a server with anexpired ESXi license to the dHCIcluster, through the vCenterplugin, you may see an errorsaying - Failed to submit a taskto add server.

Addition of a serv-er with expired ES-Xi license fails

dHCIAS-95054

Admission Control should bedisabled on a dHCI cluster forthe update to proceed.

For ESXi server update, DRS isused to migrate VMs running onthe server. If admission controlis enabled on the dHCI cluster,DRS is not able to migrate VMsoff a server.

dHCI update failswhen AdmissionControl is enabled

dHCIAS-101915

Not applicableThe Data service will wait up to300 seconds for host responseto iSCSI commands. If host doesnot abort commands that takemore than 300 seconds to com-plete, the Data service willrestart.

Data Service mayrestart when hostdoes not issueabort to timed outcommand

SANAS-108946

Not applicable.During a controller reboot, dueto resource contention betweennew Fibre Channel (FC) connec-tion attempts and shutdown ofthe FC module, the Data Serviceon the array may restart unex-pectedly.

Data Servicerestart duringshutting down FCservice on thestandby controller

SANAS-100197





Identify the offline volumes orstale targets and initiate a hostside cleanup/rescan to avoid re-peated login attempts and fail-ure to the offline or stale tar-gets.

Logins to CHAP authenticationenabled volumes could leak asmall amount of memory by re-peated failed login attempts tooffline volumes or stale targets.Over a period of days andweeks, this leak can result in oneor more of the NimbleOS pro-cesses running out of memory.As a result, NimbleOS servicesmay restart unexpectedly.

NimbleOS servicesmay restart unex-pectedly due tomemory leak in lo-gin path

SANAS-107345

Identify the offline volumes orstale targets and initiate a hostside cleanup/rescan to avoid re-peated login attempts and fail-ure to the offline or stale tar-gets.

Logins to CHAP authenticationenabled volumes could leak asmall amount of memory thesize of CHAP username. Over aperiod of days and weeks, thisleak can result in one or more ofthe NimbleOS processes runningout of memory. As a result,NimbleOS services may restartunexpectedly. The issue is exac-erbated by repeated failed loginattempts to offline volumes orstale targets.

NimbleOS servicesmay restart unex-pectedly due toslow leak withCHAP logins

SANAS-109412

Not applicableWhen the Data Service is inprocess of shutdown, the servicemay restart due to race condi-tion when shutdown threads donot wait or abort pending oper-ation threads.

Data Service mayrestart due to racecondition

SANAS-64790

Not applicableWhen the active controller isbeing shutdown, the Data Ser-vice runs into an internal errorcondition that causes the ser-vice to restart unexpectedly.Since the process is already be-ing shutdown, there is no impactto user data availability.

The Data Servicerestarts unexpect-edly during shut-down

SANAS-98042

Not applicableUnder certain conditions, theData Service on the GroupLeader array may restart unex-pectedly while removing mem-ber array. This is due to a racecondition when processing SCSIRTPG (REPORT TARGET PORTGROUPS) commands. The ser-vice should stabilize on its ownshortly following the restart.

Data Service mayrestart unexpected-ly while removingmember array

SANAS-101325





Not applicableThe Data Service or SCSI HighAvailability Service may restartdue to race condition encoun-tered during process shutdown.

Service may restartdue to race condi-tion

SANAS-89753

Not applicableA race condition may occurwhile updating records in theScale Out database when twothreads attempt to update thesame record.

Group Manage-ment service mayrestart due to racecondition

System Manage-ment

AS-110367

Not applicableGroup Management Service mayrestart unexpectedly as onethread has taken a ReadWritelock which has another writerthread, which is waiting forScale-Out Database (SODB)transaction to be completed.The service restarts due to theSODB transaction exceeding theexpected timeout.

Group Manage-ment Service mayrestart due to racecondition

System Manage-ment

AS-103766

Not applicableWhen created with the RESTAPI, by default the cloned vol-ume inherits the ACL from theparent volume. There is no wayto set or change the ACL for theclone at the time of creationthrough the REST API. Clientsshould make two REST calls, oneto create the clone and then oneto change the ACL to the de-sired value. In the future, wecould change the REST API tonot set the ACL by default andallow the client to set the ACLat the time of the clone creationto the desired value. This onlyapplies to volumes that haveagent_type set to none

REST API Clone ofvolume does notsupport settingACL at time ofclone creation

System Manage-ment

AS-54302

Not applicableGroup Management service mayrestart due to Volume Manage-ment thread and API thread forsnapshot creation/deletion en-tered race condition. The servicerecovers after restart.

Group Manage-ment Servicerestart due to racecondition

System Manage-ment

AS-105929





Not applicableDue to the multi-threaded na-ture of the Alerts and AlarmsService, it is possible for anevent with a later id to be post-ed prior to an event with anearlier id. Therefore it will havean earlier timestamp eventhough its ID is higher. This cancause the IDs to appear out oforder. However, the alarms inthe list are ordered correctly bytimestamp.

Alarm IDs in alarmlist may appear outof order.

System Manage-ment

AS-105431

Run the following command viathe array CLI: alarm --deletealarm_id

Alarms raised by a member ar-ray are visible when issuingalarm --list even after the mem-ber array is removed from thegroup.

Member arrayalarms are still visi-ble after array isremoved fromgroup

System Manage-ment

AS-106124

The problematic alarm can bemanually deleted through theCLI command: alarm --delete<alarm_id>

Due to a race condition it ispossible for an alarm to remainuncleared on the system evenwhen the alarm condition it isreporting is no longer the case.This can happen if the onsetalert and recovery alert weregenerated at close to the sametime.

Alarm may notclear as expected

System Manage-ment

AS-94398

Not applicableAPI calls occur between theEvent service and Group Man-agement service when an alarmis cleared. The calls will lookupthe alarm ID and onset event inthe Scale Out Database (SODB).If the alarm is cleared before theonset event lookup completesand is not found during thesecalls, the service may restartunexpectedly.

Event service mayrestart unexpected-ly

System Manage-ment

AS-83604

Not applicableAlerts and Alarms processingservice may restart unexpected-ly when certain operation sur-pass the designated healthcheck timeout. The process willstabilize following the restart.

Alerts and Alarmsprocessing servicemay restart unex-pectedly

System Manage-ment

AS-50821





The recipient can create an in-bound transport rule to appenda disclaimer to the messagesfrom the problematic sender.The disclaimer will append theexpected CR-LF combination tothe message so that it can bedelivered. (This disclaimer mayconsist of a single charactersuch as a period or a dash.)

When the array sends an alertvia SMTP, the message may berejected by the server indicating550 5.6.11 SMTPSEND.Bare-LinefeedsAreIllegal. This canoccur if the receiving serverdoes not support BDAT com-mand for SMTP chunking

Alerts sent viaSMTP may be re-jected

System Manage-ment

AS-72902

Not applicableThe Event service may restartunexpectedly due to a memoryaccess issue. The restart will notimpact data connections to thearray and the process will recov-er after the restart.

The Event servicemay restart unex-pectedly

System Manage-ment

AS-85608

Not applicableThe Eventd process may restartunexpectedly due to a raredeadlock condition between itsthreads. The restart of the pro-cess will clear the condition.

Eventd processmay restart unex-pectedly

System Manage-ment

AS-103567

Not applicableThe Eventd process may infre-quently crash, due to a bug inan external library used by theDNS resolver. The restart willnot impact data connections tothe array and the process willrecover after the restart.

Eventd processmay restart inter-mittently

System Manage-ment

AS-46024

Correct the DNS or SMTP config-uration to a valid address byensuring that a ping to the de-fined address succeeds.

We create multiple threads todeliver emails, but we use a non-threadsafe libcurl call to dis-patch them. Therefore, the lockneeds to be around libcurl call.If there is a misconfigured DNSor SMTP server, the curl call willtimeout. If there are greaterthan 7 emails waiting to be de-livered and all are suffering atimeout, we will starve thehealth checking for more thanthe 300-second health checktimeout causing the EventManagement service to restart.

Flood of timeoutscausing EventManagement ser-vice restart

System Manage-ment

AS-68651





The Group Management servicewill recover after the crash. Theworkaround would be to nolonger use an excessively longusername, and reduce the user-name to under 255 characters.

An excessively long usernameof over 255 characters will ex-ceed a character limit within thearray groups auditing frame-work. This would cause theGroup Management service torestart unexpectedly.

Group Manage-ment Servicerestarts due to ex-cessively long user-name

System Manage-ment

AS-82919

Not applicableEvery night, when inactive keysfor deleted encrypted volumeare deleted by the array, it cre-ates an audit log entry with rootas the user performing the ac-tion.

Purge inactive en-crypted keys ap-pears in audit log.

System Manage-ment

AS-96966

Not applicableIf a user tries to create a newuser account, but the userdoesnt have the privilege to doso, the user creation will fail.However, an audit log entry isnot created.

No Audit Log entryis created if userdoes not have theprivilege to createuser

System Manage-ment

AS-71090

Not applicableThe Group Management Serviceon the array may restart unex-pectedly when a SOAP timeoutis encountered after trying tocreate a volume. This occurs dueto a race condition where thearray attempts to delete thevolume after the creation at-tempt fails.

Group Manage-ment Servicerestarts followingvolume creationtimeout

System Manage-ment

AS-104640

Not applicableConnection to the Scale-OutDatabase can not be reinitializedwhen a transaction is inprogress. The Group Manage-ment Service will restart to re-store connection.

Group Manage-ment service mayrestart when con-nections to Scale-Out Database ex-ceeds thresholdvalue

System Manage-ment

AS-109127

Perform the Autosupport valida-tion process again.

When autosupport configurationvalidation is performed, internalprocess tracking may abortcausing the Group ManagementService to restart.

Group Manage-ment service mayrestart when per-forming Autosup-port Validation

System Manage-ment

AS-104185





Not applicableDuring key value metadatahandling, the process maydelete primary keys in one tableof the Scale-Out Database(SODB) and then try to refer-ence the deleted primary key asforeign key in another table inSODB. This causes foreign keyviolations in Postgres andeventually leads the GroupManagement service to restart.

Group Manage-ment service mayrestart due to keyvalue metadatahandling

System Manage-ment

AS-108146

Not applicableDuring key value metadatahandling, the process maydelete primary keys in one tableof the Scale-Out Database(SODB) and then try to refer-ence the deleted primary key asforeign key in another table inSODB. This causes foreign keyviolations in Postgres andeventually leads the GroupManagement service to restart.

Group Manage-ment service mayrestart due tosnapshot key valuemetadata handling

System Manage-ment

AS-105454

Not applicableDuring the startup process forGroup Management, key valuemetadata is loaded from theScale Out Database (SODB) intomemory. The Group Manage-ment Service may restart due toexceeding the expected time tocomplete startup if there is alarge amount of metadata toload.

Group Manage-ment service mayrestart duringstartup

System Manage-ment

AS-99300

Not applicableIf there is a network connectivityissue between the witness andGroup Leader array, the groupstatus CLI output will updatethe Failover Mode from Automat-ic to Manual until the connectionis reestablished. It also displaysthe Witness Status as N/A asopposed to Unreachable.

group --status CLIoutput shows incor-rect Failover Modeduring networkconnectivity issues

System Manage-ment

AS-99704





Not applicableDuring software update of arraygroup with Peer Persistenceconfiguration, AutomaticSwitchover (ASO) is disabled. Ifan array goes down (both con-trollers down) during the soft-ware update process, due to apower failure or other unexpect-ed event, hosts could lose ac-cess to data until the failed ar-ray recovers, or a manualswitchover of the affected vol-umes is done.

Peer PersistenceAutomaticSwitchover (ASO)is disabled duringsoftware update

System Manage-ment

AS-94649

This restart is non-disruptive tothe data on the array, and theArray Management recovers af-ter the restart occurs.

The arrays database systemmay become unavailable for alimited time when there is a fail-ure in setting up the BackupGroup Leader. When attemptingto discover a new Backup GroupLeader, the Array ManagementService may restart due to arace condition.

Array ManagementService restartsduring BackupGroup Leader dis-covery

System Manage-ment

AS-98953

No workaround is needed. Theservice recovers on its own.

In Automatic Switchover environ-ments, in rare instances, net-work isolation of the GroupLeader and Backup GroupLeader may cause the servicethat handles the automaticswitchovers to restart unexpect-edly.

Network isolationof the Group Lead-er and BackupGroup Leader arraymay lead to Auto-matic Switchoverservice restarts

System Manage-ment

AS-94683

Not applicableThe array group CLI commandwith limits option (group --list-limits) displays numeric internalidentifiers as part of the informa-tion listed for the volume infor-mation. These numeric identi-fiers are used by the array onlyand can be ignored.

Group limits com-mand lists internalidentifiers

System Manage-ment

AS-101342

Please contact HPE NimbleSupport.

Following a power outage, it ispossible that the Backup GroupLeader is not assigned to thegroup. This may occur if theSODB database does not startdue to an SSH key issue.

Backup GroupLeader is not as-signed due to pow-er outage

System Manage-ment

AS-99702





Not applicableThe Array Management servicemay restart unexpectedly dueto an a memory allocation fail-ure when attempting to synchro-nize configuration with a mem-ber array. The restart of the Ar-ray Management service clearsthe situation.

Array Managementservice restarts dueto memory alloca-tion issue

System Manage-ment

AS-104812

Not applicableThe Group Management servicemay restart while collectingstatistics from member array.This can occur when the requestfrom the group leader to mem-ber array exceeds timeout,causing the service to restart torecover.

Group Manage-ment service mayrestart while collect-ing member arraystatistics

System Manage-ment

AS-93469

If array is queried for statistics,then its frequency should be re-duced. This is specially relevantfor the setup where script orsome monitoring tool does statsquery frequently in a loop.

There is a memory leak in thestat (statistics) component ofthe Array Management service.Magnitude of the memory leakis proportional to the frequencyof stats query.

Array Managementservice restart dueto memory alloca-tion exception

System Manage-ment

AS-98297

The workaround would be todelete the alarm using the alarm--delete CLI.

When alarm for number of nicsin array net config reaching 120is triggered, it doesnt getcleared even when the numberof nics goes down.

Max limit of 120nics in netconfigalarm does not getcleared

System Manage-ment

AS-87749

The only workaround is to im-prove the quality of the networkconnection so that the loss ofTCP packets is minimized.

Service threads within the groupmanagement API handler arerestricted to run for no morethan 5 minutes. In a very lossynetwork, TCP throughput canbe throttled to almost nothingdue to retransmission timeoutback-off. In one observed case,where the API response wasabout 190 KB, the connectionmanaged to send only about 90KB before the 5 minute timeoutoccurred and caused the GroupManagement service restart.

A lossy networkcauses Group Man-agement servicerestart

System Manage-ment

AS-73595


The Event service may restartunexpectedly if an invalid net-work interface entry is found inthe Scale Out Database (SODB).

Event Service mayrestart due to in-valid database en-try

System Manage-ment

AS-103275





Not applicableThere is a --force switch avail-able when deleting a perfor-mance policy via the HPE Nim-ble Storage Array CLI. This --force switch does not work andwill fail with the following: ER-ROR: Failed to delete perfor-mance policy. Resource busy.The --force command is notsupported since the specifiedperformance policy should notbe removed without first check-ing its volume or folder associa-tions.

Force deletion ofuser defined perfor-mance policyshould not be sup-ported

System Manage-ment

AS-74242

Not applicableVolume move operations copyboth data and associated meta-data from the source array todestination array. For groupswith multi-array pools, copyingthe metadata can take a signifi-cant amount of time, and theestimate calculation may be in-accurate initially. These esti-mates will auto-correct them-selves by using feedback mech-anisms.

Volume Migrationestimates may beinaccurate whenmultiple volumesare being migrated

System Manage-ment

AS-101832

Not applicableIf communication betweenGroup Management and Post-gres services does not completewithin expected timeout, theGroup Management service mayrestart. The restart will resumethe communication and try thetransaction again.

Group Manage-ment service mayrestart due to com-munication timeoutexceeded

System Manage-ment

AS-97327

Not applicableArray Management servicesmay be unavailable for a shorttime due to restart during auto-matic failover.

Array Managementprocess mayrestart during auto-matic failover

System Manage-ment

AS-94835

The service will stabilize on itsown following the restarting.

The Array Management servicemay restart unexpectedly whenthe array is under high work-load.

Array ManagementService restarts un-expectedly underhigh load

System Manage-ment

AS-101420

Not applicableGroup Management service mayrestart due to health checktimeout exceeded. The servicewill recover after the restart.

Group Manage-ment Service mayrestart unexpected-ly

System Manage-ment

AS-97697





Disable automatic failover andremove witness from configura-tion.

After Group migration is per-formed, backup group leaderbecomes group leader. If witnessis disconnected for more thanfive minutes, the new leader willnot remove quorum.

After group leadermigration, quorumis not removed af-ter witness discon-nection

System Manage-ment

AS-98434

Not applicableWhen the system has a highnumber of objects, the GroupManagement service mayrestart while running RESTquery or CLI command.

Group Manage-ment service mayrestart due to highmemory usage

System Manage-ment

AS-96241

Not applicableIf a user performs multiple vol-ume collection handovers be-tween two arrays during a shorttime span, this may cause a situ-ation where both upstream anddownstream array may claimvolume collection ownership.This is due to a race conditionin the workflow.

Both upstream anddownstream mayclaim the volumecollection owner-ship when exces-sive handovers areperformed

System Manage-ment

AS-99520

Update the downstream arrayto NimbleOS 3.4.x or later.

Encrypted and deduped vol-umes cannot be replicated from3.4.x and later to previous 3.xreleases because they do notsupport encryption and dedupeat the same time. Note that thisdoes not affect replication from3.4.x and later to 2.3.x wherethe volumes will not be dedupedon the downstream running2.3.x.

Replication ofdedupe and en-crypted volumeswill stall between3.4 and previous3.x releases

System Manage-ment

AS-57574

Not applicableIn the case where the down-stream array is reaching itssnapshot rate limit and the userperforms the volume collectionhandover, the handover will beabort if the limit is surpassed.An alert will be raised but thealert message may be missingthe reason for aborting han-dover.

Alert for abortedhandover does notspecify reason

System Manage-ment

AS-98650

Not applicableWhile syncing the configchanges on the downstream ar-ray, group management servicemay restart due to a race condi-tion between protection policydeletion and adding the volumeto the protection policy.

Group manage-ment service mayrestart due to racecondition

System Manage-ment

AS-106490





Please reissue the command. Ifthe operation was already per-formed by the earlier command,an appropriate message will bereturned.

Under system busy conditions,when an excessive amount ofoperations are being issued inparallel or too many internal re-tries are occurring to performtasks, you may receive a Nomessage received error after is-suing a CLI command.

Error No messagereceived after issu-ing CLI commandto disassociate vol-ume from collection

System Manage-ment

AS-98378


Arrays with volumes that havelarge branch structures maycause internal command process-ing timeout to be exceeded. Thiswill cause the Group Manage-ment service to restart due tohealth check failure.

Group manage-ment service mayrestart unexpected-ly

System Manage-ment

AS-98155

Please reissue the command. Ifthe operation was already per-formed by the earlier command,an appropriate message will bereturned.

Under system busy conditions,when an excessive amount ofoperations are being issued inparallel or too many internal re-tries are occurring to performtasks, you may receive a Nomessage received error after is-suing a CLI command.

Error No messagereceived after issu-ing CLI commandto associate vol-ume to volume col-lection

System Manage-ment

AS-90633

Update the downstream poolsDeduplication setting to matchthe upstream pool.

If the Default Deduplication set-ting differs for upstream anddownstream pools, the configu-ration of deduplication volumesfor replication might fail with thefollowing error Deduplicationnot allowed since no applicationcategory is assigned to the per-formance policy

Configuration ofdeduplication vol-umes for syncreplication mightfail

System Manage-ment

AS-90649

vol --list can be used to deter-mine pool/folder attributes ofthese volumes.

For the volcoll --info output forsync replication volume collec-tions, the Associated volumes:and Associated pinned volumes:fields lack pool/folder qualifica-tion for the associated volumes.

volcoll --info out-put lacks pool/fold-er qualifications forassociated volumes

System Manage-ment

AS-90286

Not applicableThe group --list_limits CLI com-mand does not list the Syn-chronous Replication volumecount Limit. Synchronous Repli-cation on 5.1.0.0 and later canprotect up to 128 volumes.

Synchronous Repli-cation VolumeCount Limit

System Manage-ment

AS-89124





Not applicableDuring a planned array shut-down, the Group Managementservice may restart due a racecondition. There is no data ormanagement interruption sincethe service is already shuttingdown.

Group Manage-ment servicerestarts during thearray shutdown

System Manage-ment

AS-108765

Retry the command for thefailed volumes.

Due to a rare race condition, theGroup Management Service mayrestart unexpectedly during abulk volume update operation.

Group Manage-ment Servicerestarts duringbulk volume up-date

System Manage-ment

AS-95610

Confirm all Domain Controllersin the Active Directory environ-ment that is integrated with thearray are reachable.

In environments with an arrayintegrated with Active Directory,the create/validate session codeholds a lock on an underlyingsessions table and then goes toActive Directory to collect moreinformation about the user. Ifthis operation takes too long,the Group Management servicemay timeout to free this lock.

Group Manage-ment service mayrestart while collect-ing user informa-tion from ActiveDirectory

System Manage-ment

AS-108868

Please review the network andsee if there is a consistentpacket loss and fix any networkglitches. If you need any assis-tance, please reach out to HPENimble Storage Support.

If the network response to aREST request takes more than5 minutes, a thread performingthe REST request times out andas a result Group ManagementService restarts. The servicestabilizes itself and as long asthe network is serving the re-quests faster. A single instanceof the Group Management ser-vice restart should not causeany disruptions.

Group Manage-ment Servicerestarts due topacket loss in net-work

System Manage-ment

AS-91638

Ensure setup completes success-fully before attempting to dis-cover volumes.

If a user runs --resetup followedby setup on an array that waspreviously configured and setupdoes not complete successfully,data IPs may be unconfigureduntil the setup actually com-pletes successfully. As a result,GDD may restart unexpectedlyif any attempts are made to dis-cover the volumes on those un-configured data IPs.

Discovering vol-umes after arrayresetup may causeGroup Data servicerestart

System Manage-ment

AS-66182





Not applicableGroup Management service mayrestart due to memory exhaus-tion in configurations that ap-proach 10,000 volumes and300,000 snapshots.

Group Manage-ment service mayrestart due tomemory exhaus-tion

System Manage-ment

AS-94517

Please refer to the System Lim-its and Timeout Values sectionin the NimbleOS AdministrationGuide.

The group --list_limits CLI com-mand does not list system limi-tation for writable snapshots.Also, no alerts or alarms aregenerated as as the array groupapproaches the limit.

System limitationof writable snap-shots is not report-ed by NimbleOS

System Manage-ment

AS-68782

Not applicableIf communication betweenGroup Management and Post-gres services does not completewithin expected timeout, theGroup Management service mayrestart. The restart will resumethe communication and try thetransaction again.

Group Manage-ment service mayrestart due to com-munication timeoutexceeded

System Manage-ment

AS-97899

The TTL can be updated on thesnapshots which have a nega-tive value to a current value.The snapshot may also be re-moved if it has been confirmedit is no longer needed.

NimbleOS protects the lastreplicated collection, in somecases, the TTL expiry date onthose snapshots can becomenegative when the snapshotsexist beyond TTL.

Time to Live (TTL)expiry date on lastreplicated snap-shots can be nega-tive

System Manage-ment

AS-105944

Not applicableDue to a race condition, theGroup Management Service ona downstream group mayrestart while updating volumecollections from the upstreamgroup.

Group Manage-ment Service mayrestart due to arace condition

System Manage-ment

AS-105291

Not applicableIf clones are created using anunmanaged snapshot, then thisunmanaged snapshot will notbe deleted even if cleanup isenabled.

Unmanaged snap-shots remain aftercleanup is enabled

System Manage-ment

AS-93113

Not applicableThe Group Management per-forms queries on the informa-tion stored in the Scale OutDatabase (SODB) for the array.If the query does not completewithin the expected timeframe,the service may restart to recov-er from the issue.

Group Manage-ment service mayrestart due to inter-nal database time-out

System Manage-ment

AS-95868





Not applicableDuring planned service stop,such as array shutdown, theGroup Management service mayrestart due to a race condition.There is no data or managementinterruption since the service isalready shutting down.

Group Manage-ment service mayrestart during arrayshutdown

System Manage-ment

AS-104965

Not applicableIn rare instances, the GroupManagement Service mayrestart unexpectedly when theGroup Leader and Member ar-ray have lost connectivity dueto network outage. The servicerestart recovers GUI and CLI ac-cess, data services are not im-pacted by the restart.

Group Manage-ment Service mayrestart unexpected-ly due to networkconnectivity

System Manage-ment

AS-103982

Not applicableThe Array Management Servicerestarts unexpectedly followingautomatic Group LeaderFailover (AFO). The restart isnon-disruptive.

Array ManagementService restarts un-expectedly follow-ing automaticGroup LeaderFailover

System Manage-ment

AS-99615

Not applicableIn rare instances, the ProcessManagement service restartmay occur during software up-date. The system recovers afterthe restart of the service.

Process Manage-ment service mayrestart during soft-ware upgrade

System Manage-ment

AS-95132

Not applicableIf internal database processingfor array statistics exceeds theexpected timeout, the GroupManagement service will restartdue to health check failure torecover.

Group Manage-ment service mayrestart unexpected-ly

System Manage-ment

AS-98504

Scheduling snapshots to occurat different times instead of allat once may help alleviate thisissue.

Group Management service mayrestart on the array when thereis a high amount of snapshotactivity being performed. Theservice restart will recover fromthe condition and the snapshotoperations will resume.

Group Manage-ment service mayrestart unexpected-ly when performinghigh snapshot activ-ity

System Manage-ment

AS-105804


When the array is powered onand off excessively, servicesmay fail to start on the array.

Services may notstart on the arrayafter it is poweredon and off severaltimes

System Manage-ment

AS-101392





Not applicableThe Group Management servicemay restart when service com-munication for internal databaseprocessing is terminated. Theservice restarts to restore con-nections between the services.

Group Manage-ment service mayrestart unexpected-ly

System Manage-ment

AS-105453

No workaround available. Toavoid encountering this issue,reduce IO load when performingsoftware update. Software Up-date will succeed after one ormore System Management ser-vice restarts.

The underlying scale-outdatabase competes with CASLand other system processes forIOPS. During software update,a migration script runs againstthe database. Under heavy filesystem load, the migration stepsmay not complete within theexpected amount of time. As aresult, the migration may time-out leading to a restart of theGroup Data Service. After therestart, the migration shouldcomplete as normal without anyuser impact or intervention.

Group Data Servicemay restart whenthe array is underheavy load duringsoftware update

System Manage-ment

AS-110374

The alarms can be deletedmanually either in the GUI or onthe CLI.

After the update to 5.1.x.x orlater, the Snapshot limit warningalarm is no longer used. Thispresents a situation where stalealarms are present on the arrayand they will not be clearedeven if the space situation isrectified. The alarm follows thefollowing format: WARNINGMon DD YYYY HH:MM:SS Ac-knowledged - Volume <vol-ume name> snapshot spaceusage is over the configuredwarning limit.

Snapshot limitwarning alarmspersist after up-date to 5.1.x.x orlater

System Manage-ment

AS-98694

Confirm the usage level for vol-ume or pool indicated by thealarm is below threshold. Onceconfirmed usage level is belowthreshold, clear the alarm fromthe Command Line Interface(CLI): 1. List the alarms to findthe alert ID alarm --list 2. Deletethe alarm that is no longer validalarm --delete

Alarms are generated on the ar-ray when volume or pools ex-ceed thresholds. In some in-stances the recovery event toclear the alarm when the condi-tion is cleared does not clear thealarm.

Alarm not clearedafter volume orpool drops belowwarning threshold

System Manage-ment

AS-77045





Not applicableInfrequently the Group Manage-ment service may encounter anerror while shutting down. If theservice is being restarted it maytake a few more seconds tostart. GUI and CLI will be unavail-able for a few seconds.

Group Manage-ment service maybecome unavail-able when shutdown

System Manage-ment

AS-110123

Not applicableHigh concurrent REST volumereads with statistics may causeGroup Management service torestart. Current REST requestswill fail, GUI and CLI will be un-available. The system will recov-er after the restart of the ser-vice.

Group manage-ment service mayrestart when thereare many REST re-quests for volumestatistics values

System Manage-ment

AS-108432

The ASO checkbox is enabledby default, however ASO is notenabled until a witness has beenconfigured. In the GUI, navigateto Administration > Availabil-ity. If witness is configured andthe ASO check box is checked,disable ASO by unchecking thebox and clicking save. Performthe array software update again.If the update continues to failwith generic messaging, contactHPE Nimble Support.

Software updates to 5.1.4.200are not allowed when AutomaticSwitchover (ASO) is configured.If a software update to5.1.4.200 fails for this reason, ageneric software update failuremessage is returned in the GUI.The cause of the failure wouldneed to be determined by look-ing at the system configurationand determining if ASO is config-ured.

Arrays with Auto-matic Switchoverenabled fail soft-ware update withgeneric message

System Manage-ment

AS-106848

If a timeout occurs, the updatecan be completed using thesoftware --resume_update com-mand.

Due to the number of firmwareupdates involved, software up-date going from pre-3.3.x ver-sions to 3.3.x or later could taketoo long and eventually timeout.These update times may takelonger in larger multi-arraygroup configurations. Softwareupdate in smaller array groupsshould complete without inter-vention.

Updating arraygroups from earlierNimbleOS 3.x re-leases to 3.3.x orlater releases couldtimeout

System Manage-ment

AS-57173

Not applicableGroup management service mayrestart during software updatedue to race condition involvingunlocking the download lock file.

Group manage-ment service mayrestart during soft-ware update

System Manage-ment

AS-72559





This is an intermittent issue, soif the software update fails inthis manner it should pass if thesoftware update is resumed.

The timing is close enough thatit is possible for the individualarray precheck during softwareupdate to take long enough thatthe health check timeout is trig-gered, causing the group man-agement process to restart andthe software update to fail.

Health check time-out may causesoftware updatefailure

System Manage-ment

AS-66997

Initiate the software updateagain and answer the EULAprompt and the software updatewill proceed as expected.

When a software update is initi-ated on the array and theprompt for EULA acceptance isnot answered, the software up-date session will timeout afterseveral hours with a messageindicating Requires Authentica-tion or Contact HPE NimbleStorage Support.

Software updateinactivity timeoutmessaging

System Manage-ment

AS-54519

Running the software --re-sume_update command fromthe console will clear this condi-tion.

Under rare conditions, a soft-ware update may report an erroreven though the actual updatehas completed successfully. Thisoccurs when software updatetakes longer than 4 hours.

Timeouts duringsoftware update

System Manage-ment

AS-40516

Ignore the error message andretry the management operation

While the volume ownership isbeing changed, any manage-ment operation involving thatvolume could fail. This is expect-ed behavior. Depending on theprogress of that handover oper-ation, management operation isfailing with different errors.

Volume manage-ment operationsmay fail with unre-lated error mes-sages while han-dover is in-progress

System Manage-ment

AS-91962

Not applicableGroup Management Servicemight restart unexpectedlywhile creating Peer Persistentsnapshot during Daylight SavingTime adjustment window.

Group Manage-ment Service mayrestart unexpected-ly during creationof Peer PersistentSnapshot

System Manage-ment

AS-92209

Not applicableThe Group Management servicemay restart when internaldatabase processing exceedsthe expected timeout value. Theservice restarts to recover fromthe condition.

Group Manage-ment service mayrestart due tohealth check time-out

System Manage-ment

AS-109805





Not applicableWhen the HPE Nimble Storagearray is configured to use ActiveDirectory integration, the arrayis joined to one specific domain,as a domain member. Undernormal circumstances, users intrusted domains will also be ableto authenticate to the array. Ifone or more trusted domainsare joined to the forest using anMIT Kerberos type trust relation-ship, users and groups in anytrusted domain (e.g. not the do-main the array is joined to) willbe unable to authenticate to thearray.

HPE Nimble Stor-age array compati-bility issues withMIT Kerberos trusttypes

System Manage-ment

AS-95212

Not applicableActive Directory Authenticationcauses the arrays Group man-agement service to wait for aresponse from the Active Direc-tory. If this response is delayed,the Group Management Servicemay restart unexpectedly.

Active Directoryauthentication insome cases, maylead to a GroupManagement ser-vice restart

System Manage-ment

AS-104512


In rare instances, if the array isunder heavy load and GroupManagement service restartswhile new user is being created,the service can fail to start.

Group Manage-ment service maycrash while tryingto create userswhile array is underheavy load

System Manage-ment

AS-105035

Not applicableThe Group Management Servicemay restart unexpectedly if ittakes longer than 300 secondsto authenticate an Active Direc-tory (AD) user.

Group Manage-ment Servicerestarts while au-thenticating ADusers

System Manage-ment

AS-71137

Not applicableIf collection of user group infor-mation from Active Directorytakes longer than expected, theGroup Management servicetimeout may be exceeded. Theservice will restart to resume in-formation collection.

Group Manage-ment service mayrestart while collect-ing user informa-tion from ActiveDirectory

System Manage-ment

AS-69084

Not applicableCommand to join AD may failbecause of latency in gettingback a response from AD server.This may cause a health checkfailure for Group Managementcausing the restart of the ser-vice to recover.

Command to joinActive Directory(AD) may fail caus-ing Group Manage-ment serviceRestart

System Manage-ment

AS-66437





Not applicableWhen joining Active Directoryvia GUI or userauth commandin CLI, the command is expectedto complete within 300 seconds.If this timeout is exceeded dur-ing the process, the GroupManagement service will restartto recover.

Group Manage-ment service mayrestart when join-ing array to ActiveDirectory

System Manage-ment

AS-74830

Not applicableAD communication may be slowleading to a delay in responsesthat may cause Group Manage-ment service to crash. Leavingthe AD domain requires interac-tion with AD wherein this delaycan cause a restart.

Latency in commu-nicating with Ac-tive directory dur-ing may causeGroup Manage-ment service torestart

System Manage-ment

AS-78946

Resolve any connectivity issuesbetween the array and ActiveDirectory Domain Controller(s)to avoid unexpected servicerestarts.

If there are connectivity issuesbetween an array with ActiveDirectory integration enabledand an Active Directory DomainController, then the responsesmay take more than 300 sec-onds and timeout. As a result,the Group Management servicemay restart.

Active Directoryconnectivity issuemay result in GroupManagement ser-vice restart.

System Manage-ment

AS-65654

Not applicableWhen logging in with a Activedirectory (AD) user, delayed re-sponses from AD may lead toGroup Management servicerestart.

Group Manage-ment service mayrestart due to Ac-tive directory com-munication takingtoo long.

System Manage-ment

AS-109549

Delete unused clones or vol-umes to bring down the volumecount.

Following operations will failupon reaching the volume limit:-adding Synchronous replicationschedule to a volume collection -associating a volume toa volume collection with Syn-chronous Replication enabled -editing a Volume Collectionschedule to add SynchronousReplication partner

Enabling syn-chronous replica-tion fails uponreaching volumelimit

System Manage-ment

AS-102893





Once the array is able to deter-mine its deduplication capabili-ty, all newly created volumes willhave dedupe enabled, if speci-fied. In order to enable dedupeon the previously created vol-umes, you may run the followingcommand via the HPE NimbleStorage Array CLI: vol --edit<vol_name> --dedupe_en-abled yes

After a CSx000 array is in-stalled, it takes one minute forthe array to determine its dedu-plication capability. If a volumeis created prior to this, it will nothave dedupe enabled even if thearray is dedupe capable.

Unable to creatededupe enabledvolumes on a newinstall

System Manage-ment

AS-86545

The Group Management Servicewill eventually restart itself

Enabling and disabling thededupe setting on volumes andconcurrently deleting volumescan cause the Group Manage-ment Services to become tem-porarily unavailable on the ar-ray.

Group Manage-ment Service istemporarily unavail-able after deletingvolumes

System Manage-ment

AS-101535

Not applicableCurrently, there is no CLI sup-port for changing the witnessport. The nimble-witnessd.ser-vice file needs to be editedmanually.

No CLI support forchanging the Wit-ness Port

System Manage-ment

AS-92157

Not applicableA member array might not belisted under the Add Array toGroup option within the GUI ifthe member is configured witha different protocol (iSCSI vs Fi-bre Channel). Also when thereare multiple arrays in the sub-net, arrays which cant be discov-ered within the stipulated timemay not be listed in Add Arrayto Group.

Member arraymight not be dis-played under AddArray to Group op-tion

System Manage-ment

AS-100067

Not applicableIn rare instances, the Group Da-ta service may restart duringstartup due to a misconfigura-tion in NimbleOS pertaining toa file descriptor limit.

Group Data servicemay restart onstartup

System Manage-ment

AS-107015

Not applicableIn rare circumstances, followingan Automatic Failover (AFO) arace condition may cause theArray Management Service torestart or an unexpected con-troller takeover.

Array ManagementService restarts orTakeover occursunexpectedly fol-lowing automaticGroup LeaderFailover

System Manage-ment

AS-99431





Not applicableWithin the HPE Nimble Storagearray GUI, custom SSL certifi-cate import is only supportedon Google Chrome version 71or later.

Custom SSL certifi-cate import notsupported on olderversions of GoogleChrome

System Manage-ment

AS-99343

To keep the same reminder fre-quency and reset the next re-minder time based on the cur-rent time, change the reminderfrequency to a different value,save it, and change it back andsave it, or use CLI to make thechange.

When updating an alarm fromthe Events > Alarms page inGUI, selecting an alarm andclicking CHANGE REMINDERbutton, without changing thereminder frequency time, andclicking SAVE button, does notchange next reminder time. Thisbehavior is different from CLI.Setting alarm reminder frequen-cy to the same value from CLIresets the next reminder timebased on the current time.

Setting alarm re-minder frequencyto the same valuefrom GUI does notchange next re-minder time

System Manage-ment

AS-98177

When the controller is back up,all the information is displayedcorrectly on hardware page.

When a controller is down, theuser may see incorrect represen-tation of physical ports withinthe Hardware Page of the arrayGUI. This is due to the lack ofinformation from the missingcontroller.

Incorrect informa-tion on hardwarepage displayedwhen controller isdown

System Manage-ment

AS-87701

Use Google Chrome or MozillaFirefox browser.

The Array GUI incorrectly re-turns an error when a valid val-ue for the folder overdraft limithas been entered. This happensonly in Internet Explorer andMicrosoft Edge browsers.

GUI Error when en-tering a valid folderoverdraft limit val-ue

System Manage-ment

AS-102299

To work around this issue: 1.Log into the destination arrayto resolve the conflicts. 2. At-tempt the group merge again.

Currently within the HPE NimbleStorage Array GUI, when per-forming a group merge, if thereis a large amount of groupmerge conflicts (1000 or more),the GUI is unable to process andresolve all of them.

Group Merge viaGUI unable to pro-cess large amountof conflicts

System Manage-ment

AS-77372

Not applicableDuring group merge, the GUImight show Successful messageeven though the group mergebackend processing fails.

GUI may show Suc-cessful messagewhen group mergefails

System Manage-ment

AS-87886





Not applicablePool merge is not allowed ifSynchronous Replication is en-abled and pool merge is not al-lowed when a witness config-ured. If an array group has awitness configured for Automat-ic Switchover and has Syn-chronous Replication config-ured, when a user tries to per-form a pool merge, the followingerror is generated: “pool mergeis not allowed when witness isconfigured”. In this case, if theuser removes the witness andthen re-attempts the poolmerge, the following error isthen generated: “Pool merge isnot allowed when involved insync replication”. This errorshould supersede the previouserror.

Incorrect orderingof pool merge errormessages whenSynchronous Repli-cation and Witnessare configured

System Manage-ment

AS-95591

Edit the pool name and assign/ un-assign the array a in sepa-rate steps.

When attempting to edit a stor-age pool and assign an array atthe same time, you receive thefollowing error: Cannot updatearray list and name or descrip-tion simultaneously.

Unable to edit astorage pool andassign an array atthe same time

System Manage-ment

AS-94575

Fix the name conflict on thedownstream array.

When associating multiple vol-umes to a volume collectionfrom, the volume association forall volumes can fail due to aname conflict for one of the vol-umes on the downstream.

Volume collectionassociation for avolume can fail dueto a name conflicton the downstreamarray

System Manage-ment

AS-104099

If the page hangs or throws aweb server communication er-ror, try to clean the cache andrefresh. If the problem persists,restart the browser.

When the GUI runs for a longtime, the browser may en-counter an out of memory issue.

Browser throws aweb server commu-nication error

System Manage-ment

AS-48847

Run the following command viaCLI: snap --list --all --unman-aged

The Array GUI does not specifywhich snapshots are unman-aged and no longer belong to avolume collection.

Array GUI does notspecify which snap-shots are unman-aged

System Manage-ment

AS-93157





After a new custom certificatehas been imported or existingcertificate is deleted, pleaseclose the browser where the ac-tion was performed and reopena new one to guarantee a newconnection request to the Nim-bleOS web interface.

After changing a certificate, theGUI may present an error suchas follows: The web service isvery slow or unreachable...

Browser becomesunstable upon cer-tificate change

System Manage-ment

AS-99024

The values should report correct-ly within 24 hours after the up-date has completed.

The volume performance num-bers displayed in the GUI underManage > Data Storage >Volumes > Performance Tabmay display invalid values tem-porarily after an array softwareupdate.

Volume perfor-mance numbersmay report invalidvalues after soft-ware update

System Manage-ment

AS-92634

There is no workaround. Toavoid encountering this issue,reduce IO load when performingsoftware update.

The underlying scale-outdatabase competes with CASLand other system processes forIOPS. During software update,a migration script runs againstthe database. Under heavy filesystem load, the migration stepsmay not complete within theexpected amount of time. As aresult, the migration may time-out leading to a restart of theGroup Data Service. After therestart, the migration shouldeventually complete as normalwithout any user impact or inter-vention.

Group Data Servicemay restart whenthe array is underheavy load duringsoftware update


Not applicableWhen the Group Leader at-tempts to complete the BackupGroup Leader promotion, ifthere is not a healthy data path,the Backup Group Leader promo-tion fails. Despite, the networkerror, the Backup Group Leaderpromotion goes into a loop andultimately leads to an unexpect-ed restart of the Array Manage-ment Service.

Array ManagementService restartswhen Group Lead-er cannot reachBackup GroupLeader






Not applicableDue to a race condition, the Da-ta Service can crash during agraceful shutdown causing unex-pected Data Services restartmessages to be generated. Thisshould not cause any I/O impactbecause the Data Service is al-ready in the process of shuttingdown.

Data Service canrestart unexpected-ly during shutdownprocess


Please contact Nimble StorageSupport to increase the scratchpage pool size.

Under unique customer work-loads, the index footprint mayexhaust default scratch pageallocation in memory. As a re-sult, the Data Service mayrestart unexpectedly.

Data Service mayrestart when thearray runs out ofscratch pages


