Pushing the Envelope; .NET Code in SCOM Management Packs

Nathan Foreman@Nate_ForemanNathan@Practice2Perfect.com

Jonathan Almquist@jtalmquistjonathan@scomskills.com

#MMSMOA

@Nate_Foreman

Solutions Engineer

Practice2Perfect.com

10+ years in IT

Nathan Foreman

#MMSMOA

@jtalmquist

Consultant, blogger, author, speaker

TechNet Forum

Blogging since 2007

16 years experience in IT, past 10 years focused on Operations Manager

Jonathan Almquist

Overview and Agenda

Theory•Why .NET?• What’s .Net Modules• Pros (& Cons)• Functionality

• Microsoft Examples• Elastic Search DW

• Development Concepts• Threading• Batching• Persistence• Acknowledgments/

Notifications• Module Lifecycle

Implementation• Making the Library (.dll)• Create VS Project

• Signing the Builds• References

• Class Construction• Required Methods• SyncLock• Data & Control Flow

• Get the Module in SCOM• Create Library MP• Add Deployable Assembly• Create & Use Module

Why .NET?It’s not quicker or easier,so what can we gain by going this route?

What are .NET modules?

One of three Module Implementation [3,4]• Native• Managed• CompositeThey’ve been around for a while• SCOM 2007 [4]• “Creation or modification of native modules and

managed modules is not supported...”• SCOM 2012 [5]• Only create them, when provided modules “are

insufficient for your needs”

Advantages• Persisted State• Compiled .NET code can work faster [1]•Harder to Tamper/Reverse Engineer• Full Featured IDE• Leverage .NET Coding Professionals

Disadvantages•More Complicated than Scripted Alternatives• Additional Knowledge required to write Code• Requires additional Steps for Distribution•Updates to Code take longer to deploy than scripts.

Why .NET? – Pros and Cons

Functionality & Examples

Microsoft Examples• PowershellProbeBase• PowershellProbe

• PowershellDiscoveryProbeBase• PowershellDiscoveryProbe

• SystemCenter.GroupPopulator• SystemCenter.EventWriter• .DiscoveryDataWriter• .PerformanceDataWriter

Elastic Search DW•High Speed Logging• 4,000 items Per Second• Perf Collection Every 2

Seconds• Reduce Database Size• Smaller OpsDB• Extended DW Retention• Fast Searching and

Reporting

Demo.NET Elastic Search DW (Frontend)

Development ConceptsTo get the most from your .NET modules there are a few things to keep in mind.

Design and Development notes from Microsoft:https://msdn.microsoft.com/en-us/library/hh769912.aspx

Development Concepts

Threading•Not a Unique concern to .NET•Health Service is a Shared Environment• IO Bound: Use Async Methods• CPU Bound: Perform work Synchronously

Batching [6]• Allows Multiple Items per Execution•Not relevant for composite module types• Consolidation Modules with multiple Items

Development Concepts

Persistence• Save State between Executions•Module Host handles data• Cleared on Module Change

Acknowledgments• Transactional Data Flow not Supported [5]• Responsible until Acknowledgment

Completion Notifications• Notifies Health Service work is Complete•Must Notify Completion before next Request

Development Concepts

Startup and Shutdown• Constructor• Start• Called After Constructor Completes• Must be Thread Safe with NotifyStop

• NotifyStop• Notifies of a graceful shutdown

• Shutdown• About to be Unloaded, time to save State

Engine may Unload without Notice• Module must not corrupt threads

Making the LibraryImplementation is where the fun really is.

New Visual Studio Project

Create a Class Library Project• Sign the Build• Set the Namespace• Strong Named Assemblies Requiredhttps://msdn.microsoft.com/en-us/library/ms227566(v=vs.80).aspx

Handle Required References• Are they on the Target?• Embed Resources you may Needhttp://www.codeproject.com/Articles/528178/Load-DLL-From-Embedded-Resource

Class Definition

Attributes•MonitoringModule(ModuleType.WriteAction)•ModuleOutput (True/False)

Inherits ModuleBase Of DataItemBaseGlobal Variables• SyncLock Object• Shutdown in Progress Boolean

[MonitoringModule(ModuleType.WriteAction)]

[ModuleOutput(false)]

public sealed class WriteToES

: ModuleBase<DataItemBase>

{

//Shared objects, accessible from all instances

static P2PLogging Logger;

static bool ShutdownInProgress;

//Global object, controls activity via SyncLocks

private object shutdownLock;

//Making Global, reducing instantiation cost

private ElasticSearchConnector ESConnector;

private DataItemProcessor DataItemCollection;

<MonitoringModule(ModuleType.WriteAction)> _

<ModuleOutput(False)> _

Public NotInheritable Class WriteToES

'Inherit the ModuleBase we will be working off of

Inherits ModuleBase(Of DataItemBase)

'Shared objects are accessible from all instances

Shared Logger As P2PLogging

Shared ShutdownInProgress As Boolean

'Global object, controls activity via SyncLocks

Private shutdownLock As Object

'Making Global, reducing instantiation cost

Private ESConnector As ElasticSearchConnector

Private DataItemCollection As DataItemProcessor

Class Construction: CodeVB.NET C#C

#VB.NET

Required Methods - New

Signature• moduleHost As ModuleHost(Of

DataItemBase)• configuration As XmlReader• previousState As Byte()

Setup your Module• Logging• Libraries• Configuration Parsing

public WriteToES(ModuleHost<DataItemBase> moduleHost, XmlReader configuration, byte[] previousState) : base(moduleHost) {

//Verify we Have everything we needif (configuration == null) {

throw (new Exception("configuration is nothing"));}

//Add the Additional Libraries to our instantiationLibraryLoader LibraryLoader = new LibraryLoader(Assembly.GetExecutingAssembly(), false);LibraryLoader.LoadLibrariesFromResources();

//Extract from Config what we need and initiate the connections.ParsedConfigData InstanceConfig = new ParsedConfigData();InstanceConfig = ExtractConfigData(configuration);

//Create the Classes we will use throughout the life of this moduleLogger = new P2PLogging();DataItemCollection = new DataItemProcessor(Logger);shutdownLock = new object();

Public Sub New(moduleHost As ModuleHost(Of DataItemBase), configuration As XmlReader, previousState As Byte())

'Call the Base Constructor MyBase.New(moduleHost)

'Verify we Have everything we need If configuration Is Nothing Then Throw New Exception("configuration is nothing")

'Add the Additional Libraries to our instantiation Dim LibraryLoader As New LibraryLoader(Assembly.GetExecutingAssembly(), False) LibraryLoader.LoadLibrariesFromResources()

'Extract from Config what we need and initiate the connections. Dim InstanceConfig As ParsedConfigData InstanceConfig = ExtractConfigData(configuration)

'Create the Classes we will use throughout the life of this module Logger = New P2PLogging DataItemCollection = New DataItemProcessor(Logger) shutdownLock = New Object()

New Method: CodeVB.NET C#C

#VB.NET

Required Methods - Start

Empty SignatureCheck for ShutdownLock•ModuleHost.RequestNextDataItem()

public override void Start()

{

lock(shutdownLock)

{

//We don't need to continue if Shutdowns are starting.

if (ShutdownInProgress)

{

return ;

}

//Request the first data batch.

//The ME keyword unambiguously refers to this instance

this.ModuleHost.RequestNextDataItem();

}

}

Public Overrides Sub Start()

SyncLock shutdownLock

'We don't need to continue if Shutdowns are starting.

If ShutdownInProgress Then

Return

End If

'Request the first data batch.

'The ME keyword unambiguously refers to this instance

Me.ModuleHost.RequestNextDataItem()

End SyncLock

End Sub

Start Method: CodeVB.NET C#C

#VB.NET

Required Methods - OnNewDataItems

Denote Input Stream Number• [InputStream(0)]

Signature• dataItems As DataItemBase()• logicallyGrouped As Boolean• acknowledgedCallback As DataItemAcknowledgementCallback• acknowledgedState As Object• completionCallback As DataItemProcessingCompleteCallback• completionState As Object

Do work with the DataItems()Lock• ModuleHost.RequestNextDataItem()

[InputStream(0)]public void OnNewDataItems(DataItemBase[] dataItems, bool logicallyGrouped, DataItemAcknowledgementCallback acknowledgedCallback, object acknowledgedState, DataItemProcessingCompleteCallback completionCallback, object completionState) {

// If we have been shutdown stop processing.if (ShutdownInProgress){

Logger.WriteInformation("Shutdown in Progress");return ;

}

// This is where you will do your work, code has been removed for Space

lock(shutdownLock) {this.ModuleHost.RequestNextDataItem();}

}

<InputStream(0)> _Public Sub OnNewDataItems(dataItems As DataItemBase(), logicallyGrouped As Boolean, acknowledgedCallback As DataItemAcknowledgementCallback, acknowledgedState As Object, completionCallback As DataItemProcessingCompleteCallback, completionState As Object)

' If we have been shutdown stop processing. If ShutdownInProgress Then Logger.WriteInformation("Shutdown in Progress") Return End If

' This is where you will perform you work with the DataItems() SyncLock shutdownLock Me.ModuleHost.RequestNextDataItem()

End SyncLock

End Sub

OnNewDataItems Method: CodeVB.NET C#C

#VB.NET

Required Methods - Shutdown

Empty SignatureLock•Mark Global Shutdown in Progress• Cleanup as Needed• Finalize()

public override void Shutdown()

{

//Lock to prevent other operations during Shutdown

lock(shutdownLock)

{

ShutdownInProgress = true;

//Cleanup Here

}

}

Public Overrides Sub Shutdown()

'Lock to prevent other operations during Shutdown

SyncLock shutdownLock

ShutdownInProgress = True

Me.Finalize()

End SyncLock

End Sub

Shutdown Method: CodeVB.NET C#C

#VB.NET

Getting The Module in SCOMIt’s time to get back into familiar territory

Creating a Library Management Pack

DeployableAssembly• Fully Qualified Assembly Name<Resources> <DeployableAssembly ID="P2P.ManagedScomModules" Accessibility="Public" FileName="ManagedSCOMModules.dll" HasNullStream="false" QualifiedName="ManagedSCOMModules, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d49ecc768c087c65"/> </Resources>

DLL in Management Pack Project• Set Build Action = Embedded Resource

Create the Write Action Module<ModuleTypes> <WriteActionModuleType ID="P2P.Library.WriteToElasticSearch" Accessibility="Public" Batching="true"> <Configuration> <xsd:element name="ElasticSearchNode" type="xsd:string" /> <xsd:element name="WinEventIndex" type="xsd:string" /> <xsd:element name="AllOtherIndex" type="xsd:string" /> <xsd:element name="TryBulkInsert" type="xsd:boolean" /> </Configuration> <ModuleImplementation Isolation="Any"> <Managed> <Assembly>P2P.ManagedScomModules</Assembly> <Type>ManagedSCOMModules.WriteToES</Type> </Managed> </ModuleImplementation> <InputType>System!System.BaseData</InputType> </WriteActionModuleType> </ModuleTypes>

Create a Pack with Actions<Rule ID="P2P.Custom.LogTester.ProcessProc" Target="Windows!Microsoft.Windows.Server.Computer" Enabled="true" ConfirmDelivery="false" Remotable="true" Priority="Normal" DiscardLevel="100"> <Category>PerformanceCollection</Category> <DataSources> <DataSource ID="DS" TypeID="Perf!System.Performance.DataProvider"> <ComputerName>.</ComputerName> <CounterName>% Processor Time</CounterName> <ObjectName>Process</ObjectName> <AllInstances>true</AllInstances> <Frequency>2</Frequency> </DataSource> </DataSources> <WriteActions> <WriteAction ID="WriteToES" TypeID="PCMML!P2P.Library.WriteToElasticSearch"

Target="SC!Microsoft.SystemCenter.CollectionManagementServer"> <ElasticSearchNode>http://p2p-elastic-01.cloudapp.net:13580</ElasticSearchNode> <WinEventIndex>SCOMWinEvents</WinEventIndex> <AllOtherIndex>SCOMDataItems</AllOtherIndex> <TryBulkInsert>true</TryBulkInsert> </WriteAction> </WriteActions> </Rule>

DemoPutting Everything Together.

References1. http://

leporelo.eu/blog.aspx?id=powershell-performance-comparison-compiled-code-cmdlet-or-net-call

2. https://msdn.microsoft.com/en-us/library/7k989cfy(v=vs.90).aspx

3. http://social.technet.microsoft.com/wiki/contents/articles/15219.operations-manager-management-pack-authoring-module-implementations.aspx

4. https://technet.microsoft.com/en-us/library/ff381347.aspx

5. https://msdn.microsoft.com/en-us/library/hh769912.aspx

6. https://nocentdocent.wordpress.com/2014/07/05/one-module-to-rule-them-all-a-custom-scom-managed-module-sysctr/

Evaluations: Please provide session feedback by clicking the EVAL button in the scheduler app (also download slides). One lucky winner will receive a free ticket to the next MMS!Session Title: Pushing the Envelope; .NET Code in SCOM Management Packs

Discuss…

Ask your questions-real world answers!Plenty of time to engage, share knowledge.

SPO

NSO

RS