Apportal Instrumentation and Telemetry - Microsoft...Application Insights for Visual Studio Online (VSO) – the older version, which works with Visual Studio 2012 and Visual Studio

3009 112th Ave NE, Suite 200 Bellevue, WA 98004 425.702.8175 www.invisocorp.com

Apportal Instrumentation and Telemetry 4/17/2015

Tom Tran

Software Architect

Inviso Corporation

This article focuses on instrumentation for monitoring and alerting in Windows Store and Windows Phone applications. Effective application instrumentation involves efficient and scalable telemetry data collection, and enables both development and operational efficiency when it comes to identifying, troubleshooting, and resolving issues in Production. Since an Apportal is typically implemented as a Windows Store app, the following sections also apply to Apportal instrumentation and telemetry.

Instrumentation and Telemetry Applications typically include diagnostics codes that generate custom monitoring and debugging information, especially when an error occurs. This is referred to as instrumentation. The process of capturing, gathering remote information that is generated by instrumentation is referred to as telemetry. A common feature of instrumentation is an ability to support a configurable level of detail that is collected on-demand. Under normal conditions, only events such as errors and warnings are logged. This reduces the cost of storage and the processing cycles required to collect data. When there is an issue with the application, the diagnostics logging and instrumentation systems can be configured to collect verbose informational event data, as well as error and warning messages. Telemetry is typically performed using asynchronous mechanisms that support massive scaling and wide distribution of application services. In large and complex applications, information is usually captured in a data pipeline (with offline support and resilient to transient errors) and stored in a form that makes it easier to analyze, and capable of presenting information at varying levels of granularity. This information is used to discover trends, gain insights into usage and performance, and to detect and isolate problems in software.


Application Insights Application Insights is a set of services that provide a robust platform for developers to efficiently incorporate instrumentation into ASP.NET applications, Windows Store apps, and Windows Phone apps so that telemetry data about application performance, availability, usage, and exceptions can be gathered, analyzed, and reported at scale. Currently, there are two versions of Application Insights:

Application Insights in Microsoft Azure Preview – the new version, which works with Visual Studio 2013 Updates 3 and 4, Visual Studio 2015 and later.

Application Insights for Visual Studio Online (VSO) – the older version, which works with Visual Studio 2012 and Visual Studio 2013.

Microsoft recommend using the new version in Azure, namely Azure Application Insights. They’ve been working hard to migrate the VSO features into the Azure Application Insights. There are new features and improvements every week. The old VSO version will be switched off eventually.

Integrating Your Windows 8.1 Universal App Developers can add Application Insights to their new or existing Universal App projects in Visual Studio 2013 Update 3, or later. There are a few simple steps: create Application Insights resource, add Application Insights NuGet package, update the ApplicationInsights.config file to use appropriate instrumentation key, and enable network access for your app. See the Application Insights: Get started with Windows device apps for more details.

Notes: Use the same Instrumentation Key for both your Window Phone and Windows Store applications to have all of your telemetry data in the same place, allowing for analysis and comparisons about differences in behavior between the two apps. If however you find that you view the telemetry from each application separately, or find yourself collecting a different set of telemetry from each platform, you may want to use a dedicated instrumentation key for each app. This will isolate your data from each other but in some situation that may be more helpful for analysis. This scenario is fully supported as you can specify a different Instrumentation Key in the ApplicationInsights.config file of each project to allow for isolation of data collection.

Multi-environment Support Application Insights can collect telemetry from your application at any stage in its development, from debugging to live production. If you use the same Application Insights resource in Azure (hence same instrumentation key), all these different streams will go into the

http://azure.microsoft.com/en-us/documentation/articles/app-insights-windows-get-started/


same data reservoir and interfere with each other. For example, if you run heavy stress tests, the test results could give a false impression of the number of visitors you’ve had at your live site. To avoid this problem, you may want to create separate application resources in Application Insights to receive the data from each lifecycle stage. The destination of the telemetry is determined by the instrumentation key, which is sent along with every telemetry message. In the Application Insights portal, similar events and metrics with the same instrumentation key are aggregated to give you charts of average durations, event counts, the sum of users, and so on. To direct telemetry to different application resources, you can create several resources with different instrumentation keys. Then you can change the instrumentation key in the application at each transition in its lifecycle. Microsoft Azure provides a convenient way to group related resources so you can create groups such as Dev, Test, UAT and Production to support multi-environment telemetry data collection.

Analytics Out-of-the-box Once Application Insights SDKs have been integrated into your project, telemetry data is automatically collected and flows directly to the Application Insights portal. The SDK will collect basic telemetry automatically. This telemetry includes things like basic user and sessions tracking, collection of page views, and collection of contextual metadata such as device model, operating system, and screen resolution. The following picture shows a sample page view chart with customization, filters, and grouping. Users and sessions are defined as follows.1

In web applications, users are counted by using cookies. A person who uses several browsers, clears cookies, or uses the privacy feature will be counted several times. A web session is counted after 30 minutes of inactivity. A session on a phone or other device is counted when the app is suspended for more than a few seconds.

1 From Microsoft Azure documentation, Usage analysis with Application Insights

http://azure.microsoft.com/en-us/documentation/articles/app-insights-windows-usage/



Custom Events and Metrics Application Insights API surface area is pretty uniform across platforms Windows Store, Windows Phone, and ASP.NET (with some minor variations). Application Insights data collectors use the following APIs to send standard telemetry data such as page views and exception reports. You can also use them to send your own custom telemetry.

Method Purpose Example Usage

TrackPageView Records pages, screens, blades or forms

TelemetryClient.TrackPageView("SelectIndustry");


TrackEvent Records user actions and other events. Used to track user behavior or to monitor performance.

IDictionary<string, string> properties = new Dictionary<string, string>(); properties.Add("TileName", tileName); properties.Add("QueryURI", queryUrl); App.TelemetryClient.TrackEvent("TwitterTileClicked", properties);

TrackMetric Records performance measurements such as queue lengths not related to specific events

TelemetryClient.TrackMetric("Queue", queue.Length);

TrackException Records exceptions for diagnosis. Trace where they occur in relation to other events and examine stack traces.

catch (Exception e) { TelemetryClient.TrackException(e); . . . }

TrackRequest Records the frequency and duration of server requests for performance analysis. Notes:

// Operation Id is attached to all telemetry and helps you // identify telemetry associated with one request: TelemetryClient.Context.Operation.Id = Guid.NewGuid().ToString(); var stopwatch = System.Diagnostics.Stopwatch.StartNew(); // ... process the request ... stopwatch.Stop(); telemetryClient.TrackRequest(requestName, DateTime.Now, stopwatch.Elapsed, "200", true); // Response code, success

TrackTrace Records diagnostic log messages. You can also capture 3rd-party logs

TelemetryClient.TrackTrace(message, SeverityLevel.Warning);

The above API methods have overloads that allow developers to attach properties, and measurements to their metrics, events, page views, and other telemetry data. Properties and metrics are defined as follows.2

Properties are string values that you can use to filter your telemetry in the usage reports. There's a limit of about 1k on the string length. (If you want to send large chunks of data, use the message parameter of TrackTrace.)

Metrics are numeric values that can be presented graphically. For example, you might want to see if there's a gradual increase in the scores your gamers achieve. The graphs can be segmented by the properties sent with the event, so that you could get separate or stacked graphs for different games. Metric values should be >= 0 to be correctly displayed.

The following picture shows how custom events are displayed in the Application Insights portal.

2 From Microsoft Azure documentation, Custom events and metrics with the Application Insights API

http://azure.microsoft.com/en-us/documentation/articles/app-insights-custom-events-metrics-api/


Monitoring and Alert You can monitor the application health and performance with the standard charts that displays a variety of metrics such as active users, sessions, page views, and crashes. Alerts can be set up to notify an administrator when a certain metric passes a configured threshold value.


Offline Mode When offline, Application Insights data collectors cache telemetry data in current session into local files and send on next session when online. The following picture shows temporary files accumulated in the Telemetry local folder when the Windows Store app runs offline. When online, Application Insights empties this folder as it sends cached telemetry data to Azure.


Diagnostic Search You can drill into the events, metrics, exceptions from the standard reporting charts, or by navigating directly to the Diagnostics Search to get additional details and stack trace for the standard event types.

Trace - Diagnostic logs including TrackTrace, log4Net, NLog, and System.Diagnostic.Trace calls.

Request - HTTP requests received by your server application, including pages, scripts, images, style files and data. These events are used to create the request and response overview charts.

Page View - Telemetry sent by the web client, used to create page view reports.

Custom Event - If you inserted calls to TrackEvent() in order to monitor usage, you can search them here.

Exception - Uncaught exceptions in the server, and those that you log by using TrackException().


Privacy and Data Retention Policy According to Microsoft, the telemetry data is stored in Azure storage at rest unencrypted. However, the data is always encrypted in transit through the secure socket layer (SSL). PII data can potentially be collected and stored.3

3 From Microsoft Azure documentation, Data collection, retention and storage in Application Insights

http://azure.microsoft.com/en-us/documentation/articles/app-insights-data-retention-privacy/


Most standard telemetry (that is, telemetry sent without you writing any code) does not include explicit PII. However, it might be possible to identify individuals by inference from a collection of events.

Exception reports might include PII in parameter data.

Custom telemetry – that is, calls such as TrackEvent that you write in code using the API or log traces – can contain any data you choose.

The following table documents the telemetry collected.

Collected data class Includes (not an exhaustive list)

ServerContext Machine name, locale, OS,

ClientContext Browser type, OS, locale, language, network, window resolution

DeviceContext Locale, language, Device model, Device language, network, network type, OEM name, screen resolution

Perf counters Processor time, available memory, request rate, exception rate, process private bytes, IO rate, request duration, request queue length

Requests HTTP request string, duration, response code

Dependencies Type(SQL, HTTP, ...), connection string or URI

Exceptions Type, message, call stacks, parameter data

Crashes Process id, parent process id, crash thread id, application patch, obfuscated symbols and registers, binary start and end addresses, binary name and path, cpu type

Session session id

Anon users GUID

Page URL and page name

Auth users

Inferred geo location from IP address, timestamp, OS, browser

Properties Any data - determined by your code

The official Microsoft retention policy is as follows.4

Up to 30 days for individual events (that is, data items that you can inspect in Diagnostic Search).

Aggregated data (that is, counts, averages and other statistical data that you see in Metric Explorer) are retained at a grain of 1 minute for 30 days, and 1 hour or 1 day (depending on type) for 13 months.

Application Insights data collectors infer geo location data from captured IP addresses such as latitude/longitude, continent, province, country. The IP addresses have the first part “blanked” out with zero, e.g. 0.54.247.34 (see also below picture).

4 From Microsoft Azure documentation, Data collection, retention and storage in Application Insights



By default, the IP addresses are only captured if the TelemetryClient.Context.Location.Ip property is null (which is the default). Developers can prevent the geo location data from being captured by setting this property to “0.0.0.0”. See the below code snippet for an example.


Data Export Telemetry data export is possible with the Application Insights’ Continuous Export feature. Data is delivered in JSON files, with .blob extension, to a designated blob storage container, and organized in date and time (UTC) folders.


These JSON files can be further processed/ingested (ETL) into a data mart or data warehouse environment for custom reporting and analytics. ETL processes that pull data from these JSON files can be implemented using the Blob Storage API within Azure Data Factory, SSIS, or a custom data ingestion service (e.g. worker role). Below is sample JSON data file for an event.


Pricing Application Insights pricing model has not been announced yet. It is free at the present. The only charge would incur if you use the Continuous Export service, in which case you pay for blob storage.


Find More Information Application Insights: Get started with Windows device apps http://azure.microsoft.com/en-us/documentation/articles/app-insights-windows-get-started/ Monitor usage in Windows apps and Windows Phone apps http://azure.microsoft.com/en-us/documentation/articles/app-insights-windows-usage/ Export telemetry from Application Insights http://azure.microsoft.com/en-us/documentation/articles/app-insights-export-telemetry/ Data collection, retention and storage in Application Insights http://azure.microsoft.com/en-us/documentation/articles/app-insights-data-retention-privacy/

http://azure.microsoft.com/en-us/documentation/articles/app-insights-windows-get-started/

http://azure.microsoft.com/en-us/documentation/articles/app-insights-windows-usage/

http://azure.microsoft.com/en-us/documentation/articles/app-insights-export-telemetry/


Documents

Apportal Instrumentation and Telemetry - Microsoft...Application Insights for Visual Studio Online (VSO) – the older version, which works with Visual Studio 2012 and Visual Studio