Upload
maarten-balliauw
View
169
Download
0
Embed Size (px)
Citation preview
What is going on?Application Diagnostics on AzureMaarten Balliauw@maartenballiauw
Who am I? Maarten Balliauw Antwerp, Belgium Developer Advocate, JetBrains Founder, MyGet AZUG Focus on web ASP.NET MVC, Azure, SignalR, ... Former MVP Azure & ASPInsider Big passion: Azure http://blog.maartenballiauw.be @maartenballiauw
Agenda Logging Why? Logging sucks! The need for semantic/structured logging Application Insights SDK, Azure portal Application Insights Analytics Find the needle in the haystack! Build, measure, improve
Logging
Why logging? Troubleshooting – Did a problem occur? Where? Our code? The machine?
Performance / Cost – Did we make the application slower? Improvement – Can a problem be detected or avoided? Trends – Need more storage? Need more servers? Customer Experience – Are people happy? Or seeing error after error?
Business Decisions – Can we increase sales based on log data / telemetry?
So here’s what we typically do...System.Diagnostics.Trace.TraceInformation(
"Something happened");
System.Diagnostics.Trace.TraceWarning("Error! " + ex.Message);
Does this help troubleshooting? Improve the application? Analyze trends?
And here’s a typical log...App.exe [12:13:03:985] Information: 0 : Customer address updated.App.exe [12:13:04:011] Error: 8 : System.NullReferenceException occurred. Value can not be null.App.exe [12:13:04:567] Information: 0 : Machine policy value 'Debug' is 0App.exe [12:13:04:569] Verbose: 9 : ******* RunEngine ******* Action: ******* CommandLine: **********App.exe [12:13:04:578] Information: 9 : Entered CheckResult()App.exe [12:13:04:689] Debug: 0 : Created. Req=0, Ret=8, Font: Req=null
Does this help troubleshooting? Improve the application? Analyze trends?
No.
Log files suck. Stupid string data Unless using ETW / proper System.Diagnostics listener Typical log file has no “context” Typical log file has no correlation How to get data off our machines? How to report/analyze?
Log files suck. Stupid string data Typical log file has no “context” Process/thread Machine name User performing the action Data specific to the event being logged Typical log file has no correlation How to get data off our machines? How to report/analyze?
Log files suck. Stupid string data Typical log file has no “context” Typical log file has no correlation, e.g.: What was the server load when this was logged? How much memory was consumed at the time? Which web request invoked this action? How to get data off our machines? How to report/analyze?
Log files suck. Stupid string data Typical log file has no “context” Typical log file has no correlation How to get data off our machines? How to report/analyze?
Log files suck. Stupid string data Typical log file has no “context” Typical log file has no correlation How to get data off our machines? How to report/analyze? Log processing and analysis – log, store, process, query, proactive
analysis Logstash, ElasticSearch, ... – cumbersome to setup
What if we had (better)contextual information?
Structured/Semantic logging “Of or relating to meaning, especially meaning in language” – the dictionary
Windows Event Log Event Tracing for Windows (ETW) Semantic Logging Application Blog (PnP SLAB) Serilog …
Event Source XML representation containing contextual information application-specific data elements Can be read by humans (just a message, like any log) Can be read by machines (XML, baby!) add “meaning” – we know what the data points can be allow automated aggregaton / trend analysis Fast! (not very developer friendly)
Using EventSourceDEMO
Event Source Super awesome It has structured logging and context Can collect all kinds of data combined (e.g. HTTP + own event source) Hard to maintain Versioning Ceremony One method for every single thing we want to log (or at least an event
id) Requires thinking about the events to log Lots of data
Serilog - https://serilog.net/ Logging library like any other, mostly Sinks https://github.com/serilog/serilog/wiki/Provided-Sinks Console, file, event log Database ElasticSearch AppInsights Structured data as a first-class concept Enrichers, filters, …
Structured data in Serilog Simple scalar values (bool, string, int, …) Collections and dictionaries Full objects
Rendered by a sink to, for example, a string Can be stored by a sink to provide additional dimensions on data
Serilog - BasicsDEMO
Enrichment of data Enrich log entries with additional context Machine name User name Process / thread information ASP.NET client IP / hostname ASP.NET user agent … Using Serilog enrichers (when applicable) In custom ASP.NET middleware
But still... How to get the data off our machine(s)? How to analyze/aggregate/predict? Which customer had this Exception and why? Did our sales go up after we made this page faster? What button on the front-end crashes this microservice? Are # Exceptions increasing? Do I need special infrastructure to store and process logs?
The cloud to the rescue? (Azure) Azure Insights Azure platform’s core monitoring and alerting services (standard Metrics, Alerts,
Autoscale, Email Notifications and Audit Logs)
Application Insights Application monitoring & logging for your applications (web, mobile, desktop, …) in
cloud or other server
Log Analytics (OMS) Log Analytics, part of the Operations Management Suite (OMS) ingests data from
Azure servers and provides rich ‘global’ view & advanced log search based alerting capabilities
Application Insights
Application Insights Azure Service + library/SDK Solves “where to store”, “how to ship”, “how to analyze” Enriches data Correlates data (e.g. client + server + dependency/DB/...) Telemetry! Allows structured logging Allows rich querying, alerting Works for many, many platforms and languages Web, Windows, Xamarin, any application type really .NET, Java, JavaScript, Objective-C, PHP, Python, Ruby, ...
Getting StartedDEMO
Do I have to run on Azure? No. REST API to send data to Various tools and SDK’s to collect data specific to language/platform Status Monitor on non-Azure machines
Data collected Performance Counters Requests (both server/client side) Traces (more later) Exceptions Dependencies Custom Metrics & Events That is a lot of data that can be correlated!
Different Views
Application Insights portal Dashboard Basic metrics (# Alerts, Response times, Exceptions, Failed Requests, ...) Gateway to all of the below Application map Smart Detection / Alerts Live Metrics Stream Search, Availability, Failures, Performance, ... Metrics (and events) Analytics
Application MapDEMO
Smart Detection / AlertsDEMO
Live Metrics StreamDEMO
Search, Availability, Failures, Performance, ...DEMO
Metrics and Events Send custom events and metrics Name of event/metric + a value Events – help find how the application is used and can be optimized User resized column in a grid User logged in User clicked “Share” No more stock ... Metrics – help alerting / finding missing functionality Purchase occurred Search returned zero results # Support calls ...
Metrics and EventsDEMO
So what about logging... Configure Serilog to use Application Insights as a sink Custom data becomes available (*) So we can correlate our own data with performance, requests, traces,
exceptions, depenencies, ... But how to do this? And how to consume?
(*) Make sure the entire object is in the log message, no tracking of non-printed data
LoggingDEMO
Application Insights Analytics
Application Insights Analytics Former MS-internal tool “Kusto” Near-realtime log ingestion and analysis Lets you run custom queries
requests | where timestamp > ago(24h) | summarize count() by client_CountryOrRegion | top 10 by count_ | render piechart
Queries over logs! Input dataset traces customEvents pageViews requests dependencies exceptions availabilityResults customMetrics performanceCounters browserTimings
| Operators where count project join limit order
| Renderer table chart (bar, pie, time, area, scatter)
https://docs.microsoft.com/en-us/azure/application-insights/app-insights-analytics-reference
Application Insights AnalyticsDEMO
Not covered in this talk VS plugin for Application Insights Run some analysis inside VS Continuous export Export data from Application Insights continuously if you want to
analyze in other tools Retain data longer than 7 days Join with other data (preview) https://docs.microsoft.com/en-us/azure/application-insights/app-insights-analytics-using#import-data Join logging and metrics with other data sources, query with AI
Analytics
Conclusion
But... Why? Seriously, why are you in this session...
But... Why? Troubleshooting – Did a problem occur? Where? Our code? The machine?
Performance / Cost – Did we make the application slower? Improvement – Can a problem be detected or avoided? Trends – Need more storage? Need more servers? Customer Experience – Are people happy? Or seeing error after error?
Business Decisions – Can we increase sales based on log data/telemetry?
Structured logging Enrich technical logs with business data Allows correlating when things go wrong Allows making business decisions when using a good data analysis tool on top
Application Insights Is that good analysis tool! (for web, mobile, desktop!) Solves Collecting various sources Ingesting various sources Enrichment with structured logging (e.g. Serilog) Helps analyzing all that data Default views, metrics, alerts, monitoring Custom querying – Application Insights Analytics Build, measure, improve! Both technical and business – there isn’t always a divide between both...
Thank you!Need training, coaching, mentoring, performance analysis? Hire me! https://blog.maartenballiauw.be/hire-me.html
http://blog.maartenballiauw.be@maartenballiauw