Embed Size (px)
Citation preview
SERVER MONITORING 101
WHAT WE ARE GOING TO DISCUSS TODAY
Why we need monitoring
What do we monitor
Different monitoring services
Cloudwatch
Datadog
Demo
WHY NEED MONITORING?We need to make sure our application
Does not FAIL Makes our users HAPPY
That means:
• Application is working properly
• Customers getting what they want
• The responses are not slow
• Not getting errors
WHY NEED MONITORING?We need some indicators that we can measure
1200 1230 1300 1330 1400 1430 15000
20406080
100
CPU Utilization
Server 1 Server 2 Server 3
1200 1230 1300 1330 1400 1430 15000
200400600800
Error Counts
Server 1 Server 2 Server 3
About what?WEB DB
Cloud Resources
Application
So that we can- Know about how our application is performing
- Get alert if something goes wrong
- Take steps necessary to prevent failure- Improve user experience- Realize trends of resource utilization
WHAT DO WE MONITOR
InfrastructureApp
Availability
Logs
System info
Exceptions/ Errors
Events (404)
Servers (Web, DB)
CPU Utilization
Cloud storage
Notification Services
Disc Storage
Data Transfer
Load Balancer
Response time
Response status
Request count
Latency
Others Read/Write Latency
AVAILABLE SERVICES
And many more …
Cloudwatch
AMAZON CLOUDWATCH
- Collect and track metrics
- Collect and monitor log files
- Set alarms based on defined thresholds
- Automatic scaling
Monitoring service for AWS resources- Elastic Load Balancers- EC2 instances- RDS instances- EBS volumes- Route53 health checks- DynamoDB tables- SNS topics- SQS queues
What it does
CLOUDWATCH LOGS SETUPDocumentation URL: http://amzn.to/2d3AXsI
Install Agent
sudo yum update –ysudo yum install -y awslogs
Configure settings
sudo vim /etc/awslogs/awscli.conf##### configure the settings belowregion = region = ap-northeast-1# configure following if required, or just use IAM role # aws_access_key_id = <YOUR ACCESS KEY># aws_secret_access_key = <YOUR SECRET KEY>
sudo vim /etc/awslogs/awslogs.conf##### configure the settings below[/usr/local/jetty/logs/app.log]datetime_format = %b %d %H:%M:%Sfile = /usr/local/jetty/logs/app.logbuffer_duration = 5000log_stream_name = {instance_id}initial_position = start_of_filelog_group_name = demo-app-logs
{ "Version": "2012-10-17", "Statement": [
{ "Effect": "Allow", "Action": [
"logs:CreateLogGroup”,
"logs:CreateLogStream”,
"logs:PutLogEvents", "logs:DescribeLogStreams" ], "Resource": [
"arn:aws:logs:*:*:*" ]
} ]
}
Policy required for the IAM role
CLOUDWATCH LOGS SETUPDocumentation URL: http://amzn.to/2d3AXsI
Start Agent
sudo service awslogs startsudo chkconfig awslogs on
We are almost there!
CLOUDWATCH LOGS SETUPDocumentation URL: http://amzn.to/2eVGNzO
Cloudwatch provides APIs to conduct the followings:
DeleteAlarmsDescribeAlarmHistoryDescribeAlarmsDescribeAlarmsForMetricDisableAlarmActionsEnableAlarmActionsGetMetricStatisticsListMetricsPutMetricAlarmPutMetricDataSetAlarmState
AndroidiOS
Java
PHP
Python
Windows.NET
Ruby
SUPPORTS
DATADOGSimilar to Cloudwatch Infrustructure Monitoring As A Service and it’s Open Source!
Can manage a variety of stacks from different infrastructure platformsAnd brings the data in one place
Even brings data from other Monitoring services Cloudwatch
DATADOGAPI Support
Can get performance metrics data using the API
Python
Ruby Shell
Can even get embeddable graph metrics through API calls
There are Community Libraries availableFor other languages (C#, Go, Java etc)Documentation URL: http://bit.ly/2gc0KjK
Documentation URL: http://bit.ly/2g8uuw5
DATADOG INSTALLATIONYou need to create AWS IAM Role for cross account access:
Documentation URL: http://bit.ly/2fT6PRb
Refer to documentation for required policy.
DATADOG AGENT INSTALLATIONOne Step Install
DD_API_KEY=xxx bash -c "$(curl -L https://raw.githubusercontent.com/DataDog/dd-agent/master/packaging/datadog-agent/source/install_agent.sh)”
Step by step
DATADOG SLACK INTEGRATION
Install Datadog app in Slack
DATADOG SLACK INTEGRATIONGet the service hook URL from Slack andingrate from Datadog admin panel.
DEMO
