Using Cacti To Graph MySQL’s Metrics - Percona · PDF fileUsing Cacti To Graph...

Using Cacti To Graph MySQL’s MetricsKenny Grypkenny.gryp@percona.comPrincipal Consultant @ Percona

Collaborate 2011

Percona

•MySQL/LAMP Consulting•MySQL Support•Percona Server (XtraDB)•Percona XtraBackup• InnoDB Recovery Toolkit, tcprstat•(maatkit, innotop, aspersa, mysql-mmm, mysql-cacti-

templates) • ...•http://www.percona.com•http://www.mysqlperformanceblog.com

Topics

•Why Graph?•What is Cacti and mysql-cacti-templates?•How Cacti works• Installation•Add Graphs•Demo•Graphs!•Extra’s

Topics

Why Graph?

•Troubleshooting-behavior over time-sudden spikes in graphs after application updates

•Capacity Planning-traffic patterns-predictions

•Not only for Operations, also for Development and Management

Topics

Cacti?

•open source graphing solution•uses RRDTool•advanced graphs, scriptable•web interface•quite complex to add graphs•adding servers is a manual process

mysql-cacti-templates?

•collection of templates for cacti•graph MySQL metrics•Has a lot of non-MySQL Metrics too: IO stats,

apache, memcache, mongodb, java...

Topics

How does Cacti work?

Database Servermysqld

/proc/diskstats

MySQL(contains cacti configuration)

.rrd Files(contains rrd data)

Apache(serves web

interface)

Poller(cronjob which

fetches data from servers)

Web Serverhttpd

Web Browser

stores

Topics

Installation

•Cacti•Mysql-cacti-templates•Configuring Servers

Installation

Setting up Cacti: Requirements

•httpd•php, php-mysql, php-snmp•mysql-server (server and client utilities)•net-snmp (snmpget)•rrdtool

Setting up Cacti: Installation

• Install rpm/deb/... package or extract yourself•Configure webserver: php5, cacti scripts•Create cacti mysql user• Import cacti database•Edit configuration•Setup cronjob for poller•Finish installation using your webbrowser

Setup: Extract

•Recommended is using apt/yum/... to install cacti• if not:

cacti# cd /var/www/html/ # may varycacti# tar xzvf cacti-0.8.7g.tar.gzcacti# ln -s cacti-0.8.7g cacticacti# chown -R cactiuser rra/ log/

Setup: Webserver

•Apache (may be something else)•Enable php•Set DirectoryIndex index.php

LoadModule php5_module modules/libphp5.soAddHandler php5-script .phpAddType text/html .phpDirectoryIndex index.php

Setup: MySQL

•Create Cacti MySQL user:

• Import Cacti Schema

•Edit include/config.php

cactimysql> GRANT ALL PRIVILEGES ON cacti.* TO cactiuser@localhost IDENTIFIED BY 'password';

cacti# mysql -e ‘CREATE DATABASE cacti;’cacti# mysql cacti < cacti.sql

$database_type = "mysql";$database_default = "cacti";$database_hostname = "localhost";$database_username = "cactiuser";$database_password = "password";

Setup: Poller

•Add cronjob (/etc/cron.d/cacti)

*/5 * * * * cactiuser php /var/www/html/cacti/poller.php > /dev/null 2>&1

Setup: UI

Installation

download & installation

cacti# wget http://mysql-cacti-templates.googlecode.com/files/better-cacti-templates-1.1.8.tar.gzcacti# tar -xzf better-cacti-templates-1.1.8.tar.gz cacti# cd better-cacti-templates-1.1.8/cacti# cp \ scripts/{ss_get_mysql_stats.php,ss_get_by_ssh.php}\ /var/www/html/cacti/scripts/

install: mysql script

cacti# vi /var/www/html/cacti/scripts/\ss_get_mysql_stats.php ...$mysql_user = 'cactiuser';$mysql_pass = 'password';...

install: generate ssh key

cacti# mkdir /etc/cacticacti# ssh-keygen Generating public/private rsa key pair.Enter file in which to save the key (/root/.ssh/id_rsa): /etc/cacti/id_rsaEnter passphrase (empty for no passphrase): Your identification has been saved in /etc/cacti/id_rsa.Your public key has been saved in /etc/cacti/id_rsa.pub.cacti# chown -R www-data: /etc/cacti/{id_rsa,id_rsa.pub}cacti# chmod -R 0400 /etc/cacti/id_rsa

install: ssh script

cacti# vi /var/www/html/cacti/scripts/ss_get_by_ssh.php ...$ssh_user= 'cacti'; # SSH username$ssh_port= 22; # SSH port$ssh_iden='-i /etc/cacti/id_rsa';# SSH identity...

install: templates

Installation

Configure Servers

•configure net-snmpd•add a cacti user in MySQL•configure ssh access

Configure Servers: snmpd

• Install snmpd

•Make sure snmpd listens on the network and that a community password is set:

•Verify on the cacti server if snmp is working remotely

server# apt-get install snmpd

server# vi /etc/snmp/snmpd.conf:agentAddress udp:161rocommunity publicserver# /etc/init.d/snmpd restart

cacti# snmpget -v2c -c public ubuntu-server-test iso.3.6.1.2.1.1.5.0

iso.3.6.1.2.1.1.5.0=STRING:"ubuntu-server-test"

Configure Servers: mysql user

•Add cacti user

•Verify if login works from cacti servers

servermysql> GRANT SUPER, PROCESS ON *.* TO 'cactiuser'@'cactiserver' IDENTIFIED BY "password";

cacti# mysql -h ubuntu-server-test -e "SELECT VERSION()" -u cactiuser -ppassword+-------------------+| VERSION() |+-------------------+| 5.1.49-1ubuntu8.1 |+-------------------+

Configure Servers: ssh access

•Add shell user cacti

•Copy the public key

•Verify

server# useradd cactiserver# getent passwd cacticacti:x:1001:1002::/home/cacti:/bin/sh

server# mkdir /home/cacti/.ssh/server# cat > /home/cacti/.ssh/authorized_keys << EOFssh-rsa AAA...r2E7z5 root@ubuntu-server-testEOFserver# chmod 0400 /home/cacti/.ssh/authorized_keys

cacti# ssh -i /etc/cacti/id_rsa cacti@ubuntu-server-test

Topics

Adding Graphs

•Apply wanted templates on the device- MySQL & GNU/Linux templates

•Click on Create Graphs:

•Select graphs to create and Add!•Might ask some more questions

Adding Graphs

•Assign ucd/net SNMP template to have:-cpu usage/load-memory used/available-network statistics-free disk space

•Different way of creating

Adding Graphs

•What if I have multiple disk devices to monitor?-Go to ‘Create graphs for this host’-Select graph to add in select menu-Create and fill in other device when asked

Adding Graphs: Graph Trees

•Tree to browse through graphs•By host/individual graph, mixed

Topics

MySQL Handlers

Table scans, backup?

MySQL Handlers

Take backup here

MySQL Connections

max_connections not reached yet

Why more incoming connections?

MySQL Replication

Replication was stopped

Replication never caught up

MySQL Replication

In 2 hours, almost 6900 seconds replication lag

MySQL Temporary Objects

keep an eye on:* temp tables* temp disk tablesif it grows > query optimization

MySQL Select Types

Why this sudden peak in rangescans?

MySQL Select Types

Query behavior changed?Other explain plan chosen?

MySQL Sorts

The peaks should be understood, not necessarily be removed as they might be really fast in terms of rows

MySQL Command Counters

MySQL Binary Logs

expire_logs_days

Response Time Distribution

+----------------+-------+------------+| time | count | total |+----------------+-------+------------|| 0.000001 | 0 | 0.000000 || 0.000010 | 17 | 0.000094 || 0.000100 | 4301 | 0.236555 || 0.001000 | 1499 | 0.824450 || 0.010000 | 14851 | 81.680502 || 0.100000 | 8066 | 443.635693 || 1.000000 | 0 | 0.000000 || 10.000000 | 0 | 0.000000 || 100.000000 | 1 | 55.937094 || 1000.000000 | 0 | 0.000000 || 10000.000000 | 0 | 0.000000 || 100000.000000 | 0 | 0.000000 || 1000000.000000 | 0 | 0.000000 || TOO LONG QUERY | 0 | 0.000000 |+----------------+-------+------------+

MySQL Query Response Time

InnoDB Buffer Pool

DB Restart

Time to fill buffer

InnoDB Checkpoint Age

sudden increase in unflushed writes

db restart, no write flushing

InnoDB Buffer Pool Activity

InnoDB I/O

Using Cacti To Graph MySQL’s Metrics - Percona · PDF fileUsing Cacti To Graph...

Documents

CACTI Manual

Cacti, Succulents, Miscellaneous - Green Valley Gardenersgreenvalleygardeners.com/docs/arid garden brochure.pdfGolden Barrel Cacti Hat Pin Cacti Haworthia Hesperaloe, Brake Lights

Cacti Guide

Cacti Documento

HP NNMi iSPI Metrics Vs CACTI

Arizona Cacti

Kaktuszok /cacti

Tutorial Cacti

Snmp Cacti 1

GALAPAGOS (CACTI)

Membangun Cacti

Cacti & rocks

Biodiversity of Cacti

Cara Instal Cacti

Libro CACTI Castellano

Cacti Setup

Cacti Fortigate Original

Cacti Workshop

Cacti Install Manual

Cisco Cacti