Shit happensCentralize logs and get your insight into the errors that

affect your customers.

Jackson F. de A. Mafrahttp://about.me/jacksonfdam https://bitbucket.org/jacksonfdam https://github.com/jacksonfdam http://linkedin.com/in/jacksonfdam @jacksonfdam

Software Engineer at Aggrega Group, mobile training instructor at Targettrust. Developer for 15 years with background in e-commerce projects and real estate, since 2009 with focused interests for the development of mobile and MEAP and applications interfaces.

Aspect oriented programming (AOP) allows us to keep implement different concerns in

isolation

Cross-cutting concerns are conceptually separate from (but often embedded directly

within) the application’s business logic. Separating these cross-cutting concerns from the business logic is where aspect- oriented

programming (AOP) goes to work.

Whereas DI helps you decouple your application objects from each other, AOP helps you decouple cross-cutting concerns from the

objects that they affect.

Centralize concerns implementation More reusable code

Cleaner code Write less code

Easy to understand More maintainable

Less boilerplate code More interesting work

Why AOP?

Caching Profiling Security Pooling

Exception Handling Transactions

Logging

Concern

Program execution

Join Points

Advice

Pointcut

Terminology

Aspects are often described in terms of advice, pointcuts, and join points.

Terminology

Advice defines what needs to be applied and when. Jointpoint is where the advice is applied. Pointcut is the combination of different joinpoints where the advice needs to be applied. Aspect is applying the Advice at the pointcuts.

Definitions

Definitions

Method Method Method

Concern

Concern

Advice

Join Points

Logger

TransactionManager

Advice Types

Method

Method

Method

Method

Exception

Before advice

After advice

After returning advice

Around advice

Throws advice

AOP is a PECL extension that enables you to use Aspect Oriented Programming in PHP, without the need to compile or proceed to any other intermediate step before publishing your code.

The AOP extension is designed to be the easiest way you can think of for integrating AOP to PHP.

AOP aims to allow separation of cross-cutting concerns (cache, log, security, transactions, ...)

https://github.com/AOP-PHP/AOP

AOP

You can use pecl

sudo pecl install aop-beta

Installation

<?php class UsersServices { public function authorizeUser () { //some stuff only the admin should do echo "Calling authorizeUser"; }

public function authorizeGroup () { //some stuff only the admin should do echo "Calling authorizeGroup"; } }

Basic tutorial

<?php class UsersServices { public function authorizeUser () { //some stuff only the admin should do echo "Calling authorizeUser"; }

public function authorizeGroup () { //some stuff only the admin should do echo "Calling authorizeGroup"; } }

Basic tutorial

Now you want your code to be safe, you don't want non admin users to be able to call authorize methods.

Basic tutorial

Add some code to check the credentials "IN" you UsersServices class. The drawback is that it will pollute your code, and your core service will be less readable.

Let the clients have the responsibility to check the credentials when required. The drawbacks are that you will duplicate lots of code client side if you have to call the service from multiple places

Add some kind of credential proxy that will check the credentials before calling the actual service. The drawbacks are that you will have to write some extra code, adding another class on the top of your services.

What are your solutions ?

Moreover, those solutions tends to increase in complexity while you are adding more cross-cutting concerns like caching or logging.

What are your solutions ?

That's where AOP comes into action as you will be able to tell PHP to do some extra actions while calling your MyServices's admin methods.

What are your solutions ?

So let's first write the rule needed to check if we can or cannot access the admin services.

<?php function adviceForDoAdmin () { if ((! isset($_SESSION['user_type'])) || ($_SESSION['user_type'] !== 'admin')) { throw new Exception('Sorry, you should be an admin to do this'); } }

What are your solutions ?

Dead simple : we check the current PHP session to see if there is something telling us the current user is an admin (Of course we do realize that you may have more complex routines to do that, be we'll keep this for the example)

What are your solutions ?

Now, let's use AOP to tell PHP to execute this method "before" any execution of admin methods.

<?php aop_add_before('UsersServices->authorize*()', 'adviceForDoAdmin');

What are your solutions ?

Now, each time you'll invoke a method of an object of the class UsersServices, starting by authorize, AOP will launch the function basicAdminChecker before the called method.

What are your solutions ?

<?php //session is started and we added the above examples to configure UsersServices & basicAdminChecker

$services = new UsersServices(); try { $services-> authorizeUser();//will raise an exception as nothing in the current session tells us we are an admin } catch (Exception $e) { echo "You cannot access the service, you're not an admin"; }

$_SESSION['user_type'] = 'admin';//again, this is ugly for the sake of the example

try { $service-> authorizeUser(); $service-> authorizeGroup(); } catch (Exception $e) { //nothing will be caught here, we are an admin }

Logging is an important part of the app development/maintenance cycle.

Logging

To know the best method of logging data of different contexts for specific environments

such as test/dev and production

Take Away

Even with use of computers there was a real need to measure the overall performance of any reasearch

Early 1980's there was a Instrument called VELA (virtual laboratory) used for data harvesting

History of Logging

Late 1980's, A device was invented to collect information through sensors

Later then data logging/harvesting has been used widely in all applications/reasearches/products.

History of Logging

Track Users activity/Movement

Transaction Logging

Track user errors

System level failures/warnings

Research Data collection and Interpretation

Need of Logging

Error / Exception logs

Access logs

System logs

Application logs

Database logs

Transaction logs

Mailer logs etc...

Types of Logging

Apache NGINX

PostgreSQL MySQL

php php-fpm

System Logs

Debug Information - Errors (connections, uncaught exceptions, resource exhaustion)

Narrative Information - Methods Calls, Event Triggers

Business Events - Purchases, Logins, Registrations, Unsubscribes

Application Log

ssh webserver@mydomain.net tail -f /var/log/nginx/my-site.access.log tail -f /var/log/my.application.log

ssh data@mydomain.net tail -f /var/log/mysql/mysql.log

ssh q@mydomain.net

tail -f /var/log/rabbitmq/nodename.log

Keeping Track Of All This

Apache/PHP <VirtualHost *:80>

<Directory /var/www/html/> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all

</Directory> ErrorLog ${APACHE_LOG_DIR}/error.log LogLevel warn CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

Current Conventions

Monolog is a PHP library that support different levels of logging for PHP Applications and depends on PSR.

Inspired by Python Logbook library

Provides stack of handlers

More Powerful than conventional way of logging in applications

Monolog Enters Here

Monolog sends your logs to files, sockets, inboxes, databases and various web services.

Channel based approach

Different stack of handlers for specific channels

Pile up handler stack based on severity.

Format Interpretation depending on severity and channel

Prevents Bubbling when severity is reached

What's different ?

Log Levels 2013 - PSR03 - PHP Logging Interface Standard

Phrase / Severity

emergency Emergency: system is unusable alert Alert: action must be taken immediately critical Critical: critical conditions error Error: error conditions warning Warning: warning conditions notice Notice: normal but significant condition info Informational: informational messages debug Debug: debug-level messages

http://www.php-fig.org/psr/psr-3/

Log Levels

What about Apache’s error_log?

error_log is too basic (message, file, line)

difficult to read / parse

depends on “error_reporting” setting

Why?

monolog

phpconsole

log4php

RavenPHP + Sentry

FirePHP (dev environment)

Roll your own Logging Options

Logging Options

Fire & forget

Minimum or zero latency

Highly available

Should be PSR-3 compatible

Log everything:

- Exceptions - Errors - Fatal Errors

Requirements (for everyone)

Typical PSR-3 Compatible Design

Capture Method

Logger (PSR-3)

Handler / Adapter

Data Storage

MonologMonologErrorHandler ->

handleException()

MonologLogger ->log()

MonologHandler ->handle()

MongoDB

Option to have different channel for different module

Custom detailing

Different handlers for different development

Thorough participation in different stages of lifecycle

Open for third party integration

Readable and Beautiful Layered message

Advantages

PSR-3 makes it easy

However you want…

Monolog has loads:

- syslog-compatible / error_log

- Email, HipChat

- AMQP, Sentry, Zend Monitor, Graylog2

- Redis, MongoDB, CouchDB

Sending Log Messages

CakePHP - https://github.com/jadb/cakephp-monolog Symfony2 - https://github.com/symfony/MonologBundle Slim – https://github.com/flynsarmy/Slim-Monolog Zend2 - https://packagist.org/packages/enlitepro/enlite-monolog CodeIgniter - https://github.com/pfote/Codeigniter-Monolog Laravel – Inbuilt Support. Drupal - https://drupal.org/project/monolog Wordpress - https://packagist.org/packages/fancyguy/wordpress-monolog

more: https://github.com/Seldaek/monolog#frameworks-integration

Do you use Frameworks / CMS ?

Monolog is available on Packagist, which means that you can install it via Composer.

composer require 'monolog/monolog:1.13.*'

Installation

<?php

use Monolog\Logger;

use Monolog\Handler\StreamHandler;

// create a log channel

$log = new Logger('name');

$log->pushHandler(new StreamHandler('path/to/your.log', Logger::WARNING));

// add records to the log

$log->addWarning('Foo');

$log->addError('Bar');

Basic Usage

<?php

use MonologLogger;

use MonologHandlerFingersCrossedHandler;

use MonologHandlerStreamHandler;

$logEnv = getenv('LOG_LEVEL');

$level = empty($logLevel) ? $logEnv : Logger::WARNING;

$appLog = new Logger('AppLog');

$strHandler = new StreamHandler('/var/log/app.log', Logger::DEBUG);

$fcHandler = new FingersCrossedHandler($strHandler, $level);

$appLog−>pushHandler($fcHandler);

$appLog−>debug('LOGGING!');

Loggers And Handlers

<?php

// Set A Log Level $logEnv = getenv(‘LOG_LEVEL');

$level = empty($logLevel) ? $logEnv : Logger::WARNING;

// Create A Logger $appLog = new Logger('AppLog');

Loggers And Handlers

<?php

// Create Handlers $strHandler = new StreamHandler('/var/log/app.log', Logger::DEBUG);

$fcHandler = new FingersCrossedHandler($strHandler, $level);

// Push The Handler And Start Logging $appLog−>pushHandler($fcHandler);

$appLog−>debug('Start Logging!'); $appLog−>emergency('Something Terrible Happened');

Loggers And Handlers

<?php

use Monolog\Logger; use Symfony\Component\EventDispatcher\EventDispatcher;

$dispatcher = new EventDispatcher();

$dispatcher−>addListener( "business.registration.post",

function () use ($busLog) { $busLog−>info("Customer registered");

} );

$dispatcher−>dispatch("business.registration.post");

Event Logging

http://www.sitepoint.com/logging-with-monolog-from-devtools-to-slack/

More usages

Stop logging exceptions the old fashioned way.

The Elk Stack

Indexing and search engine

Near real-time

Distributed, auto-discover clustering

– AWS Plugin

Elasticsearch

Collects logs

Parses, extracts and formats data

Passes data to Elasticsearch

Logstash

example filter { if [file] == "/var/log/secure" and (

[syslog_message] =~ /Invalid user/ or

[syslog_message] =~ /User root from/ ) {

grok {

add_tag => [ "LOGIN" ]

match => {"syslog_message" => “user %{ WORD:username}

from %{IP:srcip}" }

}

}

}

Logstash

Web interface to query Elasticsearch

node.js

Kibana

Kibana

Kibana

WHAT IS REALTIME?

THERE IS ALWAYS A DELAY

HOW MUCH DELAY CAN YOU ACCEPT?

ARCHITECTURE OF DELAY

DATA LIFECYCLE

DATA LIFECYCLE

DATA LIFECYCLE

DATA LIFECYCLE

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

Logstash Architecture

AWS Architecture

Questions?

Thank you.