Agile data presentation 3 - cambridge

WARNINGThis presentation may contain:

‣ Alternattive opinions‣ Mention of phrase "new framework"‣ Mention of paid services and/or software‣ Out-of PHP concepts (Java, Scala, Ruby)

Love and Hatebetween ORM

andQuery

Builders

Myself‣ Love Web Apps‣ 25 years of

coding‣ Entrepreneur‣ I buy Software

My PHP Startup

‣ Helps developers

‣ Open-source‣ PHP in

Enterprise‣ Code Sharing

About Me

Query‣ Control‣ Advanced

Features‣ Multi-Record

ORM‣ 10+ tables‣ SoftDelete,

Audit‣ Domain Model

Database Interaction Today

MapperEntity Query

ORM: How it should work

Database

Business LogicORM

MapperEntity SQL Query

The Reality

SQL Database

Business LogicORM

no ORM

SQL Query

Why Bother…

SQL Database

Business Logic no ORM

Love Compatibility

Query Builder‣ Query‣ Conditions‣ Statements‣ Expressions

ORM‣ Entity‣ Property‣ Operations

(C.R.U.D.)

What do we work with?

Love CompatibilityEntity Property Operations

Query ❤️

Condition

Statements

Expression

ORM Components

Que

ry C

ompo

nent

s

Love CompatibilityEntity Property Operations

Query ❤️ 😔 😔

Condition

Statements

Expression

ORM Components

Que

ry C

ompo

nent

s

Love CompatibilityEntity Property Operations

Query ❤️ 😔 😔

Condition 😐 ❤️

Statements

Expression

ORM Components

Que

ry C

ompo

nent

s

Love CompatibilityEntity Property Operations

Query ❤️ 😔 😔

Condition 😐 😔 ❤️

Statements

Expression

ORM Components

Que

ry C

ompo

nent

s

Love CompatibilityEntity Property Operations

Query ❤️ 😔 😔

Condition 😐 😔 ❤️

Statements 💀 💀 💀

Expression

ORM Components

Que

ry C

ompo

nent

s

Love CompatibilityEntity Property Operations

Query ❤️ 😔 😔

Condition 😐 😔 ❤️

Statements 💀 💀 💀

Expression 💀 💀 💀

ORM Components

Que

ry C

ompo

nent

s

Reimagine a better data access framework. With fresh

start.

Agile Data

Agile DataEntity Property Operations

Query

Condition

Statements

Expression

ORM Components

Que

ry C

ompo

nent

s

Properties

/** @Column(type="datetime", name="posted_at") */private $postedAt;

protected $casts = [ 'is_admin' => 'boolean',];

public function setFirstNameAttribute($value){ $this->attributes['first_name'] = strtolower($value);}

Fields in Agile Data

$book->addField('total_cost', ['type'=>'money']);

Like properties, but smarter.

Property‣ Handy and

Native‣ No type‣ No logic‣ No meta-info

Field‣ Array-access‣ Type enforcing‣ Extensible logic‣ Meta-info

Fields

Fields

class Document extends \atk4\data\Model{ public $table = 'document'; function init() { parent::init(); $this->addField('ref_no', ['type'=>'string']); $this->addField('date', ['type'=>'date']); $this->addField('type', ['enum'=>['invoice','payment']]); $this->hasOne('contact_id', new Contact()); $this->addField('is_archived', ['type'=>'boolean']); }}

Generic UI

$form = new ui\Form();$form->setModel(new smbo\Model_Job($db));

echo $form->render();

Fastest imaginable way to build good looking data entry

form?

Generic UI

Expression

$client->addExpression( 'balance', 'coalesce([total_invoice]-[total_paid],0)');

Define custom SQL code for your field.

Aggregate Field

$client->hasMany('Order') ->addField('total_orders', [ 'field'=>'amount', 'aggregate'=>'sum' ]);

Express related model aggregate as sub-query.

Agile DataEntity Field Operations

Query ❤️

Condition ❤️

Statements ❤️

Expression ❤️

ORM Components

Que

ry C

ompo

nent

s

EntityClient

name address_id Order

titletotal_order

ExpressionSubQuery

hasManyhasOne

6 objects

Is this slow?6

objects 5 rows*

Object representing collection of records

Data Sets

Record Mapping

‣ Active Record and ORM map individual records

‣ Multiple records require Multiple objects in memory

Model_Client object

user type=client

user type=client

user type=client

user type=client user type=admin

user type=admin

table userModel_Client object

DataSet Mapping

‣ Agile Data works with DataSets

‣ Multiple records require Single objects in memory

Model_Client object

user type=client

user type=client

user type=client

user type=client user type=admin

user type=admin

table user

DataSet

Client Admin

User

Conditions

class Client extend User { function init() { parent::init();

$this->addCondition('type', 'client'); }}

Narrow the scope of accessible records

Conditions

$client = new Client($db);$client->load(12);$client['type'] = 'admin';$client->save(); // <== Exception

Prevent human error

Project

DataSet $m = new Project($db);

Project

$m = new Project($db);$m->addCondition( 'client_id', $c_id);

client=$c_id

Project

$m = new Project($db);$m->addCondition( 'client_id', $c_id);

$m->addCondition( 'is_cancelled', false);

client=1client=$c_id

is_cancelled = "N"

Conditions

$client = new Client($db);$client->load($_GET['user_id']);$project = $client->ref('Project');$project->insert($_POST);

Prevent human error

Conditions

$profitable_client = new Client($db);$profitable_client->addCondition($this->expr( '[income]>[expense]'));$data = $profitable_client->export();

Dynamically add expressions for reports.

Deep Traversal

foreach( $author->withID(20) ->ref('App') ->ref('Review') ->addCondition('rating', 1) as $bad_review ) {

$bad_review->sendApologyEmail(); }

Traverse without querying

QRM->Q.B.‣ Static scope‣ Limited

inheritance‣ Persistence layer

DataSet Model‣ Dynamic scope‣ Fully OOP‣ References infer

conditions‣ Domain layer

Model

Agile DataModel(DataSet) Field Operations

Query ❤️ ❤️

Condition ❤️ ❤️

Statements ❤️ ❤️

Expression ❤️ ❤️

ORM Components

Que

ry C

ompo

nent

s

ORM limit multi-record features and other database

capabilities.

N+1 problem

foreach (Client::all() as $client) { foreach($client->orders() as $order){ $sum += $order->amount; }}

N+1 problem

‣ 20 users in database

‣ 20 extra queries to fetch orders

foreach (Client::all() as $client) { foreach($client->orders() as $order){ $sum += $order->amount; }}

Memory Problem

‣ 20 users in database

‣ array of 20 objects

‣ other properties are also fetched

foreach (Client::all() as $client) { foreach($client->orders() as $order){ $sum += $order->amount; }}

Lazy-loading; ID Stuffing

‣ only load IDs

‣ trigger loading on property access

‣ select from `order` where user in (1,2,3,…)

Action

$client = new Client($db);$sum = $client->ref('Order') ->action('fx', ['sum', 'amount']) ->execute();

Event from Domain Model to your database

Multi-record operations

$author->withID(20) ->ref('App') ->ref('Review') ->addCondition('rating', 1) ->action('delete') ->execute();

Delete all entries in data-set

Multi-record operations

$author->withID(20) ->ref('App') ->ref('Review') ->addCondition('rating', '<', 3) ->action('update') ->set('rating=rating+1') ->execute();

or just update rating

Aggregation

$q = $client->action('field', ['industry']);

$q->field('count(*)', 'c');$q->group('industry');

$data = $q->get();

Create any query around Model

Union

$q_inv = $client->ref('Invoice') ->action('field',['date','total']);

$q_pay = $client->ref('Payment') ->action('field',['date','paid']);

foreach($db->expr( 'select * from ([] UNION []) u1', $q_inv, $q_pay) as $row) { /// }

Prepare client statement

ORM‣ Emit query-

builder‣ Limited scope

support‣ SQL only‣ Fully Persistence

logic

Agile Actions‣ Emit DSQL‣ All conditions

applied‣ Extensible to

NoSQL‣ Domain model

support

Multi-record operations

Agile DataModel(DataSet) Field Operations

Query ❤️ ❤️ ❤️

Condition ❤️ ❤️ ❤️

Statements ❤️ ❤️ ❤️

Expression ❤️ ❤️ ❤️

ORM Components

Que

ry C

ompo

nent

s

Summary‣ ORM and Query Builders has a bumpy

relationship‣ Luckily there are other patterns‣ Consider query latency‣ Consider flexibility‣ Consider abstraction

Execute less queries

Design a beautiful API

Friendly with UIExtensions

Agile Data

Download

Result

Make UI better

git.io/ad

✓ Works in any framework / PHP app✓ Lightweight and Agile✓ Integrates with UI frameworks (like

ATK)✓ Commercial Support

Agile Data is Open-Source

Join my OSS project

‣ Agile Data v1.1.5git.io/ad

‣ Agile UI (developing)github.com/atk4/ui

My other topics‣ Agile Toolkit

Full-stack PHP UI

More examplesif there is time

Query into Field

$client->addExpression('last_sale', function($m){ return $m->refLink('Invoice') ->setLimit(1) ->setOrder('date desc') ->action('field',['total']); }, 'type'=>'money');

Convert Entity into Query then use inside Field / Expression.

Join on Expression

$invoice->hasOne('client_id', new Client()) ->addField('client_country_id', 'country_id');

$invoice->join('country', 'client_country_id') ->addFields([ 'country_short_code'=>'short_code', 'country_is_eu'=>'is_eu', 'country'=>'name' ]);

If you need country data inside your Invoice report, but country_id is defined through

Client.

Aliasing

Self-referencingclass Folder extends \atk4\data\Model{ public $table = 'folder'; public function init() { parent::init(); $this->addField('name');

$this->hasMany('SubFolder', [ new Folder(), 'their_field'=>'parent_id'] )->addField('count', [ 'aggregate'=>'count', 'field'=>$this->expr('*')] );

$this->hasOne('parent_id', new Folder()) ->addTitle(); }}

Unique Aliases

select `id`,`name`,

(select count(*) from `folder` `S` where `parent_id` = `folder`.`id`) `count`,

`folder`.`parent_id`,

(select `name` from `folder` `p` where `id` = `folder`.`parent_id`) `parent` from `folder`

✓ ACL, Audit, Undo, Logging, ..✓ File Management✓ Import/Export utility✓ RestAPI server

Commercial Extensions

git.io/ad