App auto crud

10.04.2023 - Page 1

DépartementOfficeApp::AutoCRUD

An application for searching and manipulating data in relational

databases

YAPC::EU::2014

laurent.dami@justice.ge.chEtat de Genève, Pouvoir Judiciaire

DépartementOffice

Agenda

• Introduction• Overview (screenshots)• Getting started• Architecture• Extensibility• Conclusion

10.04.2023 - Page 1

DépartementOffice

Introduction

Business need : tool for the support team

RegularUsers

database

RegularApp

DatabaseAdmins

AdminTools

SupportTeam

? ? ?

Data access for support team

Needs• Browse DB schema• CRUD on all data• Navigate

– Follow relationships– Optimize for most

common tables and colums

Solutions• Program an admin

GUI– too much work !

• Give them DBA tools– too low-level– no customization

Need a CRUD app !

CRUD landscape

Admin toolsOracle SQLDeveloperSQLite SqlitemanPostgres pgadminphpMyAdmin...

Readyto use

Needsprogramming

Fixedfeatures

Customizable

Scaffolding toolsRuby on RailsCatalystCatalystX::CRUD...

AutoCRUD appsCatalyst::Plugin::AutoCRUD (2011)App::AutoCRUD (Jan 2014)RapidApp (Feb 2014) [ WebAPI::DBIC ] (Jul 2014)Interchange TableEditor (2013??)

App::AutoCRUD main features

• Powerful search syntax• Distinctive URL for every resource

– no session, all information in URL– easy admin links from the "real App"

• Joins as hyperlinks• Bulk update / delete operations• Customizable table order / column order• Navigation through tree navigator

10.04.2023 - Page 1

DépartementOffice

Quick overview

To run the demo :cd examplesperl download_db.plplackup

Homepage

Specified in config file

http://chinookdatabase.codeplex.com/https://code.google.com/p/sakila-sample-database-ports/

Demo: free sample

databases

Datasource entry point

Auto-generateddatamodel

Grouping, order & comments

from config file

Table description

Grouping, order & comments

from config file

Hyperlinks to foreign tables

from DB schema

Search formMultiple values

(OR)

LIKE

Multiple views

Choose columns

BETWEEN

Results from search

Hyperlinks to foreign records or lists

Only show non-null columns

Also navigate through LEFT/RIGHT arrow keys

Excel output

YAML output

Bulk update (selective)

2) Call the update form

1) Check records to be modified

Update form

Keys of selected records

Bulk update (from WHERE criteria)

Call the update form

Update form

Criteria for the update

Fields to update

± RESTful URLs

• /Chinook/table/Track/search?GenreId=1,3&Name=Ab*– GET display pre-filled search form– POST redirect to list

• /Chinook/table/Track/list?GenreId=1,3&Name=Ab*– search results

• /Chinook/table/Track/id/399– single record

• /Chinook/table/Track/delete?where_pk.TrackId=399– GET display pre-filled delete form– POST database operationNote : browser-oriented, not API-oriented

No use of HTTP methods PUT and DELETE

10.04.2023 - Page 1

DépartementOffice

Getting Started

YAML Config : connection parameters

app: name: Test AutoCRUD

datasources : Source1 : dbh: connect: - dbi:SQLite:dbname=some_file - "" # user - "" # password - RaiseError : 1 sqlite_unicode: 1

Config : specifying table grouping / ordering

tablegroups : - name: Music descr: Tables describing music content node: open tables : - Artist - Album - Track

- name: Playlist ...

Config : specifying col grouping / ordering

Employee: colgroups: - name: keys columns: - name: EmployeeId descr: primary key - name: ReportsTo descr: foreign key to the manager - name: Employee details columns: - name: Title - name: FirstName - name: LastName

"crud.psgi" startup file

use App::AutoCRUD;use YAML qw/LoadFile/;

my $config = LoadFile "/path/to/config.yaml";my $crud = App::AutoCRUD->new(config => $config);

my $app = $crud->to_app;

Start the application

• From the command lineplackup crud.psgi

• Within Apache<Location /crud>

SetHandler perl-scriptPerlResponseHandler Plack::Handler::Apache2 PerlSetVar psgi_app /path/to/crud.psgi

</Location>

• Other possibilities : see L<Plack>

10.04.2023 - Page 1

DépartementOffice

Architecture : external modules

External modules : Global view

TemplateToolkit

Alien::GvaScript

Prototype.js

Excel::Writer:

:XLSX

View

DBIx::DataModel

DBI

SQL::Abstract::More

Model

Plack

CGI::Expand

SQL::Abstract:

:FromQuery

Controller

Infrastructure

MooseData::Domai

nYAML

Infrastructure modules

• Moose for OO– Modern Perl OO framework

• YAML for config data– Human-readable– Support for directed acyclic graph (DAG) reusable subtrees– But can be replaced by JSON, XML, Config::General or Perl

source code

• Data::Domain for data validation

Controller modules

• Plack for HTTP support– Abstraction over various engines– Many middleware options

• CGI::Expand for transforming form inputs – <input name="foo.1.bar" value=1234>– { foo => [ {...}, { bar => 1234 } ] }

• SQL::Abstract::FromQuery for parsing user queries– HTML form SQL::Abstract::More SQL– Parses booleans, < > !=, LIKE, BETWEEN, etc.

Model modules

• DBI for database independance– Perl standard for databases

• DBIx::DataModel for object-relational mapping – Why not DBIC ? See

http://www.slideshare.net/ldami/dbixclass-vs-dbixdatamodel

• SQL::Abstract::More for SQL generation– Emit SQL from Perl datastructures

View modules

• Template Toolkit for HTML generation

• Alien::GvaScript for widgets – Tree::Navigator

• Prototype.js for Javascript abstraction– Navigator abstraction, OO programming, etc.

• Excel::Writer::XLSX for generating Excel worksheets

10.04.2023 - Page 1

DépartementOffice

Architecture : internal structure

Internal structure : global view

AutoCRUD

AutoCRUD:

:View

AutoCRUD:

:Controller

AutoCRUD:

:Context

AutoCRUD:

:DataSource

Plack::Component

Moose::Object

Controller::Table

Controller::Schema

View::XLSX

View::TT

...

App::AutoCRUD

• Entry point• Autodiscovers component subclasses• isa Plack::Component• Has

– Name– Config– Datasources

App::AutoCRUD::DataSource

• Encapsulates DBIx::DataModel schema– Autogenerate Perl code if not already present

• Gathers and merges metadata– From the DB schema

• Tables, columns, foreign keys– From config

• Table groups, column groups, ordering, display parameters

App::AutoCRUD::Context

• Gathers contextual information for the current request– Plack::Request object– Datasource object– View to apply– Parameters for the view (for example TT template name)

Controller::Schema

• Lists table groups & tables• Publishes the DBIx::DataModel schema

Controller::Table

• Implements CRUD operations• Methods

– Descr– List– Id– Search– Update– Delete– Insert

View::*

• Render results– TT– XLSX– YAML– JSON– ...

10.04.2023 - Page 1

DépartementOffice

Extensibility

Regular subclassing

• Extend the apppackage My::Private::AutoCRUD;use Moose;extends 'App::AutoCRUD';

• Subclass and redefine existing methods– Automatic subclass discovery– Good candidates :

• DataSource::prepare_for_request • DataSource::_query_parser

Adding URLs

• New controllers– Crud/<db_name>/<controller_name>/....

• New methods in existing controllers– Crud/<db_name>/table/<table_name>/<method>/...

Adding Views

• Create View/XYZ.pm with "render" method– Serves /path/to/resource.xyz?p1=v1&p2=v2&...

Relooking

• Override CSS – My/Private/AutoCRUD/share/static/css/styles.css

• Override TT templates – My/Private/AutoCRUD/share/templates/src/table/list.tt

10.04.2023 - Page 1

DépartementOffice

Conclusion

Status

• Still young, lots of TODO• Already in production at Geneva Justice• Has some original features not seen other CRUD

apps• Could be useful for many people

– Also outside the Perl community • Perl marketing needs general-purpose apps

– Needs stabilization & doc– Needs marketing– You can help !