Upload
others
View
38
Download
0
Embed Size (px)
Citation preview
Kristian Ducharme
BADCamp 2016
BFFs 4eva: Drupal 8 and Custom Databases
Introduction
■ Kristian Ducharme - Senior Engineer / Technical Lead @ CivicActions
■ Past Projects: City of Los Angeles, Acquia Projects
■ Development / Drupal Background
Project Background
https://www.digitaldemocracy.org
■ California Legislative Hearings, ML Transcription, Data Aggregation
■ Custom MySQL Database - 100+ tables (DDDB)
■ DB Team supporting schema, data import, transcription
■ Growth Plans - Multi-state, Tiered offerings, Sites for Advocacy Groups + Organizations, Campaigns, Advanced Search
Drupal / Data Architecture
Current D7 Drupal Architecture
■ DDDB is located on same server, connected via read-only MySQL user
■ Nightly import through migration scripts
■ Data imported into fielded nodes with its own structure, incorporating some similarities to DDDB schema
■ Drupal content types are loosely connected via ID fields, not utilizing entity references
Current D7 Issues
■ Synchronization of data between DDDB and Drupal has to be maintained
■ Duplication of data
■ Keeping DDDB an independent DB important
■ Drupal features limited due to architecture (Views / Content Admin / Search)
■ Schema changes in DDDB require 2-3 layers of Drupal changes / testing
=
DDDB Schema
D7 or D8?
Use Cases for D8 External Custom Entities
■ Complex External Relational Databases
■ Too much changing data for periodic migration
■ DB Schema with intricate relationship, foreign key constraints
■ Read-only data sources, need to tie with Drupal content types
Advantages Disadvantages
Advantages
■ Gain all Drupal 8 functionality on dataset
■ Fieldable Entities■ Options for headless
Drupal/dataset APIs■ Search API / Facets■ No import/migration process■ Real-time data■ No Data Duplication■ External DB Replication■ High level of
customization/overrides
Disadvantages
Advantages
■ Gain all Drupal 8 functionality on dataset
■ Fieldable Entities■ Options for headless
Drupal/dataset APIs■ Search API / Facets■ No import/migration process■ Real-time data■ No Data Duplication■ External DB Replication■ High level of
customization/overrides
Disadvantages
■ Complexity of Views■ Configuring Relationships■ Heavy initial development to
establish Custom Entities■ More Performance Tuning
Required■ Requires External DB to be on
same server as Drupal DB■ For local dev, requires copy
of external DB
D8 Custom Entities In Action
Addressing Drupal DB/Table Needs
Possible Issues with External DB■ Multiple Primary Keys, Non-Integer Primary Keys■ Relationship tables (all become entities?)■ DB Not On Same Server as Drupal
Solutions■ Adding new unique ID column to tables■ Generated Columns (MySQL 5.7.6+)■ Computed Fields■ SQL Views, Custom Entity built on views■ Database Replication to sync w/Drupal server
Custom Entity Creation Process
■ Creating Custom Modules To Hold Entities
■ Mapping Tables/Columns to Entities/Fields
■ Drupal Console drupal generate:entity:content
■ Adjusting Annotations
■ Getters/Setters for all fields
■ Adding fields to baseFieldDefinitions
■ Adding Table Prefix to settings.php
Custom Entities Annotation
Custom Entities Services
■ Overriding Uninstall Services
ContentUninstallValidator Override
Custom Entities Class Inheritance
■ DB Tables mapped to entities w/common fields (state, status, location, etc)
■ Extending “base” entity class(es) to provide common functionality to many entities
■ Use of custom SqlContentEntityStorage
“State” Filtering Example on Entity
/ Views
Use of “State” field in
DdBaseStateField Class
Hearing Entity extending
DdBaseStateField Class
Use Of Custom SqlContentEntityStorage Handlers
dd_base.views_execution.inc - hook_views_query_alter() to add State filtering
Custom Entities w/Views
■ Post Drupal Console generation, ready w/only relationships defined in Entity baseFieldDefinitions
■ Modifying xxxxViewsData.php(Reference: views.api.php hook_views_data)
■ Ability to add relationships to tables that have no custom entities
■ Support for table joins w/multiple conditions
Drupal\dd_hearing\Entity\DdHearing
Views “Add Relationships” for Hearing
DdPersonViewsData - “Add Relationships” for Person
Base Table Name
Unique ID For Relationship Column
Table to join
Field to join on
Field on Person Table to join
Resulting SQL Query:SELECT Person.pid AS pid, Legislator_Person.pid AS legislator_person_pid FROM {Person} Person INNER JOIN {Legislator} Legislator_Person ON Person.pid = Legislator_Person.pid
Views “Add Relationships” for Person
Limitation Of Views (w/o Search API)
■ Handling sitewide search
■ Performance
■ No Facets
■ No Big Data Tools
Custom Entities w/Search API
■ Entity Reference fields automatically available
■ Computed Fields allow data to be added to Search Index during entity load
■ Patch to allow computed fields to be visible: https://www.drupal.org/node/2814925
■ Faceted Search on Views built on Search Indexes
dd_hearing.moduleSet Computed Field On Entity Storage Load
Search API Index Fields
The End Result
Q & A
Contact Information
Kristian [email protected]
Drupal.org: kducharm GitHub: kducharm
BFFs 4eva: Drupal 8 and Custom Databases
http://www.civicactions.com