Integrating Salesforce with Business Intelligence & Data WarehousesTim Deagan, CORT Business Services@TimDeagan

All about CORT Business Services. A Berkshire Hathaway Company

CORT, a Berkshire Hathaway Company, is the world’s largest provider of rental furniture and much, much more. When life creates transition, CORT swings into action. Whether you are relocating permanently or temporarily and in need of a place to live, furnishings and local knowledge, a business needing swing space furnishings or a new identity, or just looking for total flexibility and lifestyle options, CORT finds innovative solutions that make practical sense. When you have a special event, as a company or individual, like a corporate gala, wedding, trade show, product launch or company meeting, CORT is there to make your event special and memorable. .

▪ ~1000 Salesforce licenses

▪ Customer for 5 years

▪ Force.com active developers

▪ 59 Custom Objects, ~650,000 API requests/day

Salesforce data is great in the Cloud, until you need it down to earth for…

▪ Enterprise Data Integration• Cross System Reporting• Moving Data within the Enterprise

▪ Reporting across more objects that Salesforce supports• More than four object relationships (~= JOINS)

▪ Advanced Historical Data Queries

Bring your Salesforce data into an RDBMS▪ Query and Report with your internal tools

• SQL, not SOQL• ODBC to MS Excel, MS Access, etc.

▪ Populate your Data Warehouse with Salesforce fact data• Objects become raw tables• ETL into ROLAP, OLAP, HOLAP, MOLAP, DOLAP

CORT uses DBAmp

http://www.forceamp.com/

DBAmp Functionality▪ A SQL Server Linked Server to Salesforce production and sandbox

instances via HTTPS▪ Real-time CRUD access▪ Replication procedures to maintain local copies of Salesforce data▪ Uses Salesforce API security▪ Salesforce org based license (usable on unlimited servers)▪ Great on-line support by the author

Two methods for real-time data accessSQL & SOQL

▪ SQL (technically Transact SQL or T-SQL)• Uses ‘4-part’ naming convention

– ServerAlias.Catalog.Schema.LinkedTable – Salesforce…Contact (Linked_Server_Name…Table)

• PRO– Allows full range of SQL manipulation

• CON– Pulls entire table into memory prior to executing WHERE clause or SQL

functions

Two methods for real-time data access SQL & SOQL

▪ SOQL (technically SOQL via OPENQUERY)• OpenQuery executes the passed query on the linked server• PRO

– Higher performance

• CON– To pass parameters, you must use a stored procedure that builds the query

string and executes it

CRUD accessCREATE & READ

▪ Create == Insert (Tables cannot be created)• INSERT INTO Salesforce...Note (Body, IsPrivate, ParentId, Title) VALUES

('Body of Note 2','false', '00130000005ZsG8AAK','ToDelete')• Stored procedure sf_bulkops provides batched bulk inserts

▪ Read == Select• Select * from Salesforce...Contact• Select * from openquery(Salesforce, ‘Select Id, CreatedDate, CreatedById

from Account’)

CRUD accessUPDATE & DELETE

▪ Update• Update Salesforce…Account Set AnnualRevenue = 4000 Where

Id='00130000005ZsG8AAK'• Update OPENQUERY(Salesforce, 'Select Id, AnnualRevenue from Account

where Id=''00130000005ZsG8AAK'' ') set AnnualRevenue = 4000

▪ Delete (Rows, not tables)• Delete from Salesforce…Account Where Id='00130000005ZsG8AAK'• Delete openquery(Salesforce,''Select Id from Account where

Id=''00130000005ZsG8AAK'' ‘)

Replicating Salesforce data locallyIncremental scheduled updates (hourly, daily, weekly, etc.)

▪ Stored Procedures for REFRESH and REPLICATE• SF_Refresh brings over changes• SF_Replicate brings over complete tables

▪ Schema changes are automatically detected and brought over▪ Archived and deleted rows can be pulled over▪ DRAMATICALLY faster queries than real time SQL or SOQL▪ Local backup of Salesforce

How we use this system at CORT

▪ Tech Stack• SQL Server 2008 R2, SQL Server 2012• SQL Server Reporting Services (SSRS)• SQL Server Integration Services (SSIS)

▪ ROLAP (Relational Online Analytical Processing) Environment• No cubes • Nightly jobs crunching data into user friendly tables and views (some de-

normalization)• SSRS reports providing CSV for Excel usage as well as straight reporting• Combines data from ERP, CRM, eCommerce, Call Center and Financials

CORT Architecture

Examples at CORTFind and delete unnecessary custom fields

▪ Created “Field Usage” reports ▪ Developed a ‘hit list’ and eliminated hundreds of unnecessary fields,

merging the data into existing fields as appropriate

Emergency restores▪ Over-eager Salesforce bulk loader user crufted 10,000+ records▪ We restored the relevant data from a SQL Server backup

Examples at CORTUpload data from other systems

▪ Revenue data from our National Accounts portal needed to be linked to Salesforce accounts

• SQL was written to get the National Account data, cross link it to the Salesforce Account ID and upload the revenue nightly to Salesforce so salespeople can see it

▪ Data from our HR system is used to mod or delete User records in Salesforce.

• Systems are kept in sync with nightly job

Examples at CORTSalesforce as a CMS

▪ Our District Policies are maintained by the Districts in Salesforce

• SSRS Report publishes the Policies to Truck Drivers and Warehouse workers without Salesforce licenses

Examples at CORTCreation of complicated associations

▪ Our call center gets sales credit for the leases they ‘touch’▪ This is determined by any one of a number of activities▪ Requires determining if they were the Opp owner at any point in the

past▪ We created a View in SQL Server that identifies all the Opportunties

that they get credit for touching…

Were they ever the opp owner?

Were they the initial opp owner?

Did they ever own the lead that became the opp?

Did they create the lead that

became the opp?

We wanted to segregate a line of business into a new Salesforce instance/org.

▪ Extract all relevant Salesforce data (opps, leads, contacts, accounts, attachments, etc.) for the LOB into SQL Server

▪ Create tables of all ID based relationships• 415 fields containing keys

▪ Upload records to the new org▪ Find new object IDs▪ Change all references to the old IDs to the new IDs in SQL Server▪ Update records with correct relationship IDs

Examples at CORT

How much data are we talking about?

dbo.Account dbo.CallCenter dbo.ContractStatus dbo.Exhibit_House_Client_del_del__History dbo.Partner dbo.SelfServiceUserdbo.AccountContactRole dbo.Campaign dbo.Corporate_Housing__c dbo.FeedComment dbo.PartnerRole dbo.SetupEntityAccessdbo.AccountFeed dbo.CAMPAIGNCAL__Campaign_Color__c dbo.Corporate_Housing__Share dbo.FieldPermissions dbo.Period dbo.SFDC_520_Approval_Request__cdbo.AccountHistory dbo.CampaignFeed dbo.CORT_Location__c dbo.FiscalYearSettings dbo.PermissionSet dbo.SFDC_520_Quote__cdbo.AccountPartner dbo.CampaignMember dbo.CORT_Location__History dbo.Folder dbo.PermissionSetAssignment dbo.SFDC_520_QuoteLine__cdbo.AccountShare dbo.CampaignMemberStatus dbo.CORT_Showroom__c dbo.ForecastShare dbo.Pricebook2 dbo.SFGA__Ad_Group__cdbo.AccountTeamMember dbo.CampaignShare dbo.Coupon_Code__c dbo.General_Contractor__c dbo.Pricebook2History dbo.SFGA__Ad_Group__Sharedbo.AdditionalNumber dbo.Case dbo.Coupon_Code__History dbo.General_Contractor__History dbo.PricebookEntry dbo.SFGA__Google_Campaign__cdbo.ApexClass dbo.CaseComment dbo.Credit__c dbo.Group dbo.ProcessInstance dbo.SFGA__Google_Campaign__Sharedbo.ApexComponent dbo.CaseContactRole dbo.Credit__History dbo.GroupMember dbo.ProcessInstanceStep dbo.SFGA__Keyword__cdbo.ApexLog dbo.CaseFeed dbo.CronTrigger dbo.HashtagDefinition dbo.ProcessInstanceWorkitem dbo.SFGA__Keyword__Sharedbo.ApexPage dbo.CaseHistory dbo.CurrencyType dbo.Holiday dbo.product_issue_tracker__c dbo.SFGA__Search_Phrase__cdbo.ApexTestQueueItem dbo.CaseShare dbo.Customer_Profile__c dbo.Housing_Market__c dbo.product_issue_tracker__History dbo.SFGA__Search_Phrase__Sharedbo.ApexTestResult dbo.CaseSolution dbo.Customer_Profile__History dbo.Housing_Market__Share dbo.product_sku__c dbo.SFGA__Text_Ad__cdbo.ApexTrigger dbo.CaseStatus dbo.Dashboard dbo.Idea dbo.product_sku__History dbo.SFGA__Text_Ad__Sharedbo.ApplicationNameValue__c dbo.CaseTeamMember dbo.DashboardComponent dbo.IdeaComment dbo.Product2 dbo.SFGA__Version_33_0_3__cdbo.Approval dbo.CaseTeamRole dbo.DashboardComponentFeed dbo.Lead dbo.Product2Feed dbo.SFGA__Version_33_0_3__Sharedbo.APXT_CMTR__Conga_Merge_Template__c dbo.CaseTeamTemplate dbo.DashboardFeed dbo.LeadFeed dbo.Profile dbo.Sitedbo.APXT_CMTR__Conga_Merge_Template__Share dbo.CaseTeamTemplateMember dbo.DatedConversionRate dbo.LeadHistory dbo.Profile_Contact_Role__c dbo.SiteDomaindbo.Asset dbo.CaseTeamTemplateRecord dbo.DFSS_SnapShot_Item__c dbo.LeadShare dbo.Profile_Contact_Role__History dbo.SiteFeeddbo.AssetFeed dbo.CategoryData dbo.DFSS_SnapShot_Push__c dbo.LeadStatus dbo.Project__c dbo.SiteHistorydbo.AssignmentRule dbo.CategoryNode dbo.District_Policy__c dbo.Lease_Types__c dbo.Project__History dbo.Solutiondbo.AsyncApexJob dbo.ChatterActivity dbo.District_Policy__History dbo.Lease_Types__Share dbo.purchase_order__c dbo.SolutionFeeddbo.Attachment dbo.ClientBrowser dbo.Document dbo.LineitemOverride dbo.purchase_order__History dbo.SolutionHistorydbo.AuthProvider dbo.CMC_Services__c dbo.DocumentAttachmentMap dbo.LoginHistory dbo.PushTopic dbo.SolutionStatusdbo.Blacklist_Audit__c dbo.CMC_Services__History dbo.Domain dbo.LoginIp dbo.QtlContactRole__c dbo.StaticResourcedbo.Blacklisted_Word__c dbo.CollaborationGroup dbo.DomainSite dbo.MailmergeTemplate dbo.QuantityForecast dbo.TableRefreshTimedbo.BLND_DFAC_Shared_View__c dbo.CollaborationGroupFeed dbo.dsfs__DocuSign_Envelope__c dbo.Market__c dbo.QuantityForecastHistory dbo.Taskdbo.BLND_DFCS_Carousel__c dbo.CollaborationGroupMember dbo.dsfs__DocuSign_Envelope_Document__c dbo.Market__Share dbo.QueueSobject dbo.TaskFeeddbo.BLND_DFCS_Slide__c dbo.CollaborationGroupMemberRequest dbo.dsfs__DocuSign_Envelope_Recipient__c dbo.National_Account__c dbo.Quote dbo.TaskPrioritydbo.BLND_DFDT_Action_Item__c dbo.CollaborationInvitation dbo.dsfs__DocuSign_Recipient_Status__c dbo.National_Account__History dbo.Quote_To_Lease__c dbo.TaskStatusdbo.BLND_DFDT_Discussion__c dbo.College__c dbo.dsfs__DocuSign_Recipient_Status__History dbo.NewsFeed dbo.Quote_To_Lease__History dbo.Topicdbo.BLND_DFDT_Highlight__c dbo.College__History dbo.dsfs__DocuSign_Status__c dbo.Note dbo.QuoteDocument dbo.Userdbo.BLND_DFDT_Issue__c dbo.Community dbo.dsfs__DocuSign_Status__History dbo.ObjectPermissions dbo.QuoteLineItem dbo.User_Loaddbo.BLND_DFDT_Linker__c dbo.Contact dbo.dsfs__DocuSignAccountConfiguration__c dbo.Opportunity dbo.RecordType dbo.UserAccountTeamMemberdbo.BLND_DFDT_Project__c dbo.ContactFeed dbo.EmailMessage dbo.OpportunityCompetitor dbo.Referral_Payment_Account__c dbo.UserFeeddbo.BLND_DFDT_Resource__c dbo.ContactHistory dbo.EmailServicesAddress dbo.OpportunityContactRole dbo.Referral_Payment_Account__History dbo.UserLicensedbo.BLND_DFDT_Task__c dbo.ContactShare dbo.EmailServicesFunction dbo.OpportunityFeed dbo.Referral_Payment_Transaction__c dbo.UserPreferencedbo.BLND_DFFF_Form__c dbo.ContentDocument dbo.EmailTemplate dbo.OpportunityFieldHistory dbo.Referral_Payment_Transaction__History dbo.UserRoledbo.BLND_DFFF_Page__c dbo.ContentDocumentFeed dbo.EntitySubscription dbo.OpportunityHistory dbo.Referral_Payment_Vendor__c dbo.UserTeamMemberdbo.BLND_DFTE_Expense__c dbo.ContentDocumentHistory dbo.Evaluation_Phone__c dbo.OpportunityLineItem dbo.Referral_Payment_Vendor__History dbo.VASSnapsdbo.BLND_DFTE_Time__c dbo.ContentVersion dbo.Evaluation_Phone__History dbo.OpportunityOverride dbo.Report dbo.vendor__cdbo.BLND_DFTT_Document__c dbo.ContentVersionHistory dbo.Event dbo.OpportunityPartner dbo.ReportFeed dbo.vendor__Historydbo.BLND_DFTT_Entity_Info__c dbo.ContentWorkspace dbo.EventAttendee dbo.OpportunityShare dbo.RevenueForecast dbo.Venue__cdbo.BLND_DFTT_Folder__c dbo.ContentWorkspaceDoc dbo.EventFeed dbo.OpportunityStage dbo.RevenueForecastHistory dbo.WebLinkdbo.BrandTemplate dbo.Contract dbo.EventRelation dbo.OpportunityTeamMember dbo.Scontrol dbo.xtma_Email_Result__cdbo.BusinessHours dbo.ContractContactRole dbo.Events__c dbo.Organization dbo.screenstepslive__ScreenStepsLiveSettings__c dbo.xtma_Individual_Email_Result__cdbo.BusinessProcess dbo.ContractFeed dbo.Events__History dbo.OrganizationProperty dbo.screenstepslive__ScreenStepsLiveTags__c dbo.xtma_Link_Detail__c

dbo.ContractHistory dbo.Exhibit_House_Client_del_del__c dbo.OrgWideEmailAddress

25GB of data in lots of Tables!

DBAmp creates all the Foreign Key constraints that allow diagramming.

However, pulling all the tables into an ERD makes for a VERY LARGE diagram!

Tim DeaganCORT Business ServicesExecutive Director, Product

Architecture@TimDeagan