Big Data Management with Blackbaud CRM

  • Published on

  • View

  • Download

Embed Size (px)


By David Tunesi



2. Data and Blackbaud CRMDavid Tunesi2 #bbcon 3. Introducing ALSAC / St. JudeAlan Harrison3 #bbcon 4. Challenges we Face Migrating from Team Approach to BBCRM Migrating into existing BBCRM (Contact Center) Load large data with active users Initial loader and Iterative loader will be the same logic Close to 40 million Individual accounts in Team Approach 2 Billion Interactions 800 Million Transactions 700 TA Users Phased User Rollout by division No Big Bang cutover Minimize BBCRM extensions Data Mappings must be configurable4 #bbcon 5. ImplementationClyde Jarvis5 #bbcon 6. Keys to our Success Business and Resources Supplemented mapping and development resources from BlackbaudProfessional Services Set up Data Governance Council to help make data decisions Set up Data Stewardship group to research and fix data Brought in a Change Enablement expert to help with UserOnboarding6 #bbcon 7. Important Considerations for All Purged and Cleaned as much data as possible before migration Removed 800 million interactions Ideally delete duplicate constituents if possible We were not able to do this because of slow deletes We chose to compress the database After compression, the database size was cut in half to 3.2 Terabytes Constituent Deletes were very slow after we loaded large amountsof data We put in a custom trigger on database to turn hard deletes to softdeletes Blackbaud Product is working on a long-time solution. We had performance issues around revenue rollup Worked with Blackbaud Support and they provided patches to fixsome of these Try to stay upgraded on the latest version if possible. Some of these performance gains were in the new release.7 #bbcon 8. Not for the Faint of Heart For large systems, consider only migrating constituents that areneeded Filter based on Constituent Ranking (RFM or some other formula) On-Demand loader Will Show some slides on this later Service integration to pull over constituent data from TA whenrequested. New accounts in TA get automatically migrated BI system has all history and constituents for campaigns andanalysis Replicating large volumes between Local and DR site createdchallenges We had to slow down our initial loads to allow replication to stay up-to-date. TA and DMS are both up at the same time with data syncingbetween each Necessary for us because of slow user rollout Extremely difficult to maintain This can create extra volume for Replication to DR Site saturatingnetwork8 #bbcon 9. Not for the Faint of Heart - Services Stand up Master Data Management solution to help facilitateintegration MDM has all system constituent keys and attribute keys MDM controls constituent matching and merge MDM makes On-Demand Loader possible using exposed servicelayer. Service layer on top of BBCRM API using BBCRM Framework 3 methods that handle all constituent requests editConstituent createConstituent mergeConstituent Loaded all constituents via Service API. 17 threads max running in parallel with users on the system. At 17 we started having some concurrency issues. We stabilized at 10 to 12 threads9 #bbcon 10. MDM Constituent Search and CopySearch MDM and BBCRM and merge results 11. MDM Constituent Search and CopyPreview the Constituent from MDM before and press Copy 12. MDM Constituent Search and CopyConstituent is saved in BBCRM 13. MDM Constituent Search and CopyAll other constituent attribution flows over to BBCRM 14. In conclusion14 #bbcon 15. Summary Future Considerations Would like to consider archiving solutions in BBCRM Convert Constituents to another type (underway/delivered) Bulk deletion of Constituents More direct MDM integration with BBCRM and less batch (Hydra) Working with Blackbaud ALSAC and Services meeting regularly (Tech Summits) Coordinating across groups and departments at Blackbaud Informing product needs and highlighting performance bottlenecks Data conversion considerations and strategy15 #bbcon 16. Did this session give you the jolt youneeded?Dont forget to completea session survey!Each completed survey enters you into a drawing to win acomplimentary registration to bbcon 2015 in Austin, Texas*.*Blackbaud reserves the right to change or withdraw this promotion at any time, without advance notice. Promotion has no cash value and may not beexchanged, applied to, or combined with any other offer.16 #bbcon