Click here to load reader

1 Dawn M. Boyer Yvonne M. Riley [email protected] [email protected] [email protected]@psu.edu Manager, Student Systems Data Warehouse Consultant

  • View
    214

  • Download
    0

Embed Size (px)

Text of 1 Dawn M. Boyer Yvonne M. Riley [email protected] [email protected] [email protected]@psu.edu Manager, Student...

  • Slide 1

1 Dawn M. Boyer Yvonne M. Riley [email protected] [email protected] [email protected]@psu.edu Manager, Student Systems Data Warehouse Consultant http://ais.its.psu.edu/ssn/index.html Penn State University Administrative Information Services No More Social Security Numbers Slide 2 2 Dawn M. Boyer [email protected] Manager, Student Systems Penn State University Administrative Information Services No More Social Security Numbers Mainframe Student Systems Slide 3 3 l History l Charge from University President l December 4, 2002 l SSN Confidential, Identity Theft l Implement 2004 l Proactive Slide 4 4 No More Social Security Numbers Mainframe Student Systems l Impact l Over 20 Campuses plus Law and Medical School, World Campus l College/Departments Local Systems l New ID+ Cards l Central Offices Student Systems, Data Warehouse Slide 5 5 No More Social Security Numbers Mainframe Student Systems l University Committees l Steering l Working Group l Sub-committees l Technology l Policy l Conversion Date December Holiday Break 2004 l ID+ Slide 6 6 No More Social Security Numbers Mainframe Student Systems l Central ID Repository CIDR file l Only University File with SSN l Restricted Access l Initial Creation September 2004 Slide 7 7 No More Social Security Numbers Mainframe Student Systems l Student Systems (ISIS) l 44 Data base files 41 Conversion programs l 2.5 million Person records l 88 million records l 350 million keys l SSN fields - Student, Instructor, Adviser, Parent, Employee Slide 8 8 No More Social Security Numbers Mainframe Student Systems l What needs to be converted l All electronic files the contain SSN l Transaction logs l Data sets l Presentation (format) 9-NNNN-NNNN l Labels l Laser or pre-printed forms l Letters/emails l Processes that create students Slide 9 9 No More Social Security Numbers Mainframe Student Systems l Trial Run December 2003 l Worst file 15 million records 7 keys l Worst case 2.84 hours per million record l 45 hours of processing l Changed Strategy 12 files l Remove keys 2 minutes (revert) l Convert File 3 hours l Re-Add keys 20 minutes per key 2.5 hours (invert) l Total 5.5 hours Slide 10 10 No More Social Security Numbers Mainframe System Student Systems l Preparation l Step-by-step tasks l Meetings every 2 weeks Late Summer 2004 l Developers, DBA, Systems, Operations l Analysis of data storage l Crosswalk File SSN, PSU ID l System Tuning l Audit logs turned off l Redundant copy of data turned off l System Priorities Slide 11 11 No More Social Security Numbers Mainframe Student Systems l 4 Known Dates l December 19, 2004 12:01 AM l Disable Online Systems, Process Grades, Start Conversion l December 24, 2004 Noon l Stop All Conversion Activities for the Holiday l December 26, 2004 8:00 AM l Continue Conversion Activities l January 3, 2005 7:00AM l Enable Online Systems Slide 12 12 No More Social Security Numbers Mainframe Student Systems l Conversion l Official CIDR File December 19 l File Back-ups December 19 l Convert online files December 19 & 20 l Technical and User testing of online systems December 20 l Convert non-online files December 21 & 22 l Batch testing December 21 & 22 l Enable online Systems - December 22 Noon Slide 13 13 No More Social Security Numbers Mainframe Student Systems l Problems l Data Integrity l SSN Changes l Last Minute Decisions l Communication filtering down Slide 14 14 No More Social Security Numbers Mainframe Student Systems l What Worked l Starting early l Keep it simple l Planning l Communication l Teamwork Slide 15 15 Removing all SSNs from Penn States Data Warehouse Yvonne M. Riley [email protected] Data Warehouse Consultant Penn State University Administrative Information Services No More Social Security Numbers Slide 16 16 No More Social Security Numbers Penn States Data Warehouse l My task l Convert all social security numbers to PSU IDs. l Leaving no social security numbers on any table when tables were made available to general users. l 18 databases to convert l 120 tables l 150 million records l Analysis started 1 year prior to the conversion. Slide 17 17 No More Social Security Numbers Penn States Data Warehouse l Where did the analysis start: l Met with our DBA personnel to discuss options get suggestions. l Identify key people, who would do what. l Identify problem areas. l How would we approach the coding. l Set up a timeline for myself. Slide 18 18 No More Social Security Numbers Penn States Data Warehouse l Immediate Concerns l What to do with records that dont convert. l Decision was made to copy then to another database for cleanup later and then copied back into the original table. l What to do with old historical records that have incorrect social security numbers. l Steward offices provided us with the corrected SSN prior to the conversion, and we corrected the records. Slide 19 19 No More Social Security Numbers Penn States Data Warehouse l Immediate Concerns l Where would the CIDR file (holds the SSN and new PSU ID) reside? l Decision was made to put the CIDR file on the Warehouse. l We would use it to convert the Warehouse. l After the conversion was over, users could use this table to convert local systems. Slide 20 20 No More Social Security Numbers Penn States Data Warehouse l What was the best way to convert the Warehouse? l Preserved our original tables. l Update programs would be completely step restartable. l Cloned the structure of each table to be converted adding an extra field at the end to hold the original SSN. l Created a view for each table that pulled the PSU ID for the SSN and moved the SSN to a new holding field. l Kept our naming conventions for all views, cloned tables, scripts unique so they could easily be identified. l Example View Name: vssn_tbl_converted l Example Cloned Table Name: ssn_tbl_converted Slide 21 21 No More Social Security Numbers Penn States Data Warehouse l Sample View Code CREATE VIEW dbo.vssn_tableone_scores AS Select psu_id as SSN', Field1 as Field1', Field2 as Field2', Field3 as Field3', SSN as 'ssn_hold' from tableone_scores left join ssnconversion..conversiontable on tableone_scores.ssn = ssnconversion..conversiontable.ssn Slide 22 22 No More Social Security Numbers Penn States Data Warehouse l Update Programs were developed in 4 Phases: l Phase 1 -- Convert the Warehouse into cloned tables (leaving existing tables untouched). After phase 1. I verified. l Phase 2 -- Renamed tables, had steward office verify. l Original tables got renamed. l Cloned tables got renamed to original table name. l Phase 3 -- Dropped the extra ssn_hold field from all tables. l Phase 4 -- Dropped unconverted original tables. Slide 23 23 No More Social Security Numbers Penn States Data Warehouse l Phase 1 Update code: l Count of records to be updated. l Truncate cloned table. l Using the newly created views, inserted records into new cloned table. l Count of records not updated. l Truncate on table where unconverted records were to be moved. l Copy unconverted records to new database. l Delete unconverted records. l Count of records in converted table. Slide 24 24 No More Social Security Numbers Penn States Data Warehouse l The conversion weekend is here l Midnight On Saturday Data Warehouse was made unavailable to user community. l Complete Backup of all databases was done. l Cloned tables were created. l Arrival of the official CIDR data file on Sunday. l Conversion of the Warehouse begins. Slide 25 25 No More Social Security Numbers Penn States Data Warehouse l As the Warehouse is converting l DBA monitored the SQL server. l I reviewed all logs for any errors. l We communicated by phone and email. Slide 26 26 No More Social Security Numbers Penn States Data Warehouse l Problems l Clean up of Historical Data. l Deadlines were given to all steward offices for use to have all cleanup information. l Steward offices were overwhelmed and could not meet the deadlines. Slide 27 27 No More Social Security Numbers Penn States Data Warehouse l What Worked l Sticking to my timeline l Starting early and completing the majority of programming by September. l Keeping update programs as simple as possible and step restartable. l Verifying, verifying and re verifying everything. Slide 28 28 Questions???

Search related