Bb world 2012 using database statistics to make capacity planning decisions (1)

Using Database Statistics to Make Capacity Planning

DecisionsGeoff Mower

Senior Software Performance Engineer, Blackboard

Steve FeldmanSenior Director Software Performance Engineering and Architecture, Blackboard

Who Are We?Geoff Mower- Senior Software Performance Engineer

5 years on the frontlines as a DBA

2 hour interview on a plane

Performance and Scalability for

Outcomes & Community

2 hour interview on a plane

Who Are We?Steve Feldman

• Blackboard since 2003• Performance Engineering

from the start• Platform Architecture in

20052005• Security Engineering in

2010“Love my job…love my team. If you email me, I will respond.”

@seven_seconds

http://goo.gl/Z4Rq5

What We Want You to Do?

Mine your data Ask follow-up questionsask questions Mine your data Ask follow-up questionsask questions

about your

data

What are our obstacles?

• Certain data points are transient– Logging will need to be created in certain situations

to fill in missing data points.• See slide #45 & #46 for an example• See slide #45 & #46 for an example

What You Need

• Permissions to run SQL scripts on your Blackboard Learn environment's database

• Basic SQL fundamentals• Analysis of results• Analysis of results• Database Tools

– Aquadata client & Server or Quest’s Toad

Skill and Knowledge SetsOpen DB Documentation

• Blackboard Learn™, Release 9.1 Open Database Document– http://bit.ly/loW9hi

Asking the Right Questions

Collecting the Right Data

Organizing and Analyzing the Data

Ask Follow-up Questions

Making Decisions

Types of Data

Data Types Table Example

Event ACTIVITY_ACCUMULATOR User navigation

Process COURSE_USERS snapshot

User COURSE_CONTENT User created User COURSE_CONTENT User created

entities

Static LEARN_admin.bb_instance installation

Transient SESSIONS Tracks who has

an active session

Areas of Capacity Planning

• How can the Blackboard Learn schema help you capacity plan for your business requirements in regards to?

Strategic Functional Decisions

Expense and Budget Management

Administrative Planning



To be able to restore individual courses, can we use the course archive tool as a backup function?

Administrative PlanningAsking the Right Questions

How will we know which courses need archiving?

Frequency of course archive task?

Will the archiving process impact user experience?Will the archiving process impact user experience?

Does the course archive tool have limitation?

Location of course archives?

Administrative PlanningCollecting the Right Data

How will we know which courses need archiving?– Course entity creation and modification

Creation TablesCOURSE_CONTENTS

GRADEBOOK_MAIN

Modification TablesCOURSE_CONTENTS

GRADEBOOK_MAINGRADEBOOK_MAIN

GRADEBOOK_GRADES

QTI_ASI_DATA

QTI_RESULT_DATA

FORUM_MAIN

MSG_MAIN

GRADEBOOK_MAIN

GRADEBOOK_GRADES

QTI_ASI_DATA

QTI_RESULT_DATA

FORUM_MAIN

MSG_MAIN

CALENDAR

STAFFINFORMATION

Administrative PlanningOrganizing and Analyzing the Data

• Data Points: Course/Organizations to be archived SQL/Oracle

COURSE_ID Number of New/Modified Course Entities

Hist_101 42

Eng_252 42

Sci_139 38

Eng_429 37

Art_202 35

Eco_323 35

Administrative PlanningMaking Decisions from the Data

• How many new or modified course/organization entities are required for a course/organization to be archived?

• How many courses/organizations should we archive each time the archive task is execution?

• What is the maximum time between archive tasks?

Administrative PlanningActing on Your Descion

SQL/OracleHist_101, c:\Course_Archives\2011-07-09\Eng_252, c:\Course_Archives\2011-07-09\Sci_139, c:\Course_Archives\2011-07-09\Eng_429, c:\Course_Archives\2011-07-09\……• Create a JOB that would run the SQL script to generate

a txt file containing the query results

• Create a schedule task that would run the batch_ImportExport.bat tool and direct it to the txt file created by the JOB in the previous step.


Asking the Right QuestionsExpense and Budget Management

• How much do we need to budget for in order to fund our database requirements?


• When are we going to hit our 95% disk spaced used threshold.


• Available disk space issue– Talk to Network Security Manager (owner of the SAN)

• We need more disk space

Expense and Budget ManagementAsking the Right Questions

• What did I miss?– When will we run out of space?– How long do we expect the requested disk space to

last us?

• Facts vs. guess work

Expense and Budget ManagementCollecting the Right Data

• Data Points: Database file size• Binary• Data Files (.mdf)• TempDB• Transaction Logs (.ldf)• Transaction Logs (.ldf)• Back-up/restore process

Expense and Budget ManagementOrganizing and Analyzing the Data

• Data Points: Database file size SQL/OracleDatabase Name Physical Name Size in MB

BBLEARN_stats F:\data\BBLEARN_stats_data.mdf 70338

BBLEARN F:\data\BBLEARN_data.mdf 45520

cms_files_inst F:\data\cms_files_inst_data.mdf 5704

BBLEARN G:\logs\BBLEARN_log.ldf 3083BBLEARN G:\logs\BBLEARN_log.ldf 3083

cms_files_inst G:\logs\cms_files_inst_log.ldf 1361

cms_files_courses F:\data\cms_files_courses_data.mdf 1072

cms_files_users F:\data\cms_files_users_data.mdf 926

msdb F:\data\MSSQL.1\MSSQL\DATA\MSDBData.mdf 792

cms_files_library F:\data\cms_files_library_data.mdf 720

cms_files_orgs F:\data\cms_files_orgs_data.mdf 384

msdb F:\data\MSSQL.1\MSSQL\DATA\MSDBLog.ldf 259

cms_files_users G:\logs\cms_files_users_log.ldf 214

cms_files_courses G:\logs\cms_files_courses_log.ldf 207


• Data Points: Table Row Count and Sizes SQL/Oracle

Table Name

Row

Count Reserved Data Index Size Unused

ACTIVITY_ACCUMULATOR 377266616 84097056 46905888 37183888 7280

COURSE_USERS 3837931 496024 258272 237496 256

USERS 303687 139480 96848 42440 192

COURSE_MAIN 121623 86024 47760 37984 280

NAVIGATION_ITEM 1876 2032 608 1128 296

SYSTEM_TRACKING 3550 1936 1672 160 104

USER_ROLES 4114 544 160 288 96


Data Points: Table growth rate (by year) of the Activity_Accumulator SQL/Oracle

100000000

120000000

0

20000000

40000000

60000000

80000000

2003 2004 2005 2006 2007 2008 2009 2010


Data Points: ACTIVITY_ACCUMULATOR growth rate (by month) SQL/Oracle

10000000

12000000

0

2000000

4000000

6000000

8000000


• Data Points: COURSE_USERS table growth rate (by month)

700000

800000

900000

0

100000

200000

300000

400000

500000

600000

August August August

Expense and Budget ManagementMaking Decisions from the Data

• Conclusion• Due to opening of school tasks we will see our

available disk space decrease below 5% during the month of August.

• In order to make it 3 years we will need a minimum • In order to make it 3 years we will need a minimum 120 GB.



• Can our database support all classroom testing if it was online?

Strategic Functional DecisionsAsking the Right Questions

• Study…• Online & offline test taking populations• The tests themselves• Test taking behavior • How testing impacts the database• How testing impacts the database• Locating when you will see resource contention

Strategic Functional DecisionsHistorical Use of Online Testing

What is the percent of students from the previous term that have completed a test online? SQL/Oracle

Online

Offline

Strategic Functional DecisionsCollecting the Right Data

Previous Use Full Adoption

Courses with Tests 11,274 55,504

Enrollments with Tests 225,480 1,110,080

Number of questions

submitted in a term

4,219,032 25,694,266

Peak Question Submission 60,346 297,094Peak Question Submission

by hour

60,346 297,094


• Determine CPU and I/O cost of testing (SQL server)– Track the Stored Procedures performing the works

SQL/Oracle» qti_result_data_cr» update_gradebook_grade» attempt_cr» attempt_cr» update_qti_res_bbmd_dtmod» gb2_is_log_enabled» gb2_attempt_logger

Strategic Functional DecisionsOrganizing and Analyzing the Data

• Determine CPU and I/O cost of testing (SQL server)– Track the Stored Procedures performing the works SQL/Oracle


• Determine CPU and I/O cost of testing– Track the queries performing the work SQL/Oracle


Determine IO cost of all cached stored procedures SQL/Oracle

500

600

700

Average IO

0

100

200

300

400

Strategic Functional DecisionsMaking Decisions from the Data

• A reasonable understanding…• How tests are currently used• How tests will be used• How much each question costs the database upon

submissionsubmission

• Lack information on…• Baseline of database’s IO and CPU resources• Non-testing database transactions


• Determine the breaking point of CPU and I/O of testing– Create a CPU logging script SQL/Oracle

• SQL server logging limitation– 4 hours of CPU tracking– Lacks IO counters

– Create a IO logging script SQL/Oracle• Snapshot in time

– Need to log the delta between the last logged IO metric and current query results

– Create a Query logging script SQL/Oracle• Compares test question submission to all other transactions


• Create a separate database from the Blackboard Learn’s databases (you can use the database server Blackboard Learn is running on)

• Create tables with in the new database that match • Create tables with in the new database that match the query results you wish to store

• Create a JOB that will run each query and insert the results into the corresponding table and database.

Capacity Planning

is an art if done

properly…

or a disaster if not

done at all.

[email protected]@blackboard.com

@@gvmowergvmower

@@seven_secondsseven_seconds

sevenseconds.wordpress.comsevenseconds.wordpress.com

[email protected]@blackboard.com

http://goo.gl/Z4Rq5

Please provide feedback for this session by [email protected].

The title of this session is:

Using Database Statistics to Make Capacity Planning Decisions

Appendices

Administrative PlanningWhat courses meet the course archive criteria

declare @Last_Archive datetime

declare @Datediff int

declare @Number_of_New_Entities int

set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran

set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.

set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived

select course_id, CC.CC + GM.GM + GG.GG + QA.QA + A.A + ConM.CONFERENCE_MAIN + FM.FORUM_MAIN + MM.MSG_MAIN + C.C + S.S AS [Number of New/Modified Course Entities]

from course_main cm

left join (select crsmain_pk1, COUNT(cc.pk1) CC

from course_contents cc

left join course_main cm on cm.pk1 = cc.crsmain_pk1

where (cc.dtcreated > @Last_Archive

or cc.dtmodified > @Last_Archive)

and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff

group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(gm.pk1) GM

from gradebook_main gm

left join course_main cm on cm.pk1 = gm.crsmain_pk1

where (date_added > @Last_Archive

or date_modified > @Last_Archive)

and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff

group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(gg.pk1) GG


left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1

left join course_main cm on gm.crsmain_pk1 = cm.pk1

where (gg.date_added > @Last_Archive

OR gg.date_modified > @Last_Archive)

and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff

group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(qa.pk1) QA

from qti_asi_data qa

left join course_main cm on cm.pk1 = qa.crsmain_pk1

where (bbmd_date_added > @Last_Archive

OR bbmd_date_modified > @Last_Archive)

and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff

group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(a.pk1) A

from attempt a

left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1

left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1


where (a.attempt_date > @Last_Archive

or a.date_modified > @Last_Archive)

and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediffand datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff

group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1

left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN

from conference_owner CO

left join conference_main CON on CON.conference_owner_pk1 = CO.pk1

left join course_main cm on cm.pk1 = co.owner_pk1

where co.owner_table = 'COURSE_MAIN'

and (CON.dtcreated > @Last_Archive

OR CON.dtmodified > @Last_Archive)

and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff

group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1

left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN



left join forum_main fr on fr.confmain_pk1 = CON.pk1


where owner_table = 'COURSE_MAIN'

and (fr.dtcreated > @Last_Archive

OR fr.dtmodified > @Last_Archive)

and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff

group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1

left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN




left join msg_main mm on mm.forummain_pk1 = fr.pk1

left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1



and (mm.dtcreated > @Last_Archive

OR mm.dtmodified > @Last_Archive)

and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff

group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1

left join (select crsmain_pk1, COUNT(c.pk1) C

from calendar c

left join course_main cm on cm.pk1 = c.crsmain_pk1

where c.dtmodified > @Last_Archive

and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff

group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(s.pk1) S

from staffinformation S

left join course_main cm on cm.pk1 = s.crsmain_pk1

where s.dtmodified > @Last_Archive

and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff

group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1


set serveroutput on size 100000

---Administrative PlanningWhat courses meet the course archive criteria

declare

v_Last_Archive date;

v_Datediff int;

v_Number_of_New_Entities int;

v_Directory varchar(256);

v_File_Name varchar(256);

Begin

v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran

v_Datediff := 1 ; --this is the day

--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.

v_Number_of_New_Enti ties := 1;

--Enter the number of new course entities required for a course to be archived

select 'usr\local\blackboard\Course_Archi ves \' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '\' into v_Directory FROM Dual;

--Alter the string to build the directory you wish to store your archives

dbms_output.enable(9999999);

dbms_output.new_line();

FOR rec IN (

select course_id | | v_Directory as output

from course_main cm




where (cc.dtcreated > v_Last_Archive

or cc.dtmodified > v_Last_Archive)

and cm.dtcreated- cc.dtcreated> v_Datediff





where (date_added > v_Last_Archive

or date_modified > v_Last_Archive)

and cm.dtcreated- gm.date_added> v_Datediff






where (gg.date_added > v_Last_Archive

OR gg.date_modified > v_Last_Archive)

and cm.dtcreated- gg.date_added> v_Datediff





where (bbmd_date_added > v_Last_Archive

OR bbmd_date_modified > v_Last_Archive)

and cm.dtcreated- qa.bbmd_date_added> v_Datediff



from attempt a




where (a.attempt_date > v_Last_Archive

or a.date_modified > v_Last_Archive)

and cm.dtcreated- a.attempt_date > v_Datediff







and (CON.dtcreated > v_Last_Archive

OR CON.dtmodified > v_Last_Archive)

and cm.dtcreated-con.dtcreated > v_Datediff








and (fr.dtcreated > v_Last_Archive

OR fr.dtmodified > v_Last_Archive)

and cm.dtcreated- fr.dtcreated > v_Datediff










and (mm.dtcreated > v_Last_Archive

OR mm.dtmodifi ed > v_Last_Archive)

and cm.dtcreated- mm.dtcreated > v_Datediff



from calendar c

left join course_main cm on cm.pk1 = c.crsmain_pk1left join course_main cm on cm.pk1 = c.crsmain_pk1

where c.dtmodified > v_Last_Archive

and cm.dtcreated- c.dtmodified > v_Datediff





where s.dtmodified > v_Last_Archive

and cm.dtcreated- s.dtmodified > v_Datediff


where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )

LOOP

dbms_output.put_line (rec.output);

END LOOP;


end;


declare @Last_Archive datetimedeclare @Datediff intdeclare @Number_of_New_Entities intdeclare @Directory varchar(256)declare @File_Name varchar(256)

set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ranset @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archivedset @Directory = (SELECT ', c:\Course_Archives\' + CAST(CONVERT(date, GETDATE()) as varchar(10)) + '\') --Alter the string to build the directory you wish to store your archives

select course_id + @Directoryfrom course_main cmleft join (select crsmain_pk1, COUNT(cc.pk1) CC

from course_contents ccleft join course_main cm on cm.pk1 = cc.crsmain_pk1where (cc.dtcreated > @Last_Archiveor cc.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediffgroup by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(gm.pk1) GMfrom gradebook_main gmleft join course_main cm on cm.pk1 = gm.crsmain_pk1where (date_added > @Last_Archiveor date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, gm.date_added) > @Datediffgroup by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(gg.pk1) GGfrom gradebook_main gmleft join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1left join course_main cm on gm.crsmain_pk1 = cm.pk1where (gg.date_added > @Last_ArchiveOR gg.date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, gg.date_added) > @Datediffgroup by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(qa.pk1) QAfrom qti_asi_data qaleft join course_main cm on cm.pk1 = qa.crsmain_pk1where (bbmd_date_added > @Last_ArchiveOR bbmd_date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediffgroup by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(a.pk1) Afrom attempt aleft join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1left join course_main cm on cm.pk1 = gm.crsmain_pk1where (a.attempt_date > @Last_Archiveor a.date_modified > @Last_Archive)or a.date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediffgroup by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1

left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAINfrom conference_owner COleft join conference_main CON on CON.conference_owner_pk1 = CO.pk1left join course_main cm on cm.pk1 = co.owner_pk1where co.owner_table = 'COURSE_MAIN'and (CON.dtcreated > @Last_ArchiveOR CON.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediffgroup by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1

left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAINfrom conference_owner COleft join conference_main CON on CON.conference_owner_pk1 = CO.pk1left join forum_main fr on fr.confmain_pk1 = CON.pk1left join course_main cm on cm.pk1 = co.owner_pk1where owner_table = 'COURSE_MAIN'and (fr.dtcreated > @Last_ArchiveOR fr.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediffgroup by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1

left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAINfrom conference_owner COleft join conference_main CON on CON.conference_owner_pk1 = CO.pk1left join forum_main fr on fr.confmain_pk1 = CON.pk1left join msg_main mm on mm.forummain_pk1 = fr.pk1left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1left join course_main cm on cm.pk1 = co.owner_pk1where owner_table = 'COURSE_MAIN'and (mm.dtcreated > @Last_ArchiveOR mm.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediffgroup by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1

left join (select crsmain_pk1, COUNT(c.pk1) Cfrom calendar cleft join course_main cm on cm.pk1 = c.crsmain_pk1where c.dtmodified > @Last_Archiveand datediff(hour, cm.dtcreated, c.dtmodified) > @Datediffgroup by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1

left join (select crsmain_pk1, COUNT(s.pk1) Sfrom staffinformation Sleft join course_main cm on cm.pk1 = s.crsmain_pk1where s.dtmodified > @Last_Archiveand datediff(hour, cm.dtcreated, s.dtmodified) > @Datediffgroup by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1

where ISNULL(CC.CC,0) + ISNULL(GM.GM,0) + ISNULL(GG.GG,0) + ISNULL(QA.QA,0) + ISNULL(A.A,0) + ISNULL(ConM.CONFERENCE_MAIN,0) + ISNULL(FM.FORUM_MAIN,0) + ISNULL(MM.MSG_MAIN,0) + ISNULL(S.S,0) + ISNULL(C.C,0) >= @Number_of_New_Entities



---Administrative PlanningWhat courses meet the course archive criteria

declare

v_Last_Archive date;

v_Datediff int;

v_Number_of_New_Entities int;

v_Directory varchar(256);

v_File_Name varchar(256);

Begin

v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran

v_Datediff := 1 ; --this is the day

--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.

v_Number_of_New_Enti ties := 1;

--Enter the number of new course entities required for a course to be archived

select 'usr\local\blackboard\Course_Archi ves \' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '\' into v_Directory FROM Dual;

--Alter the string to build the directory you wish to store your archives



FOR rec IN (

select course_id | | v_Directory as output

from course_main cm




where (cc.dtcreated > v_Last_Archive

or cc.dtmodified > v_Last_Archive)

and cm.dtcreated- cc.dtcreated> v_Datediff





where (date_added > v_Last_Archive

or date_modified > v_Last_Archive)

and cm.dtcreated- gm.date_added> v_Datediff






where (gg.date_added > v_Last_Archive

OR gg.date_modified > v_Last_Archive)

and cm.dtcreated- gg.date_added> v_Datediff





where (bbmd_date_added > v_Last_Archive

OR bbmd_date_modified > v_Last_Archive)

and cm.dtcreated- qa.bbmd_date_added> v_Datediff



from attempt a




where (a.attempt_date > v_Last_Archive

or a.date_modified > v_Last_Archive)

and cm.dtcreated- a.attempt_date > v_Datediff







and (CON.dtcreated > v_Last_Archive

OR CON.dtmodified > v_Last_Archive)

and cm.dtcreated-con.dtcreated > v_Datediff








and (fr.dtcreated > v_Last_Archive

OR fr.dtmodified > v_Last_Archive)

and cm.dtcreated- fr.dtcreated > v_Datediff










and (mm.dtcreated > v_Last_Archive

OR mm.dtmodifi ed > v_Last_Archive)

and cm.dtcreated- mm.dtcreated > v_Datediff



from calendar c

left join course_main cm on cm.pk1 = c.crsmain_pk1left join course_main cm on cm.pk1 = c.crsmain_pk1

where c.dtmodified > v_Last_Archive

and cm.dtcreated- c.dtmodified > v_Datediff





where s.dtmodified > v_Last_Archive

and cm.dtcreated- s.dtmodified > v_Datediff


where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )

LOOP

dbms_output.put_line (rec.output);

END LOOP;


end;

Expense and Budget ManagementDatabase File Size

SELECT DB_NAME(database_id) AS DatabaseName, Name AS Logical_Name, Physical_Name, (size*8)/1024 SizeMBFROM sys.master_filesorder by SizeMB desc

Expense and Budget ManagementDatabase File Size

select (select sum(bytes)/1024/1024 from dba_data_files)+(select sum(bytes)/1024/1024 from dba_temp_files) "Size in MB" from dual;

Expense and Budget ManagementTable Row Count and Sizes

select IDENTITY( int ) AS pk1, name into Table_List from sys.tables

CREATE TABLE Table_Results(name nvarchar(256),rows int,reserved nvarchar(256),data nvarchar(256),index_size nvarchar(256),unused nvarchar(256))

declare @pk1 intdeclare @table nvarchar(256)declare @end intset @pk1 = 0set @pk1 = 0set @end = (select top 1 PK1 from Table_List order by pk1 Desc)

WHILE @pk1 < @endbeginset @pk1 = @pk1 + 1set @table = (select name from Table_List where pk1 = @pk1)

insert into Table_Resultsexec sp_spaceused @table

end

select *from table_results

drop table table_listdrop table table_results

Expense and Budget ManagementTable Row Count and Sizes

Select OWNER, TABLE_NAME, TABLESPACE_NAME, NUM_ROWS, PCT_FREE , PCT_USED , (blocks*8/1024,2) as MB from dba_tablesorder by OWNER, TABLESPACE_NAME, TABLE_NAME;

Expense and Budget ManagementTable growth rate (by year)

/* depending on the table, the dtcreated could need to be changed to timestamp */

select YEAR(dtcreated), COUNT(*) from course_contents --Enter the table name heregroup by YEAR(dtcreated)

Expense and Budget ManagementTable growth rate (by year)

select TO_char(dtcreated,'YYYY') , COUNT(*) from course_contents --Enter the table name heregroup by TO_char(dtcreated,'YYYY') ;

Expense and Budget ManagementTable growth rate (by month)

/* depending on the table, the dtcreated could need to be changed to timestamp */

select YEAR(dtcreated), MONTH(dtcreated), COUNT(*)from course_main --Enter the table name heregroup by YEAR(dtcreated), MONTH(dtcreated)

Expense and Budget ManagementTable growth rate (by month)

select TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') , COUNT(*) from course_contents --Enter the table name heregroup by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM')order by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') ;

Strategic Functional DecisionsWhat is the percent of enrollments from the previous term that have

completed a test online?

declare @start_date datetimedeclare @end_date datetimedeclare @TotalPopulation intdeclare @TestingPopulation int

set @start_date = '2010-9-1 00:00:00:000'set @end_date = '2011-9-1 00:00:00:000'

set @TotalPopulation = (select Count(distinct users_pk1)from course_users culeft join course_main cm on cm.pk1 = cu.crsmain_pk1where role = 'S'and dtcreated > @start_dateand dtcreated < @end_date)

set @TestingPopulation = (select Count(Distinct cu.users_pk1)from course_users culeft join gradebook_grade gg on gg.course_users_pk1 = cu.pk1left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1left join course_main cm on cm.pk1 = cu.crsmain_pk1where gm.qti_asi_data_pk1 > 0and gg.first_attempt_pk1 > 0and dtcreated > @start_dateand dtcreated < @end_date)

select @TestingPopulation as Online_Testing, @TotalPopulation - @TestingPopulation as Offline_Testing

Strategic Functional DecisionsWhat is the percent of enrollments from the previous term that have

completed a test online?


declare v_start_date date;

v_end_date date;

v_TotalPopulation int;

v_TestingPopulation int;

Begin



v_start_date := to_date('1-JAN-2010');

v_end_date := to_date('1-JAN-2012');

select Count(distinct users_pk1) into v_TotalPopulation

from course_users cu

left join course_main cm on cm.pk1 = cu.crsmain_pk1

where role = 'S'

and dtcreated > v_start_date

and dtcreated < v_end_date;

select Count(Distinct cu.users_pk1) into v_TestingPopulation

from course_users cu

left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1

left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1

left join course_main cm on cm.pk1 = cu.crsmain_pk1

where gm.qti_asi_data_pk1 > 0

and gg.first_attempt_pk1 > 0

and dtcreated > v_start_date

and dtcreated < v_end_date;

dbms_output.put_line (v_TestingPopulation || ',' || to_char(v_TotalPopulation - v_TestingPopulation));

--select v_TestingPopulation as Online_Testing, v_TotalPopulation - v_TestingPopulation as Offline_Testing;

end;

Strategic Functional DecisionsHow many tests have been completed per enrollment?

declare @enrollment_start datetimedeclare @enrollment_end datetime

set @enrollment_start = '2010-9-1 00:00:00:000'set @enrollment_end = '2011-9-1 00:00:00:000'

select NumberOfTests, COUNT(NumberOfTests) NumberOfEnrollmentsinto #BinTestsfrom (

select AllStudents.users_pk1, NumberOfTests = Case When TestingStudents.TheCount IS NULL THEN 0When TestingStudents.TheCount > 0 THEN TestingStudents.TheCountEND

FROM (select distinct users_pk1from course_userswhere role = 'S'and enrollment_date > @enrollment_startand enrollment_date < @enrollment_end) AllStudents

left join(select cu.users_pk1, COUNT(gg.pk1) as TheCountfrom course_users culeft join gradebook_gradegg on gg.course_users_pk1 = cu.pk1left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1where gm.qti_asi_data_pk1 > 0and gg.first_attempt_pk1 > 0and cu.enrollment_date > @enrollment_startand cu.enrollment_date < @enrollment_endgroup by cu.users_pk1) TestingStudents

on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBingroup by NumberOfTests

declare @start intdeclare @end intdeclare @enrollments int

set @start = 0set @end = (Select top 1 NumberOfTests from #BinTests order by NumberOfTests desc)

create table #Results(Number_Of_Tests int,Number_Of_Enrollments int)

While @start-1 < @endBegin

set @enrollments = (select NumberOfEnrollments from #BinTests where NumberOfTests = @start)

insert into #Results(Number_of_Tests, Number_Of_Enrollments)values(@start, @enrollments)

set @start = @start + 1

end

select Number_Of_Tests, Number_Of_Enrollments = CaseWhen Number_Of_Enrollments IS NULL THEN 0When Number_Of_Enrollments >= 0 THEN Number_Of_EnrollmentsEnd

from #Results

drop table #BinTestsdrop table #Results

Strategic Functional DecisionsHow many tests have been completed per enrollment?


declare

v_enrollment_start date;

v_enrollment_end date;

sql_stmt varchar(8000);

v_start int;

v_end int;

v_enrollments int;

Begin



v_enrollment_start := to_date('2010-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');

v_enrollment_end := to_date('2011-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');

--sql_stmt :=

EXECUTE IMMEDIATE ' drop table BinTests';

sql_stmt :=''

| | 'create table '

| | 'BinTests as '

| | 'select NumberOfTests , COUNT(NumberOfTests ) as NumberOfEnroll ments '

| | 'from ( '

| | ' select AllStudents.users_pk1, nvl(TestingStudents.TheCount,0) NumberOfTests'

| | ' FROM ( '

| | ' select distinct users_pk1 '

| | ' from course_users '

| | ' where role = ''S'' '

| | ' and enrollment_date > ' | | 'to_date('''| | v_enrollment_start | | ''')'

| | ' and enrollment_date < ' | | 'to_date('''| | v_enrollment_end | | ''')' | | ' ) AllStudents '

| | ' left join( '

| | ' select cu.users_pk1, COUNT(gg.pk1) as TheCount '

| | ' from course_users cu '

| | ' left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1 '

| | ' left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1 '

| | ' where gm.qti_asi_data_pk1 > 0 '

| | ' and gg.first_attempt_pk1 > 0 '

| | ' and cu.enrollment_date > ' | | 'to_date('''| | v_enrollment_start | | ''')'

| | ' and cu.enrollment_date < ' | | 'to_date('''| | v_enrollment_end | | ''')'

| | ' group by cu.users_pk1) TestingStudents '

| | ' on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBin '

| | 'group by NumberOfTests' ;

dbms_output.put_line (sql_stmt );

EXECUTE IMMEDIATE sql_stmt ;

EXCEPTION

WHEN OTHERS THEN NULL;

End;

/


declare




v_start int;

v_end int;

v_enrollments int;

Begin





v_start := 0;

Select NumberOfTests into v_end from (select NumberOfTests from BinTests order by NumberOfTests desc) WHERE ROWNUM =1;

EXECUTE IMMEDIATE ' drop table Results';

EXECUTE IMMEDIATE ' create table Results ( Number_Of_Tests int, Number_Of_Enroll ments int)';

EXCEPTION


END;

/


declare




v_start int;

v_end int;

v_enrollments int;v_enrollments int;

Begin





While v_start-1 < v_end

loop

select NumberOfEnroll ments into v_enrollments from BinTests where NumberOfTests = v_start;

insert into Results

(Number_of_Tests, Number_Of_Enroll ments)

values

( v_start, v_enrollments );

v_start := v_start + 1;

end loop;


FOR rec IN (

select Number_Of_Tests, nvl(Number_Of_Enroll ments,0) Number_Of_Enroll ments from Results)

LOOP

dbms_output.put_line (rec.Number_Of_Tests | | ',' | | rec.Number_Of_Enrollments);

END LOOP;


EXCEPTION


--drop table BinTests

--drop table Results

END;

Strategic Functional DecisionsHow many test questions were submitted at our peak hour?

SELECT count(pk1) TheCount,cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' +cast(DATEPART(MONTH, bbmd_date_added) as varchar(2))

from qti_result_datagroup by DATEPART(YEAR, bbmd_date_added),

DATEPART(MONTH, bbmd_date_added)order by DATEPART(YEAR, bbmd_date_added),

DATEPART(MONTH, bbmd_date_added)


SELECT count(pk1) TheCount,TO_char(bbmd_date_added,'YYYY') || '-' || TO_char(bbmd_date_added,'MM')

from qti_result_datagroup by TO_char(bbmd_date_added,'YYYY'),

TO_char(bbmd_date_added,'MM')order by TO_char(bbmd_date_added,'YYYY'),

TO_char(bbmd_date_added,'MM');


SELECT top 25 count(pk1) TheCount,cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' +cast(DATEPART(MONTH, bbmd_date_added) as varchar(2)) + '-' +

cast(DATEPART(DAY, bbmd_date_added) as varchar(2)) + ' ' +cast(DATEPART(HOUR, bbmd_date_added) as varchar(2)) + ':00'

from qti_result_datagroup by DATEPART(YEAR, bbmd_date_added),

DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added), DATEPART(HOUR, bbmd_date_added)

order by count(pk1) desc


select * from (SELECT count(pk1) TheCount,

TO_char(bbmd_date_added,'YYYY') || '-' || TO_char(bbmd_date_added,'MM') || '-' || TO_char(bbmd_date_added,'DD') || '-' || TO_char(bbmd_date_added,'HH')from qti_result_datagroup by TO_char(bbmd_date_added,'YYYY'),

TO_char(bbmd_date_added,'MM'),TO_char(bbmd_date_added,'DD') ,TO_char(bbmd_date_added,'HH')order by count(pk1) desc)where rownum <26;

Strategic Functional DecisionsTest question submission by hour of the day?

declare @start_date datetimedeclare @end_date datetime


SELECT count(pk1) TheCount,cast(DATEPART(hour, bbmd_date_added) as varchar(4)) TheHour

from qti_result_datawhere bbmd_date_added > @start_dateand bbmd_date_added < @end_dategroup by DATEPART(HOUR, bbmd_date_added)order by DATEPART(HOUR, bbmd_date_added)order by DATEPART(HOUR, bbmd_date_added)

Strategic Functional DecisionsTest question submission by hour of the day?

set serveroutput on size 100000declare v_start_date date;

v_end_date date;

Begin

v_start_date := to_date('2010-9-1 00:00:00', 'yyyy-mm-dd HH24:MI:SS');v_end_date := to_date('2011-9-1 00:00:00', 'yyyy-mm-dd HH24:MI:SS');dbms_output.enable(9999999);dbms_output.new_line();

FOR rec IN (FOR rec IN (SELECT count(pk1) TheCount,

TO_char(bbmd_date_added,'HH24') TheHourfrom qti_result_datawhere bbmd_date_added > v_start_dateand bbmd_date_added < v_end_dategroup by TO_char(bbmd_date_added,'HH24')order by TO_char(bbmd_date_added,'HH24'))

LOOPdbms_output.put_line (to_char(rec.TheCount));

END LOOP;dbms_output.new_line();

end;

Strategic Functional DecisionsTest question Submission by day of the week?

declare @start_date datetimedeclare @end_date datetime


SELECT count(pk1) TheCount, TheDay = Casewhen cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 1 then

'Sunday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 2 then

'Monday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 3 then

'Tuesday''Tuesday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 4 then

'Wednesday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 5 then

'Thursday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 6 then

'Friday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 7 then

'Saturday'end

from qti_result_datawhere bbmd_date_added > @start_dateand bbmd_date_added < @end_dategroup by DATEPART(DW, bbmd_date_added)order by DATEPART(DW, bbmd_date_added) asc

Strategic Functional DecisionsTest question Submission by day of the week?

--v_start_date := '2010-9-1 00:00:00:000'--v_end_date := '2011-9-1 00:00:00:000'

SELECT count(pk1) TheCount, to_char(to_date (bbmd_date_added, 'yyyy-mm-dd HH24:MI:SS'),'DAY') from qti_result_datawhere bbmd_date_added > to_date('2010-9-1', 'yyyy-mm-dd')and bbmd_date_added < to_date('2011-9-1', 'yyyy-mm-dd')group by to_char(to_date (bbmd_date_added , 'yyyy-mm-dd HH24:MI:SS'),'DAY') order by to_char(to_date (bbmd_date_added , 'yyyy-mm-dd HH24:MI:SS'),'DAY') asc

Strategic Functional DecisionsDetermine CPU and I/O cost of testing (SQL server)

SELECT p.name AS[SP_Name],ps.total_worker_time AS[Total_Worker_Time],ps.total_physical_reads AS[Total_Physical_Reads], ps.total_logical_reads,ps.total_logical_writes AS[Total_Logical_Writes],(ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS[TOTAL_IO],ps.execution_count,ISNULL(ps.execution_count/DATEDIFF(Second,ps.cached_time,GETDATE()),0)AS[Calls/Second],ps.total_elapsed_time,ps.last_worker_time,ps.last_worker_time,ps.total_elapsed_time/ps.execution_count AS[avg_elapsed_time],ps.last_elapsed_time,ps.last_execution_time,ps.cached_time, qs.text QueryFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()--and p.name = 'qti_result_data_cr' -- Enter the name of the stored procedureORDER BY ps.last_worker_time DESC;


Typically, you would view the AWR data through Oracle Enterprise Manager screens or AWR reports. However, you can view the statistics with the following views:

V$ACTIVE_SESSION_HISTORY

This view displays active database session activity, sampled once every second. See "Active Session History (ASH)".

V$ metric views provide metric data to track the performance of the system

The metric views are organized into various groups, such as event, event class, system, session, service, file, and tablespace metrics. These groups are identified in the V$METRICGROUP view.

DBA_HIST views

The DBA_HIST views contain historical data stored in the database. This group of views includes:

DBA_HIST_ACTIVE_SESS_HISTORY displays the history of the contents of the in-memory active session history for recent system activity.DBA_HIST_BASELINE displays information about the baselines captured on the systemDBA_HIST_DATABASE_INSTANCE displays information about the database environmentDBA_HIST_SNAPSHOT displays information on snapshots in the systemDBA_HIST_SQL_PLAN displays the SQL execution plansDBA_HIST_WR_CONTROL displays the settings for controlling AWR


SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) Query,qs.execution_count,qs.total_logical_reads, qs.last_logical_reads,qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO],qs.total_worker_time 'Total_CPU_Time',qs.last_worker_time 'Last_CPU_Time',qs.total_elapsed_time/1000000 total_elapsed_time_in_S,qs.total_elapsed_time/1000000 total_elapsed_time_in_S,qs.last_elapsed_time/1000000 last_elapsed_time_in_S,qs.last_execution_time,qs.creation_time,qp.query_planFROM sys.dm_exec_query_stats qsCROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qtCROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qpWHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) LIKE '%qti%' -- Insert the query you are interested in findingORDER BY qs.last_execution_time DESC


Typically, you would view the AWR data through Oracle Enterprise Manager screens or AWR reports. However, you can view the statistics with the following views:

V$ACTIVE_SESSION_HISTORY

This view displays active database session activity, sampled once every second. See "Active Session History (ASH)".

V$ metric views provide metric data to track the performance of the system

The metric views are organized into various groups, such as event, event class, system, session, service, file, and tablespace metrics. These groups are identified in the V$METRICGROUP view.tablespace metrics. These groups are identified in the V$METRICGROUP view.

DBA_HIST views

The DBA_HIST views contain historical data stored in the database. This group of views includes:

DBA_HIST_ACTIVE_SESS_HISTORY displays the history of the contents of the in-memory active session history for recent system activity.DBA_HIST_BASELINE displays information about the baselines captured on the systemDBA_HIST_DATABASE_INSTANCE displays information about the database environmentDBA_HIST_SNAPSHOT displays information on snapshots in the systemDBA_HIST_SQL_PLAN displays the SQL execution plansDBA_HIST_WR_CONTROL displays the settings for controlling AWR

Strategic Functional DecisionsDetermine I/O cost of question submission (SQL server)

SELECT p.name AS[SP_Name],(ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS AVG_IOFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()and p.name = 'qti_result_data_cr'ORDER BY (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count DESC;

Strategic Functional DecisionsDetermine I/O cost of question submission (SQL server)

please look at AWR reportsear qti_result_data_cr

Strategic Functional DecisionsDetermine IO cost of all cached stored procedures

SELECT p.name AS[SP_Name],(ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS AVG_IOFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()and ps.database_id =DB_ID()ORDER BY (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count DESC;

Strategic Functional DecisionsDetermine IO cost of all cached stored procedures

--look at AWR reportset pagesize 60;column executions format 999,999,999;column Mem_used format 999,999,999;

SELECT SUBSTR(owner,1,10) Owner,SUBSTR(type,1,12) Type,SUBSTR(name,1,20) Name,SUBSTR(name,1,20) Name,executions,sharable_mem Mem_used,SUBSTR(kept||' ',1,4) "Kept?"

FROM v$db_object_cacheWHERE type in ('PROCEDURE','PACKAGE BODY','PACKAGE')ORDER BY executions desc;

HERE IS THE OUTPUT OF MEMORY.SQL:SQL> @memory

Strategic Functional DecisionsDetermine total IO cost of all cached stored procedures

SELECT p.name AS[SP_Name],(ps.total_logical_reads +ps.total_logical_writes) AS TOTAL_IOFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()ORDER BY (ps.total_logical_reads +ps.total_logical_writes) DESC;

Strategic Functional DecisionsDetermine total IO cost of all cached stored procedures

AWR report

Strategic Functional DecisionsQuery that compares the test taking activity cost vs. to other activity

select 'QTI_IO' as TYPE, SUM(TOTAL_LOGICAL_READS + TOTAL_LOGICAL_WRITES) IO

FROM (SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,

((CASE qs.statement_end_offset

WHEN -1 THEN DATALENGTH(qt.TEXT)

ELSE qs.statement_end_offset

END - qs.statement_start_offset)/2)+1) Query,

qs.execution_count,

qs.total_logical_reads, qs.last_logical_reads,

qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO],

qs.total_worker_time 'Total_CPU_Time',

qs.last_worker_time 'Last_CPU_Time',

qs.total_elapsed_time/1000000 total_elapsed_time_in_S,

qs.last_elapsed_time/1000000 last_elapsed_time_in_S,

qs.last_execution_time,

qs.creation_time,

qp.query_plan

FROM sys.dm_exec_query_stats qs

CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt

CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp

WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)

LIKE '%qti%' -- Insert the query you are interested in finding

and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)

NOT LIKE '%sys.dm%'

--and qs.last_execution_time + .005 > GETDATE()

)QTI_IO)QTI_IO

UNION

select 'NON_QTI_IO' as TYPE, SUM(TOTAL_LOGICAL_READS + TOTAL_LOGICAL_WRITES) IO

FROM (SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,

((CASE qs.statement_end_offset

WHEN -1 THEN DATALENGTH(qt.TEXT)

ELSE qs.statement_end_offset

END - qs.statement_start_offset)/2)+1) Query,

qs.execution_count,

qs.total_logical_reads, qs.last_logical_reads,

qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO],

qs.total_worker_time 'Total_CPU_Time',

qs.last_worker_time 'Last_CPU_Time',

qs.total_elapsed_time/1000000 total_elapsed_time_in_S,

qs.last_elapsed_time/1000000 last_elapsed_time_in_S,

qs.last_execution_time,

qs.creation_time,

qp.query_plan

FROM sys.dm_exec_query_stats qs

CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt

CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp

WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)

NOT LIKE '%qti%' -- Insert the query you are interested in finding

and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)

NOT LIKE '%sys.dm%'

--and qs.last_execution_time + .005 > GETDATE()

)NON_QTI_IO

Strategic Functional DecisionsQuery that compares the test taking activity cost vs. to other activity

AWR reportand v$sql, sqlarea

on qti_result_data_cr

Strategic Functional DecisionsQuery that joins the QTI tables (bbmd_data_added) with the cost of the

queries associated with the insert

declare @start_date datetimedeclare @end_date datetimedeclare @question_io int

set @start_date = '2010-9-1 00:00:00:000'set @end_date = '2011-9-1 00:00:00:000'set @question_io = (SELECT ps.last_logical_reads + ps.last_logical_writes Current_IO

FROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()and p.name = 'qti_result_data_cr')and p.name = 'qti_result_data_cr')

SELECT count(pk1) * @question_io Test_Question_IO, cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' +cast(DATEPART(MONTH, bbmd_date_added) as

varchar(2)) + '-' +cast(DATEPART(DAY, bbmd_date_added) as

varchar(2)) TheDatefrom qti_result_datawhere bbmd_date_added > @start_dateand bbmd_date_added < @end_dategroup by DATEPART(YEAR, bbmd_date_added),

DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added)

order by DATEPART(YEAR, bbmd_date_added),DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added)

Strategic Functional DecisionsQuery that joins the QTI tables (bbmd_data_added) with the cost of the

queries associated with the insert

AWR reportand v$sql, sqlarea

on qti_result_data_cr

Strategic Functional DecisionsCreate a logging script

declare @ts_now bigintdeclare @start intdeclare @end intdeclare @Start_time datetimedeclare @End_time datetimedeclare @Bb_Activity intdeclare @QTI_RESULTS_Activity int

set @ts_now = (SELECT cpu_ticks/(cpu_ticks/ms_ticks)FROM sys.dm_os_sys_info);

SELECT TOP(256) SQLProcessUtilization AS [SQL_Server_Process_CPU_Utilization],SystemIdle AS [System_Idle_Process],100 - SystemIdle - SQLProcessUtilization AS [Other_Process_CPU_Utilization],DATEADD(ms, -1 * (@ts_now - [timestamp]), GETDATE()) AS [start_Time],DATEADD(minute, 1, DATEADD(ms, -1 * (@ts_now - [timestamp]), GETDATE())) AS [end_Time],IDENTITY( int ) AS TheCount

into #cpuFROM (SELECT record.value('(./Record/@id)[1]', 'int') AS record_id,

record.value('(./Record/SchedulerMonitorEvent/SystemHealth/SystemIdle)[1]', 'int') AS [SystemIdle],record.value('(./Record/SchedulerMonitorEvent/SystemHealth/ProcessUtilization)[1]', 'int') AS [SQLProcessUtilization], [timestamp]FROM (SELECT [timestamp], CONVERT(xml, record) AS [record]

FROM sys.dm_os_ring_buffersWHERE ring_buffer_type = N'RING_BUFFER_SCHEDULER_MONITOR'AND record LIKE N'%<SystemHealth>%') AS x) AS y

ORDER BY record_id DESC

set @start = 0set @end = (select MAX(TheCount) from #cpu)

CREATE TABLE [dbo].[#cpu_Activity](CREATE TABLE [dbo].[#cpu_Activity]([Activity_Accumulator_Activity] [int] NULL,[QTI_RESULTS_Activity] [int] NULL,[SQL_Server_Process_CPU_Utilization] [int] NULL,[System_Idle_Process] [int] NULL,[Other_Process_CPU_Utilization] [int] NULL,[start_Time] [datetime] NULL,[end_Time] [datetime] NULL,[TheCount] [int] NULL)

WHILE @start < @endBEGIN

SET @start = @start + 1

SET @Start_time = (select Start_Time from #cpu where TheCount = @start)SET @End_time = (select End_Time from #cpu where TheCount = @start)

set @Bb_Activity = (select COUNT(*) from activity_accumulator where timestamp > @Start_time and timestamp < @End_time)set @QTI_RESULTS_Activity = (select COUNT(*) from qti_result_data where bbmd_date_modified > @Start_time and bbmd_date_modified < @End_time)

Insert into #cpu_ActivitySELECT @Bb_Activity, @QTI_RESULTS_Activity, *from #cpuwhere thecount = @start

end

select *from #cpu_activity

drop table #cpu;drop table #cpu_activity;


AWR report, v$sqlselect ... rom activity_accmultor where event_type ='login';


CREATE TABLE [dbo].[#FILE_IO](

[name] [sysname] NULL,

[num_of_bytes_read] [bigint] NOT NULL,

[num_of_bytes_written] [bigint] NOT NULL,

[num_of_reads] [bigint] NOT NULL,

[num_of_writes] [bigint] NOT NULL,

[size_on_disk_bytes] [bigint] NOT NULL

) ON [PRIMARY]

declare @start int

declare @end int

set @start = 5

set @end = (select top 1 database_id

from sys.master_files

where database_id > 5

order by database_id desc)

while @start < @end

begin

insert into #FILE_IO

select name, num_of_bytes_read, num_of_bytes_written, num_of_reads, num_of_writes, size_on_disk_bytes

from sys.dm_io_virtual_file_stats(@start,NULL) s

left join sys.master_files f on s.database_id = f.database_id and s.file_id = f.file_id

set @start = @start + 1

end

select *

from #FILE_IO

drop table #FILE_IO


AWR report, v$sqlselect ... rom activity_accmultor where event_type ='login';


SELECT 'TEST QUESTION' Type, SUM(qs.execution_count) Execution_Count,SUM(qs.total_logical_reads) Total_Reads,SUM(qs.total_logical_writes) Total_Writes,SUM(qs.total_worker_time) Total_CPUFROM sys.dm_exec_query_stats qsCROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qtCROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qpWHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) LIKE '%INSERT INTO QTI_RESULT_DATA%' --Enter a portion of the query text you are interested inand SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) NOT LIKE 'SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,%'UNIONSELECT 'OTHER TRANSACTIONS' Type,SUM(qs.execution_count) Execution_Count,SUM(qs.total_logical_reads) Total_Reads,SUM(qs.total_logical_writes) Total_Writes,SUM(qs.total_worker_time) Total_CPUFROM sys.dm_exec_query_stats qsCROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qtCROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qpWHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) NOT LIKE '%INSERT INTO QTI_RESULT_DATA%' --Enter a portion of the query text you are interested inand SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) NOT LIKE 'SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,%'


AWR report, and use v$sql, sqlarea to find INSERT INTO QTI_RESULT_DATA%'