Broad-Based Office Broad-Based Office Skills TestingSkills Testing

Los Angeles County Office of EducationLos Angeles County Office of EducationPERSONNEL COMMISSIONPERSONNEL COMMISSION

The ProblemThe Problem

Many clerical/secretarial classificationsMany clerical/secretarial classifications Differ on specific job content and levels of responsibilityDiffer on specific job content and levels of responsibility Many obstacles to class consolidation - logistical, political, Many obstacles to class consolidation - logistical, political,

fiscalfiscal Each class has a unique examinationEach class has a unique examination

Variably content and qualityVariably content and quality Not consistently updatedNot consistently updated 2-3 month recruitment & examination cycle time2-3 month recruitment & examination cycle time

No overall competency architecture across the classesNo overall competency architecture across the classes Core competencies and competency progressionCore competencies and competency progression Pass point paradoxes (lower pass points for higher level Pass point paradoxes (lower pass points for higher level

jobs based on selection ratios)jobs based on selection ratios)

Purpose and ObjectivesPurpose and Objectives

Improve testing quality, reliability, and Improve testing quality, reliability, and efficiencyefficiency

Improve coherence of clerical examination Improve coherence of clerical examination plansplans

Reduce redundant testingReduce redundant testing Reduce time to place candidates on eligibility Reduce time to place candidates on eligibility

listslists Ensure eligibility lists are always availableEnsure eligibility lists are always available

StrategyStrategy

Core test battery for as many classifications as Core test battery for as many classifications as possiblepossible

Annual calendar of assessment opportunities Annual calendar of assessment opportunities (e.g., quarterly)(e.g., quarterly)

Computer-based testingComputer-based testing Score banking and certification to different Score banking and certification to different

exam plansexam plans Different weights and cut-off scores for different Different weights and cut-off scores for different

classifications based upon relevancy and level classifications based upon relevancy and level needed.needed.

Strategy: Strategy: Similar to SAT & ACT for applying to collegeSimilar to SAT & ACT for applying to college

1.1. Sign up and take Sign up and take the testthe test

2.2. Apply to the Apply to the colleges of choicecolleges of choice

3.3. Send scoresSend scores

4.4. Colleges set their Colleges set their own pass-points own pass-points and weights for final and weights for final selection decisionsselection decisions

1.1. Sign up and take Sign up and take the testthe test

2.2. Apply to the jobs of Apply to the jobs of choicechoice

3.3. Scores populatedScores populated

4.4. Each job has its Each job has its own pass-points own pass-points and weights for final and weights for final selection decisionsselection decisions

Score Data FlowScore Data Flow

Apply to Take Clerical Battery

(10 scores)

Exam Plan For Battery Score

Bank

Take Battery – CBT and Scanton

Apply to Job Title

Exam Plan For Job Title A 1

(weights and pass points for each

score)

Eligibilty List A

Electronic Certification of Scores to each Exam Plan3

Electronic Score Transfer 2

Exam Plan For Job Title B 1

(weights and pass points for each

score)

Eligibilty List B

Development of the Test BatteryDevelopment of the Test Battery

Reviewed job analyses from Agency-Wide Reviewed job analyses from Agency-Wide Classification Study - 2004-2007Classification Study - 2004-2007

Re-stated duties into “generalized work activity” (GWA) Re-stated duties into “generalized work activity” (GWA) statements with no reference to specific job content.statements with no reference to specific job content.

Map GWAs to each classificationMap GWAs to each classification Identify relevant assessment tools for GWAsIdentify relevant assessment tools for GWAs Via GWAs, map assessment tools to ClassificationsVia GWAs, map assessment tools to Classifications Derive relevancy and weight of assessment tools to Derive relevancy and weight of assessment tools to

classifications. classifications.

Example GWAsExample GWAs

Greet, interact, and provide information to others in person and on Greet, interact, and provide information to others in person and on the telephonethe telephone

Verify, input, and correct data and informationVerify, input, and correct data and information Receive, sort, amd categorize information and materialsReceive, sort, amd categorize information and materials Understand, interpret, explain, and follow directions, rules, and Understand, interpret, explain, and follow directions, rules, and

regulationsregulations Operate a personal computer and other common electronic office Operate a personal computer and other common electronic office

equipmentequipment Format, edit, keyboard and type documents.Format, edit, keyboard and type documents. Gather compile, collate, and assemble information and materialsGather compile, collate, and assemble information and materials Maintain and coordinate calendars and schedule meetings and Maintain and coordinate calendars and schedule meetings and

eventsevents Receive, track, and record monetary/numerical information and dataReceive, track, and record monetary/numerical information and data

Matrix of GWA with ClassificationsMatrix of GWA with Classifications

ClassClass

GWAGWA

Typist ClerkTypist Clerk SecretarySecretary AccountingAccounting

ClerkClerk

ReceptionistReceptionist

Greet, interact, and Greet, interact, and provide information to provide information to others in person and others in person and on the telephoneon the telephone

11 22 11 22

Verify, input, and Verify, input, and correct data and correct data and informationinformation

22 11 22 00

Receive, sort, amd Receive, sort, amd categorize information categorize information and materialsand materials

22 22 22 11

Understand, interpret, Understand, interpret, explain, and follow explain, and follow directions, rules, and directions, rules, and regulationsregulations

22 11 22 11

Competencies Assessed - TestsCompetencies Assessed - Tests

1.1. Following Instructions (MC)Following Instructions (MC)

2.2. English Usage & Grammar (MC)English Usage & Grammar (MC)

3.3. Microsoft Outlook (MC)Microsoft Outlook (MC)

4.4. Microsoft Excel (Performance, OPAC)Microsoft Excel (Performance, OPAC)

5.5. Microsoft Word Skills (Performance, OPAC)Microsoft Word Skills (Performance, OPAC)

6.6. Telephone - Customer Service (SJ, OPAC)Telephone - Customer Service (SJ, OPAC)

7.7. Sequencing and ordering speed and accuracy (timed)Sequencing and ordering speed and accuracy (timed)

8.8. Computational speed and accuracy (timed)Computational speed and accuracy (timed)

9.9. Checking and comparing speed and accuracy (timed)Checking and comparing speed and accuracy (timed)

10.10. Data Entry speed and accuracy (OPAC)Data Entry speed and accuracy (OPAC)

Matrix of Tests with GWAsMatrix of Tests with GWAs

TestTest

GWAGWA

Telephone Telephone Customer Customer ServiceService

Data Entry Data Entry speed and speed and accuracyaccuracy

Sequencing Sequencing and ordering and ordering speed and speed and accuracyaccuracy

Following Following Written Written InstructionsInstructions

Greet, interact, and Greet, interact, and provide information to provide information to others in person and others in person and on the telephoneon the telephone

xx

Verify, input, and Verify, input, and correct data and correct data and informationinformation

xx xx

Receive, sort, and Receive, sort, and categorize information categorize information and materialsand materials

xx

Understand, interpret, Understand, interpret, explain, and follow explain, and follow directions, rules, and directions, rules, and regulationsregulations

xx

ClassClass

TestTest

Typist Typist ClerkClerk

SecretarySecretary AccountingAccounting

ClerkClerk

ReceptionistReceptionist

Telephone Telephone Customer Customer ServiceService

11 22 11 22

Data Entry Data Entry speed and speed and accuracyaccuracy

22 11 22 00

Sequencing and Sequencing and ordering speed ordering speed and accuracyand accuracy

22 22 22 11

Following Following Written Written InstructionsInstructions

22 11 22 11

Matrix of Tests with ClassificationsMatrix of Tests with Classifications

ResultResult

Linkage of tests-to-tasks-to-classifications to establish Linkage of tests-to-tasks-to-classifications to establish job relatednessjob relatedness

Derived weighting scheme for each classificationDerived weighting scheme for each classification Relevant testsRelevant tests Relevant weightsRelevant weights

Job specific competencies identified in the job analyses Job specific competencies identified in the job analyses are assessed in specialized examination partsare assessed in specialized examination parts Min QualsMin Quals InterviewInterview PerformancePerformance

Applicable ClassificationsApplicable Classifications

1.1. ClerkClerk2.2. Intermediate ClerkIntermediate Clerk3.3. Senior ClerkSenior Clerk4.4. School ClerkSchool Clerk5.5. Senior School ClerkSenior School Clerk6.6. Data Control ClerkData Control Clerk7.7. Senior Data Control ClerkSenior Data Control Clerk8.8. Typist ClerkTypist Clerk9.9. Intermediate Typist ClerkIntermediate Typist Clerk10.10. Temporary Office WorkerTemporary Office Worker11.11. Senior Typist ClerkSenior Typist Clerk12.12. Department Assistant, DanceDepartment Assistant, Dance13.13. Department Assistant, MusicDepartment Assistant, Music

14.14. Department Assistant, TheaterDepartment Assistant, Theater15.15. Department Assistant, Visual ArtsDepartment Assistant, Visual Arts

16.16. ReceptionistReceptionist17.17. ReaderReader18.18. Information Resources SpecialistInformation Resources Specialist

19.19. Media Dispatching ClerkMedia Dispatching Clerk20.20. SecretarySecretary21.21. Division SecretaryDivision Secretary22.22. Legal SecretaryLegal Secretary23.23. School Administrative SecretarySchool Administrative Secretary

24.24. Senior Division SecretarySenior Division Secretary25.25. Executive Legal SecretaryExecutive Legal Secretary26.26. Executive AssistantExecutive Assistant

Affected PositionsAffected Positions

ClassificationClassification # LACOE Positions# LACOE Positions % of Clerical Positions% of Clerical Positions

Senior School ClerkSenior School Clerk 110110 27%27%

SecretarySecretary 7878 19%19%

Division SecretaryDivision Secretary 4646 11%11%

Senior Typist ClerkSenior Typist Clerk 3535 9%9%

School Administrative School Administrative SecretarySecretary

3030 7%7%

Intermediate Typist ClerkIntermediate Typist Clerk 2828 7%7%

Senior Division SecretarySenior Division Secretary 2424 6%6%

TOTALTOTAL 351351 87%87%

Administrative PracticesAdministrative Practices

Score LifeScore Life: : 2 years 2 years

Test Administration:Test Administration: Every 3 monthsEvery 3 months

Test RetakeTest Retake: : Once every 6 monthsOnce every 6 months

Retake Scope:Retake Scope: Whole Battery – not individual tests for logistical reasonsWhole Battery – not individual tests for logistical reasons

Weighted or Qualifying-OnlyWeighted or Qualifying-Only:: Qualifying Pass/Fail-only for Promotional ExamsQualifying Pass/Fail-only for Promotional Exams Weighted for Open recruitment Weighted for Open recruitment

Effective ScoresEffective Scores:: Only the candidate’s most recent score, not the highest Only the candidate’s most recent score, not the highest

score, can be certified to a list. score, can be certified to a list. For candidates whose scores enable them to be placed For candidates whose scores enable them to be placed

on an eligibility list (pass the exam), the certified scores on an eligibility list (pass the exam), the certified scores stands for the duration of eligibility on the respective list. stands for the duration of eligibility on the respective list.

• Scores already certified are not updated by retakesScores already certified are not updated by retakes• One’s score (and relative ranking) on a list can not be improved One’s score (and relative ranking) on a list can not be improved

until eligibility expires (one year).until eligibility expires (one year). Multiple (10) Pass PointsMultiple (10) Pass Points

Low individual test pass pointsLow individual test pass points Reduced Adverse ImpactReduced Adverse Impact 3 apparent factors: 3 apparent factors:

• Computer skills, speed and accuracy, verbalComputer skills, speed and accuracy, verbal

Administrative PracticesAdministrative Practices

Continuous offering of the BBCContinuous offering of the BBC Continuous recruitment for all affected classificationsContinuous recruitment for all affected classifications

Establishes a pre-tested pool of potential candidates for Establishes a pre-tested pool of potential candidates for the affected classesthe affected classes

For most classifications, limited or no public posting for For most classifications, limited or no public posting for job specific recruitmentsjob specific recruitments Notifications sent to pre-tested pool, based on test cut-off Notifications sent to pre-tested pool, based on test cut-off

score score Invitations to apply to take the specialized exam parts for Invitations to apply to take the specialized exam parts for

the classificationthe classification

RecruitmentRecruitment

No need to go to the general population for No need to go to the general population for each recruitmenteach recruitment

General

Population

Recruit To Take BBC

PRE-TESTEDBBC POOL

Recruit for Job B

Recruit for Job A

Recruit for Job C

StatusStatus

Officially launched on February 9th, 2009Officially launched on February 9th, 2009Advertised on our PC website & Weekly Job Advertised on our PC website & Weekly Job

Opportunities emails.Opportunities emails. Two exam administrations in March & April Two exam administrations in March & April

phased in the BBC in the context of specific phased in the BBC in the context of specific classification recruitments.classification recruitments.

June and October 2009 BBC administrations as June and October 2009 BBC administrations as part of annual calendarpart of annual calendar

Process Improvements Since StartProcess Improvements Since Start

Candidates are automatically notified via email when Candidates are automatically notified via email when particular job classifications are open for recruitment.particular job classifications are open for recruitment.

Candidates receive their score report along with an Candidates receive their score report along with an Examination Information Sheet to address frequently Examination Information Sheet to address frequently asked questions.asked questions.

Scores can (soon) be viewed on-line with ID and Scores can (soon) be viewed on-line with ID and PasswordPassword

Candidates who miss the Scantron exam portion can Candidates who miss the Scantron exam portion can re-register to take the next exam administration. re-register to take the next exam administration.

Candidates who are unable to attend the first Computer Candidates who are unable to attend the first Computer Exam can attend the Computer Make-up exam typically Exam can attend the Computer Make-up exam typically scheduled the following week.scheduled the following week.

Benefits Received and ExpectedBenefits Received and Expected

Test quality improvement throughTest quality improvement through Data monitoring and psychometric analysisData monitoring and psychometric analysis Empirical validation researchEmpirical validation research Development of alternate formsDevelopment of alternate forms

Administration efficiencies throughAdministration efficiencies through Managed efficiencies in test administration and score Managed efficiencies in test administration and score

managementsmanagements Minimizing retesting of the same candidates for different Minimizing retesting of the same candidates for different

classificationsclassifications Improved communications with candidatesImproved communications with candidates

Recruitment enhancement throughRecruitment enhancement through Maintaining a pre-tested pool of applicantsMaintaining a pre-tested pool of applicants Having something to offer at job faresHaving something to offer at job fares Reduced eligibility list production cycle timeReduced eligibility list production cycle time

Next Project: BBANext Project: BBA

Broad-Based Administrative ClassificationsBroad-Based Administrative Classifications Administrative, Analyst, and Professional SeriesAdministrative, Analyst, and Professional Series Focus on college gradsFocus on college grads Same general modelSame general model Launch in Q1, 2010Launch in Q1, 2010

Questions?Questions?

Comments?Comments?

Thank you!Thank you!