This article was downloaded by: [UQ Library]On: 19 November 2014, At: 21:05Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Social Work Education: TheInternational JournalPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cswe20

The Acquisition of Professional SocialWork CompetenciesSharon VitaliPublished online: 06 Feb 2011.

To cite this article: Sharon Vitali (2011) The Acquisition of Professional Social WorkCompetencies, Social Work Education: The International Journal, 30:02, 236-246, DOI:10.1080/02615479.2011.540400

To link to this article: http://dx.doi.org/10.1080/02615479.2011.540400

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

The Acquisition of Professional SocialWork CompetenciesSharon Vitali

A three-year longitudinal cohort study followed one cohort of undergraduate social workstudents (n ¼ 38) from induction to graduation, measuring and tracking the acquisitionof the National Occupational Standards for Social Work. An efficient ‘user-friendly’ set of

mixed methods was embedded within the usual programme audit process. It is proposedthat this methodology can be adopted by any competency-based educational programme

seeking to ensure and improve the outcomes of their curriculum, as well as being used bystudents and practice educators to discuss their perspectives on the student’s progress.

Repeated measures t-tests demonstrated that student self-rating (‘ipsative’)measurement can track and quantify the acquisition of student competencies. However,

ipsative measurement was not significantly correlated with ‘external’ tutors’ and practiceeducators’ (fieldwork supervisors’) ratings or with final degree gradings. This raisesquestions around the validity and inter-rater reliability of ipsative measurement

instruments used to evaluate student performance. Ratings of interviews in the firstsemester made by Voices of Experience (service users) were correlated with placement

marks and final degree grades.

Keywords: Competency; Outcomes; Social Work Education; Professional Education;Professional Suitability; Measurement of Practice; Curriculum; Ipsative; CriterionReferenced; Assessment; Service Users, Fieldwork, Practicum

Introduction1

Macdonald et al. (1992, p. 638) argued that ‘ . . . (it) is entirely reasonable that theproposers of a new training course . . . be asked to explain why a particular approach

has been chosen and whether there is any evidence to back it’. All educators face thechallenge of assuring that their graduates have attained the academic requirements of a

specific award, and ascertaining that the grades awarded are fair and comparable toother educational institutions for the same award. A further level of challenge has been

ISSN 0261-5479 print/1470-1227 online q 2011 Taylor & FrancisDOI: 10.1080/02615479.2011.540400

Correspondence to: Dr Sharon Vitali, Programme Lead for Social Work, Faculty of Health & Life Sciences, Oxford

Brookes University, Jack Straw’s Lane, Oxford OX3 0FL, UK. Email: svitali@brookes.ac.uk

Social Work EducationVol. 30, No. 2, March 2011, pp. 236–246

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

imposed on the educators of social workers, a recently regulated profession in the UK,

to defend the reliability of assessment mechanisms used to assure professionalcompetency.

Competency Based Education and Training (CBET) and Assessment

As previously explained elsewhere (Vitali, 2010), Freeman and Lewis (1998, p. 314)

defined a competent learner as ‘ . . . one who has reached a defined standard ofperformance’. In professional practice, competence is an amalgam of demonstrable

knowledge skills and values, measured against an agreed benchmark of standards, ofwhich the UK National Occupational Standards (NOS) for Social Work (TOPPS,

2002) are an example.O’Hagan (1996) traced the origins of competency based education and training to

the popularity of functional analysis and behaviourism during the 1960s and 1970s,which demanded the disassembling of complexities, reducing phenomena to coreelements to be categorised and re-organised formulaically. Grant’s (1979) seminal

book Competence-Based Reforms in Higher Education, defined the early terminologyand assessment tools.

Whilst CBET became popular in occupations where the activities and standards ofpractice are easily defined, articulated and behaviourally measurable (such as

education, nursing and management) it encountered an equally strong, yet negativereception from other occupational disciplines, including social work. These critics

held that the approach was ‘ . . . viciously reductionist’ (Hyland, 1995), and incapableof capturing and measuring the essence of the organic whole of the individual who

provides human services of a complex nature (Dominelli, 1996). Gambrill (1983),Erault (1994) and Lishman (1998) supported the concept of professional competency,but cautioned that complex critical thinking, professional values and attitudes, which

are essential in social work, are difficult to measure.Philosophically, CBET demands a reverse educational process (Aviles, 2000, p. 1)

where ‘ . . . terminal objectives the student must achieve’ guide the development ofcurriculum, teaching materials and assessment tools (Baer and Frederico, 1978). In

reality, most CBET based programmes may simply attach competency-based outcomesas the measurement instruments for assessment, which are the focus of this project.

Challenges to Evaluating Educational Programmes

Seden (2003) held that a traditional pedagogical approach of normative assessment

(where members of a cohort are assessed against each other) conflicts with thedevelopment of self-reflective, critically thinking professionals, and suggested that

social work education should adhere to principles of androgogy (Knowles, 1980)including ipsative (or self) assessment. Freeman and Lewis (1998) agreed that thismodel (whereby students’ performance is evaluated against the baseline of their own

previous performance) is most appropriate for learners setting their own goals, as inthe case of professional development; yet cautioned that it can be highly susceptible to

Social Work Education 237

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

bias. The addition of a complementary criterion-referenced assessment mechanism

(whereby scores reflect expected levels of performance) seeks to overcome theseindividual vulnerabilities.

The RICET: Development and Evolution of a Robust Measure of Competency Based

Outcomes

The RICET (Reflective Ipsative Competency Evaluation Tool) was developed in anattempt to impose the rigour and reliability of criterion referencing upon ipsativemeasurements of competency-based outcomes. Loosely structured around Bloom’s

(1956) taxonomy of learning outcomes, and Benner’s (1982) novice-to-expert nursingcategories, the instrument employs a five-point Likert scale.

Initially used in Canada for practice learning contracting and evaluation, the RICETwas piloted and validated in a project training students for child protection work

(Csernick et al., 2000) and subsequently adapted (with cultural linguisticconsiderations) for a longitudinal programme evaluation of the effectiveness and

efficacy of the UK post-qualifying (PQ) Child Care Award (Vitali, 2005). For thepresent project it was again modified incorporating the National Occupational

Standards (TOPPS, 2002) which define the expectations for qualified UK socialworkers. The NOS document hierarchically maps five key roles, overarching 21national occupational standards (NOS), each of which branch into three or four

detailed descriptions of applications to practice. For the RICET, the 21 NOS wereselected as the outcome measures to be rated on a scale of 0–5. In pursuit of reliability,

the RICET uses clearly delineated gradients for scoring, where observable differencesprovide ‘anchoring’ to the Likert scale (Figure 1).

The RICET seeks to bridge the potential dichotomy between cognitiveunderstanding (inclusive of theory) as can be evidenced by academically capable

students, and confident proficiency in practice interventions, by demanding a fusionof knowledge and skill at the measurement point of 2 and upward. A student whoacquires one in isolation from the other cannot progress on the scale.

The initial design of the RICET sought to quantify and measure the competenciesexpected of students in their practice learning, within a format that would enable and

enhance collaborative contracting for practice learning. For this project, the RICETwas used solely to collect measurements of competency at different points in the

0. Nounderstandingor skills

1. Minimallevel ofunderstandingor skills

2. Someunderstandingof what theexpectationmeans and itspurpose

3. Increasedunderstandingand beginningto initiateattempts to putinto practice

4. Advancedunderstanding &demonstratingadequate levelof integrationof knowledge,skills, andappropriateapplication

5. Clearlyunderstands anddemonstratesconsistent andappropriateapplication ofknowledge andskills in practice

Figure 1 RICET Rating Scale.

238 S. Vitali

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

course and from different perspectives (student, tutor, practice educators). The

respondents completed the document independently (hard copy or via the virtuallearning environment on the web) at the specified measurement point. The RICET

measurements were NOT discussed as part of supervision or practice evaluation, andthe respondents were asked NOT to keep a copy for reference, in order to reduce the

bias towards incremental advancement.

Evaluation Design

A pre-test/post-test design was selected to measure the effectiveness of the qualifyingeducational programme to equip students with the required social work competencies.

The National Occupational Standards (NOS) criterion referenced competencies

were the unifying standardised outcome measurements for this project. However tocompensate for, and protect against the methodological vulnerabilities described

above, multiple measures, from different respondents [student, tutor(s), service usersand carers, practice teachers, independent researcher], using multiple methods

(academic and reflective papers, lab performance, practice performance, computerbased exams, viva performance and dissertation writing) were collected and collated.

This compensatory triangulation method of data collection (Jacob, 1990) allows fortesting of the consistency of the findings (Greene et al., 1989), and highlights possibledifferences between the findings from the various sources for further micro-

investigation.For ease of data collection, the project was designed around existing audit

mechanisms and mapped against the learning activities, thereby enabling a rollingcurriculum review with no added assessment burden. As standard programme

assessments, all students were required to participate, but participation in theevaluation was optional. Despite advising all students (and their practice teachers) of

the right to ‘opt out’ of the evaluation component of the project, there was significantenthusiasm for the project and good response compliance.

Procedure

The RICET was the most frequently used measurement instrument (Figure 2).Students completed a baseline self-assessment (PRE-) during their first week on the

programme, introducing them to the knowledge and skill focus of the course.After the first semester, students developed a learning plan demonstrating their

understanding of the competencies and strategies for acquiring further competency.They then undertook a Voice of Experience (VoE) interview. This unrehearsed

interview with a service user or carer, trained and supported in selecting, educatingand assessing students, is used to assure students’ ‘Fitness to Practice’ prior to practice

learning placements. The interview was conducted and videotaped in acommunication suite with a one-way mirror eliciting independent scores fromtutors, VoE observers and the interviewee. An independent researcher subsequently

viewed and evaluated the taped interviews to benchmark evidence of interviewproficiency and investigate the reliability of user/carer ratings.

Social Work Education 239

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

At the end of Year 1 (SES1) and prior to their first practice learning placement (SES2),the students again completed the RICET. Subsequently, while on placement, both student

and their practice educator (fieldwork supervisor) completed RICET scoring indepen-dently, at mid-placement after 30 days (SMP1) and at the end after 100 days (SEP1) in

Placement 1. This pattern was repeated in their second placement (SMP2 and SEP2).Academic measurements were achieved through a variety of assignments

(portfolios, essays, online tests, case vivas, critical career reviews and dissertations),which were graded by practice teachers and lecturers from the Social Work Unit and

from the other disciplines where curriculum is shared (e.g. critical evaluation ofresearch, partnerships in practice).

Data Analysis

Collected data were anonymised, and analysed using SPSS. Changes in mean ratings

between time points were tested using t-tests. Associations between RICET ratingsmade by participants in the study were investigated by calculating using the Pearsonproduct-moment correlation co-efficient. The small size of the sample precluded sub-

group analyses of student demographic variables and their associations withperformance on the course.

Participants

A single cohort (n ¼ 38) was followed from a baseline measurement in their first week

of classes, throughout the three years of the full-time qualifying BA (Hons) in SocialWork. Part-time candidates and those delayed through failures or personal

When? What & Who? What & Who? What & Who? What & Who? What & Who?Year1Semester 1

RICET

Self

LEARNING PLANTutor

RICET

Self

ACADEMIC WORKTutors

Year 1Semester 2

VoEINTERVIEW*Service Users &Carers

RICET

Self

PORTFOLIO

Tutor

VoE INTERVIEW *IndependentResearcher

ACADEMIC WORKTutors

Year 2Semester 1

RICET

Self

RICET

Practice Teacher

PORTFOLIO

Practice Teacher & Tutor

ACADEMIC WORKTutors

Year 2Semester 2

RICET

Self

RICET

Practice Teacher

PORTFOLIO

Practice Teacher & Tutor

ACADEMIC WORKTutors

Year 3Semester 1

RICET

Self

RICET

Practice Teacher

PORTFOLIO

Practice Teacher& Tutor

CASE VIVA

Tutors

ACADEMIC WORKTutors

Year 3Semester 2

RICET

Self

RICET

Practice Teacher

PORTFOLIO

Practice Teacher& Tutor

CRITICAL CAREER REVIEWTutors

ACADEMIC WORKTutors

PLUS: DEGREE AWARD & GRADE

Figure 2 Measurement Schedule.

240 S. Vitali

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

circumstances were excluded from this sample; however, they are a subset, which

warrants exploration to inform policies concerning admissions and programmedelivery. The sample at graduation (n ¼ 32) provided an 84% completion rate. An

overview of the sample demographics (Table 1) revealed a cohort profile typical forthis university.

Findings

The distributions of ipsative (self ratings) of students’ competences (RICET scores) areshown in Figure 3. As can be observed in the box plots, the students’ reporting of their

competency was incremental from their baseline (PRE) at the start of the course andthrough their first year of introductory academic modules and training in thecommunication suite with the Voices of Experience (S End of Semester 1 & 2).

The chart indicates that the pattern of consistent increase was interrupted duringthe students’ first placement (SMP1) when there was no apparent increase in mean

ratings. However, this was followed by an increase by the end of 100 days (SEP1),which continued throughout their second placement (SEP2).

Analysis of differences in the mean RICET ratings for matched pairs of students atsuccessive time points are shown in Table 2. This indicates statistically significant

increases between each time point, with the exception of pair 5. Note that the samplesize reduces as the number of students who completed the ratings on all precedingtimepoints decreases.

Associations between Ipsative and External Scoring

The methodology was designed to enable consideration of the associations betweenthe students’ ipsative scoring and the criterion-referenced assessments of externalparticipants such as the tutors’ and the practice educators’ RICET ratings, the VoE

observations and the academic grades. The correlations between these ratings areshown in Table 3.

Table 1 Demographic Profile of Cohort.

Variables

Age at entry Mean ¼ 29.55 years Range ¼ 18–50 years(SD ¼ 9.66)

Gender Female ¼ 32 Male ¼ 6Self-reported ethnicity British ¼ 32 African ¼ 2

European ¼ 1 Caribbean ¼ 3English First language ¼ 32 Second language ¼ 6Previous education Secondary school ¼ 14 Previous UK degree (HEI) ¼ 6

Further Education ¼ 17 Previous overseas degree ¼ 1Dyslexia Stated ¼ 5 Not stated ¼ 33Disability/medical Declared ¼ 9 No or not stated ¼ 29Childcare demands Declared ¼ 8 Not declared ¼ 30

Note: n ¼ 38.

Social Work Education 241

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

In the first and second semesters of Year 1, students’ RICET ratings were not

significantly correlated with the ratings made by their tutors, the VOE score or the

independent researcher’s rating. They were weakly correlated with the academic grades

at the end of Semester 1, but not Semester 2. The most robust correlation was between

the tutor’s RICET score and the student’s academic grade. The VOE ratings and the

researcher’s rating were also strongly correlated.

MEAN P

RE

MEAN S

ES1

MEAN S

ES2

MEAN S

MP1

MEAN S

EP1

MEAN S

EP2

MEAN S

MP2

5

4

3

2

1

0

Code: PRE = baseline; SES1 = end of semester 1; SES2 = end of semester 2;SMP1 and 2 = placement mid-point for placement 1 and 2; and SEP1 and 2 = end of placement.

Figure 3 Ipsative (Self-ratings) of Competency at Progressive Time Points (RICETScale, Range 0–5).

Table 2 Mean Ipsative (Self-) Ratings at Successive Time Points (Matched Pairs):Analysis of Differences.

Paired ratings n Mean Std dev.95% CIlower

95% CIupper t df p

1 PRE–SES 1 32 –0.403 0.7061 –0.658 –0.149 –3.23 31 0.0032 SES 1–SES 2 32 –0.372 0.6606 –0.610 –0.134 –3.19 31 0.0033 SES 2–SMP1 17 –0.404 0.6095 –0.717 –0.090 –2.73 16 0.0154 SMP1–SEP1 9 –1.026 0.6934 –1.559 –0.493 –4.44 8 0.0025 SEP1–SMP2 7 –0.279 0.3631 –0.615 0.056 –2.03 6 0.0886 SMP2–SEP2 6 –0.730 0.3007 –1.046 –0.415 –5.95 5 0.002

242 S. Vitali

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

Table

3A

sso

ciat

ion

sb

etw

een

Mea

sure

men

ts

Mea

sure

men

tC

orr

elat

ion

bet

wee

nN

Pea

rso

nSi

g

Year

1Se

mes

ters

1an

d2

Stu

den

t£

Tu

tor

RIC

ET

sco

res

(en

do

fSe

mes

ter

on

e)32

0.15

70.

390

Stu

den

tR

ICE

Tsc

ore

£A

cad

emic

Gra

de

(en

do

fSe

mes

ter

on

e)32

0.41

2 *0.

019 *

Tu

tor

RIC

ET

sco

re£

Aca

dem

icG

rad

e(e

nd

of

Sem

este

ro

ne)

320.

704 *

*,

0.00

01**

*St

ud

ent

RIC

ET

sco

re£

Vo

Esc

ore

(en

do

fSe

mes

ter

on

e)32

0.21

90.

228

Stu

den

tR

ICE

Tsc

ore

£In

dep

end

ent

Res

earc

her

sco

re32

0.02

80.

880

Stu

den

tR

ICE

TSc

ore

£A

cad

emic

Gra

de

(en

do

fSe

mes

ter

two

)32

0.26

90.

136

Vo

Esc

ore

£A

cad

emic

Gra

de

320.

392 *

0.02

7 *In

dep

end

ent

Res

earc

her

sco

re£

Aca

dem

icG

rad

e32

0.22

30.

220

Vo

Esc

ore

£In

dep

end

ent

Res

earc

her

sco

re33

0.54

3 **

0.00

1 **

Sem

este

r3

Mid

-pla

cem

ent

1St

ud

ent£

Pra

ctic

eE

du

cato

rR

ICE

Tsc

ore

s11

0.35

70.

281

Stu

den

tR

ICE

Tsc

ore

£A

cad

emic

Gra

des

17–

0.02

80.

914

Pra

ctic

eE

du

cato

rR

ICE

Tsc

ore

£A

cad

emic

Gra

de

170.

329

0.19

7Se

mes

ter

4E

nd

-pla

cem

ent

1St

ud

ent£

Pra

ctic

eE

du

cato

rR

ICE

Tsc

ore

1U

-AU

-ASt

ud

ent

RIC

ET

sco

re£

Aca

dem

icG

rad

e10

0.16

60.

646

Pra

ctic

eE

du

cato

rR

ICE

Tsc

ore

£A

cad

emic

Gra

de

3–

0.99

9 *0.

035 *

Sem

este

r5

Mid

-pla

cem

ent

2St

ud

ent£

Pra

ctic

eE

du

cato

rR

ICE

Tsc

ore

s7

0.53

90.

212

Stu

den

tR

ICE

Tsc

ore

£A

cad

emic

Gra

de

170.

465

0.06

0P

ract

ice

Ed

uca

tor

RIC

ET

sco

re£

Aca

dem

icG

rad

e11

–0.

191

0.57

5Se

mes

ter

6E

nd

-pla

cem

ent

2St

ud

ent£

Pra

ctic

eE

du

cato

rR

ICE

Tsc

ore

40.

665

0.33

5St

ud

ent

RIC

ET

sco

re£

Aca

dem

icG

rad

e6

0.50

70.

305

Pra

ctic

eE

du

cato

rR

ICE

Tsc

ore

£A

cad

emic

Gra

de

4–

0.33

10.

669

Notes

: *C

orr

elat

ion

issi

gnifi

can

tat

the

0.05

leve

l(2-

tail

ed).

**C

orr

elat

ion

issi

gnifi

can

tat

the

0.01

leve

l(2-

tail

ed).

U-A

:Can

no

tb

eco

mp

ute

db

ecau

seat

leas

to

ne

of

the

vari

able

sis

con

stan

t.

Social Work Education 243

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

Students’ RICET scores were not correlated with their practice educator’s ratings atany of the measurement points during the first or second practice placements.

The practice educator’s RICET score at the end of Placement 1 was negativelycorrelated with the academic grade, but this finding derived from three students onlyand is therefore unconvincing.

Ability of Measurements to Predict Final Degree Grade

An exploratory analysis was undertaken to assess the relationships between the variousscores and the students’ final grading. Three degree bands were awarded according tothe standard UK system from 2:2 (lowest), 2:1 and 1 (highest), based on the final

overall mark each student achieved.Students’ ipsative ratings were not significantly correlated with the final degree

grade (Table 4). There was a strong correlation between the placement grades and thefinal degree award but this is not surprising as mathematically they contribute

significantly to the grade. There was a weak association between the tutor grading atthe end of Semester 1 and the final degree award. There was a strong correlation

between the VOE score and the final grade.

Discussion

This exploratory study set out to investigate the use of multiple methods andmeasurements of students’ performance during their degree programme. The

conclusions are limited due to the very small sample size and because the data weregathered from one cohort of students at one university.

The results of the students’ ipsative RICET ratings demonstrate consistent

statistically significant increases over the course of the degree programme. Thisindicates that the students became increasingly self-confident in their ability to

Table 4 Associations between Measurements and Final Degree Grade (n ¼ 32).

Measurement(respondent)

Score/instrument Mean

Degree 2:2(n ¼ 8)

Degree 2:1(n ¼ 19)

Degree 1:0(n ¼ 5)

p Pearsoncorrelation

Baseline (Student) RICET 1.63 1.42 1.67 1.79 0.181End Sem 1 (Student) RICET 2.03 2.00 1.97 2.33 0.096End Sem 1 (Tutor) Score

PORTFOLIO1.16 0.91 1.21 1.36 0.379*

End Sem 1 (VoE) ScoreINTERVIEW

38.24 33.13 38.42 44.0 0.554**

End Sem 1(Independent Researcher)

ScoreINTERVIEW

29.39 27.38 29.42 33.4 0.375*

End Sem 2 (Student) RICET 2.40 2.17 2.41 2.76 0.322Practicum 1 Grade(Practice Educator & Tutor)

Grade/PORTFOLIO

62.46 54.13 62.84 75.60 0.748**

Practicum 2 Grade(Practice Educator & Tutor)

Grade/PORTFOLIO

60.94 53.50 60.47 74.60 0.877**

Notes: * Correlation significant at the 0.05 level (2-tailed). ** Correlation significant at the 0.01 level(2-tailed).

244 S. Vitali

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

demonstrate their competency in relation to the National Occupational Standards

required for them to graduate as a qualified UK social worker. However, the surprising

finding from this study was of the general lack of correlation between individual

students’ ipsative ratings and those made by their tutors and practice teachers. This

finding questions the increasingly common use of self-efficacy as an outcome measure

for social work education. Further investigations of the association between ipsative

ratings and external ratings are warranted with a larger sized sample.One particularly interesting finding was the ability of the ratings of the Voices of

Experience (service users) to identify those students who would do well in their

subsequent placements and their final degree award. These VOE ratings were made

following early time-limited exposure to the students, both as interviewees and

observers. This observation reinforces the commitment to embedding these experts in

the educational process.

The project demonstrates how competency based programmes can employ an

inexpensive, user-friendly methodology within a standard university audit protocol. It

can be used for measuring and evidencing the acquisition of competencies and for

identifying curriculum weaknesses for immediate redesign and refinement. Although

the ipsative and criterion-referenced measures are inconsistent, they may also be used

in demonstrating to internal and external parties where the programme is meeting the

required outcomes. The study findings also demonstrate that more time and effort

must be invested in ensuring inter-rater reliability of the measurement instruments

employed and congruence between different types of measurements or run the risk of

the ‘illusion’ of valid and reliable outcome measurements.

Note

[1] Some portions of this paper have been previously published in Vitali (2010) and also draws onVitali (2005).

References

Aviles, C. B. (2000) Creatively Adapting Mastery Learning and Outcomes Based Education to the SocialWork Classroom, Paper presented at the Joint Conference of the International Federation ofSocial Workers (IFSW) and the International Association of Schools of Social Work, IFSW,Montreal.

Baer, B. & Fredrico, R. (eds) (1978) Educating the Baccalaureate Social Worker, Ballinger, CambridgeMA.

Benner, P. (1982) ‘From Novice to Expert’, American Journal of Nursing, vol. 82, no. 3, pp. 402–407.Bloom, B. (ed.) (1956) Taxonomy of Educational Objectives: Handbook I Cognitive Domain, Longman,

London.Csernick, R., Vitali, S. & Gordon, K. (2000) ‘Student & Field Instructor Perceptions of a Child-

Welfare Competency-Based Education & Training Project’, Canadian Social Work, vol. 2, no. 2,pp. 53–64.

Dominelli, L. (1996) ‘Deprofessionalising Social Work: Anti-Oppressive Practice, Competencies andPost Modernism’, British Journal of Social Work, vol. 26, no. 2, pp. 53–175.

Erault, M. (1994) Developing Professional Knowledge and Competence, Falmer Press, London.

Social Work Education 245

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014

Freeman, R. & Lewis, R. (1998) Planning and Implementing Assessment, Kogan Page Limited,London.

Gambrill, E. (1983) Casework: A Competency-Based Approach, Prentice Hall, Englewood Cliffs, NJ.Grant, G. (ed.) (1979) On Competence: A Critical Analysis of Competence-Based Reforms in Higher

Education, Josey-Bass, San Francisco, CA.Greene, J., Caracelli, V. & Graham, W. (1989) ‘Towards a conceptual framework for mixed-method

evaluation designs’, Educational Evaluation & Policy Analysis, vol. 11, no. 3, pp. 255–274.Hyland, T. (1995) ‘Morality, work and employment: towards a values dimension in vocational

education and training’, Journal of Moral Education, vol. 24, no. 4, pp. 445–456.Jacob, E. (1990) ‘Alternative approaches for studying naturally occurring human behavior and

thought in special education research’, The Journal of Special Education, vol. 24, no. 2,pp. 195–211.

Knowles, M. (1980) The Modern Practice of Adult Education. From Pedagogy to Andragogy, 2nd edn,Prentice Hall/Cambridge Books, Englewood Cliffs, NJ.

Lishman, J. (1998) ‘Personal & professional development’, in Social Work: Themes Issues and CriticalDebates, eds R. Adams, L. Dominelli & M. Payne, Macmillan, London.

MacDonald, G., Sheldon, B. & Gillespie, J. (1992) ‘Contemporary Studies of the Effectiveness ofSocial Work’, British Journal of Social Work, vol. 22, no. 6, pp. 615–643.

O’Hagan, K. (ed.) (1996) Competence in Social Work Practice: A Practical Guide for Professionals,Jessica Kingsley Publishers, London & Bristol.

Seden, J. (2003) Counselling Skills in Social Work Practice, The Cromwell Press, Towbridge.Training Organisation for the Personal Social Services (TOPPS) (2002) National Occupational

Standards for Social Work, TOPPS, Leeds.Vitali, S. (2005) Lessons from the Front Line: Evaluating the Post Qualification Child Care Award,

DPhil thesis, University of Oxford.Vitali, S. (2010) ‘Can we measure and track the acquisition of competency on a professional social

work degree?’, in The Outcomes of Social Work Education: Developing Evaluation Methods, edsH. Burgess & J. Carpenter, Higher Education Academy Subject Centre for Social Policy andSocial Work.

246 S. Vitali

Dow

nloa

ded

by [

UQ

Lib

rary

] at

21:

05 1

9 N

ovem

ber

2014