Evaluation of the - Tasmania Fire Service · Web viewEvaluation of the. Tasmania ... State Training Authority TFS Tasmania Fire Service UFU United Firefighters Union ... you have

Evaluation of the

Tasmania Fire Service’s

Competency Based

Training

Final Report

May 1998

Evaluation of the Tasmania Fire Service’s Competency Based Training

Acknowledgements

I would like to thank the following people for their contribution to this report:

Reference Group

Robyn PearcePeter Alexander Brian HeveyRichard WarwickMarcus SkellyDavid CastellerChris ArnolGraham Campbell

John Williams, Peter Higgs of TAFE Tasmania, Gavin Freeman from the Training Division, plus all those individuals who contributed data to this evaluation via interviews, workshops and the questionnaire.

AuthorJohn Kelleherfor Tastrain27 Toorak AveLenah ValleyTAS 7008Ph 03-62282391Fax 03-62280798Mob 0418 127547

_____________________________________________________________________

Copyright, Tasmania Fire Service May 1998

This work is copyright. Apart from any use as permitted under the Copyright Act, 1968, no part may be reproduced by any process without written permission from the Tasmania Fire Service.

Further Information:Robyn PearceManager (Human Resources)GPO Box 1526RHobart TAS7001

ii


Table of Contents

Page NumberAcknowledgements ii

List of Acronyms 4

Executive SummarySummary of Recommendations

57

Background to the Evaluation 12

Aims of the Evaluation and Methodology 14

Results of the Questionnaire 15

Analysis of the Findings and RecommendationsNational Influences on the TFS TrainingThe CBT CourseDelivery Methods and Module Sequence

On Station TrainingTAFE CML TrainingBlock Training at Cambridge

Assessment ProcessesTAFE Assessment PracticesTFS Assessment Practices

Suitability of Module ContentRecognition of Prior Learning (RPL) and Recognition of Current competencies (RCC)Relationship with the Training Providers (TAFE and Training Division)

TAFETraining Division

Resource DevelopmentTraining DivisionTAFE and AFAC

Strategic Planning and Training Policy Development

303032333436394141434548

51

515152525455

Attachments

1. Terms of Reference2. Methodology3. Data Collected from the Focus Workshops4. Copy of Questionnaire

61626677

List of Figures

Figure 1 Figure 2

631

3


List of Acronyms

AFAC Australasian Fire Authorities Council

AFC Australian Fire Competencies

ANTA Australian National Training Authority

AQF Australian Qualifications Framework

ARF Australian Recognition Framework

CBT Competency Based Training

CML Computer Managed Learning

FIAT Fire Industry Assessment Template

IT Information Technology

NFROT National Framework for the Recognition of Training

NTF National Training Framework

OH&S Occupational Health and Safety

OTEN Open Training and Education Network

OTFE Office of Training and Further Education (Victoria)

OVET Office of Vocational Education and Training

RCC Recognition of Current Competency

RPL Recognition of Prior Learning

RTO Registered Training Organisation

SOP Standard Operating Procedure

TASTA Tasmanian State Training Authority

TFS Tasmania Fire Service

UFU United Firefighters Union

VEETAC Vocational Education, Employment and Training Advisory

Committee

4


Executive Summary

This evaluation was conducted in a climate of major national review of training across the Fire Services. AFAC indicated that a new national training system, to be known as the Public Safety Training Package, was under development and was likely to impact the Fire Services by mid 1999. This new training system will see the current qualifications move away from the current AFAC courses to the competency standards. CBT, as the principal training approach, will be maintained.

Qualifications will in future be based on meeting the competency standards, and the training modules and courses will become optional learning strategies for Fire Agencies. The effect of these changes will see more emphasis being placed on work based learning on station. This will result in greater responsibility being placed on Brigades to manage their own training.

With these significant changes on the horizon, the recommendations in this report have been made to encourage an orderly transformation of the training function to the users (Brigades). In effect, the TFS has a window of opportunity of approximately 12 months to ramp up its training infrastructure so that it will be in a position to fully embrace the new Training Package regime in 1999. To facilitate this process, most of the recommendations in the report are integrated to build to a holistic outcome, to streamline the training and provide quality improvement.

The most significant finding of the evaluation was that the majority of the users of the training (trainees, trainers, managers/coordinators of training on station) have significant concerns about the training. The nature of those concerns varied considerably, depending upon one’s role in the training.

The introduction of CBT in mid-1995 was felt by many to be premature, given that most of the resources and much of the infrastructure were not in place at the time. As a consequence of this, the Training Division started to develop the resources, while many people were forced to wait for access to particular modules. Compounding this was the decision to focus less on the AFAC course structure in favour of a pay point system directly related to the TFS's needs. This resulted in trainees having to pass more modules than was required under the AFAC curriculum requirement. This issue has now been addressed with the publication of the new pay point system in March 1998, although its legacy remains.

Of the three types of training delivery approaches used, the on station training and the block release training at Cambridge were the most favoured. The TAFE CML training approach was the least favoured of the three.

With the introduction of the National Training Framework from 1/1/98, it is imperative that the TFS address the issues outlined in this report. Over the period 1995 to early 1998, the Training Division was forced to operate in a reactive mode because of the lack of resources and RCC data available. The opportunity is now available to become more proactive in the way the training function is managed within the TFS.

5


Strategically, this report has attempted to position the training in such a way that conversion by the TFS to a Training Package in 1999 will be straightforward and likely to be accepted by the majority of stakeholders. At the forefront of these recommendations for change was the need to link the training to the TFS’s corporate plan in a strategic way.

Figure 1 below summarises how the various recommendations link together to provide a strategic response to the evaluation findings. The two shaded boxes depict, at the top, the external influences about to impact on the training, and the strategic responses recommended to the TFS (bottom box). In the middle (unshaded boxes) are the various issues that have been raised in the evaluation by the many respondents.

Figure 1 Summary of how the recommendations relate to the training

Public Safety Training Package in 1999National influences

Course registration requirements

Responses to the concerns expressed by career staff

Strategic responses

Recommendations

Recommendation 1

Recommendations 2, 13,

Recommendations 3, 4, 5, 6, 10,

Recommendations 7, 8, 11, 12,

Recommendations 9, 14, 15, 16, 17, 18

AccreditedAFAC Courses

Pay Point System

Course delivery, assessment and RPL/RCC

Resource Development

TAFE Component

CML

TFS Training Component

Devolution of more training responsibilities to the Brigades

6


Summary of Recommendations

Recommendation 1

That the TFS review its obligations as a Registered Training Provider, to ensure that it meets the requirements in relation to the delivery of the following accredited courses:

1927 Certificate 11 in Firefighting Operations1928 Certificate 111 in Firefighting Operations1929 Certificate 1V in Firefighting Operations1931 Diploma of Firefighting (Management)1930 Advanced Diploma of Firefighting (Management)

and that every effort be made to issue qualifications to eligible recipients as soon as possible.

Recommendation 2

That an audit of all teaching and learning resources relevant to the competency based training be undertaken across the TFS and interstate. The audit should attempt to identify all existing resources, both those commercially produced and those developed informally by TFS trainers. The purpose of the audit should be to: Identify what resources are available on each station and in each Brigade, so that

any deficiencies/gaps can be addressed The nature (brief description) of each resource The module and competency standard to which each resource relates The currency of the resource The author of the resource The availability of the resource for borrowing/purchase/downloading.

To facilitate this process, it is recommended that a single page template be developed that contains each of the above items, and that all trainers be encouraged to complete this form each time a new teaching/learning resource is developed. From the material collected in this project, it should be possible to establish an electronic data base on the TFS internal network. If all trainers are encouraged to update the data base as new material is developed, it will provide the first port of call of all trainers before they begin to develop new lesson material.

Recommendation 3

That TAFE staff be invited by the TFS to visit the four Brigades around the State at least twice a year, and provide a face to the voice at the end of the phone. During such visits, TAFE staff should be invited to hold small group and one-on-one meetings with station personnel to discuss any issues affecting their training. In addition these visits should be used to disperse information about the TAFE training options, availability of new resources, and generally bring everyone up to date with the TAFE component of the training. This strategy will require cooperation and logistical support from the Brigades.

7


Recommendation 4

That the TFS write to the Flexible Learning Centre, TAFE Tasmania executive and Margaret Thurstans, executive officer of the Public Administration, Finance and Business Services Industry Training Advisory Board, to explore the possibility of TAFE being able to allocate additional resources to support John Williams in the delivery of the AFAC curriculum to the TFS.

Recommendation 5

That the TFS negotiate with the TAFE Flexible Learning Centre about its concerns relating to the CML system and request that appropriate action be taken. Specifically: The elimination of all technical errors associated with the assessment bank e.g

overlapping/duplicated questions, incorrect answers to questions, etc Liberalising, or eliminating the time out process Eliminating the need for trainees to undertake all progressive tests before gaining

access to the final assessment (this issue is also covered later under assessment, over-training and resource development).

This recommendation, if accepted, will require additional resources be allocated to the Flexible Learning Centre (see Recommendation 4 above)

Recommendation 6

That the option of delaying the introduction of some TAFE modules for new recruits be investigated to see if it is feasible. This is in response to the concerns about the readiness of some trainees to commence self-directed learning early in their training.

Recommendation 7

That the initial block training for recruits being delivered at Cambridge be maintained at approximately 12 weeks duration. This block should focus on firefighting competencies, but also include an awareness program of approximately two to three days on Community Fire Safety. In addition, this initial block should also include face-to-face delivery of the module, Computer Skills (see also Recommendation 10 relating to this module). Approximately 9-12 months after their initial block training is completed and consolidated on shift, trainees should be required to commence the various modules relating to Community Fire Safety on station.

Recommendation 8

That the roles of assessor and trainer be merged to allow trainers to also provide final assessments, and if appropriate, allow assessors to also train. To support this action, it is likely that additional training programs (Workplace Trainer Category 1 and Workplace Assessor ) be conducted in the three regions as soon as possible.

To maintain quality assurance, the following actions should be initiated:

All assessors use a common assessment instrument developed and maintained by the Training Division

8


Grievance/appeal procedures within the Brigade structure be available The role of validator be amended to include a systematic auditing function within

the Brigade structure That all assessment instruments be held in a secure location within each Brigade

under the authority of the Brigade Chief.

Recommendation 9

That the TFS develop appropriate policy on assessment, similar to its RPL policy, to provide advice and guidance to Brigade Chiefs in the administration of the assessment process within their respective Brigades. The policy should address the general principles of competency based assessment based on the evidence1 model, and among other things, identify the role and function of the key stakeholders and elaborate on quality assurance procedures and appeal procedures available within Brigades.

Recommendation 10

That the TFS advise TAFE that in future, it intends to have the module Computer Skills delivered in a face-to-face mode, as part of the initial block training for recruits. That the TFS consult with TAFE Tasmania, or another appropriate computer training provider, or its own internal IT training personnel, to arrange for the delivery of the training.

For existing trainees, it is recommended that the TFS write to the Flexible Learning Centre and request that the Computer Skills module be delivered as a face-to-face short program (up to two days) within each region to ensure that the 20 current trainees successfully meet the learning outcomes of the module descriptor as soon as possible.

For other career staff who may wish to enrol in this module in future, it is recommended that the TFS negotiate with TAFE to deliver the module face-to-face within the local region, subject to viable class sizes being established.

Recommendation 11

That an appropriate appeal process, consistent with the TFS’s RPL policy, be established to conclude the RCC initiative. That this appeal process be managed by the Brigades as an in-house activity and that appropriate guidelines be prepared for Brigades to follow.

Recommendation 12

That the RPL policy be summarised into a simple easy to read brochure and be distributed to all personnel on all stations as part of Recommendation 11. In addition, it is recommended that copies of the full policy be distributed to all Brigades as part of Recommendation 11.

Recommendation 13

1 The evidence model is based on the assumption that assessment is a process of collecting evidence about a person’s level of competency and then judging the adequacy of that evidence.

9


That if not already done so, the Training Division investigate interstate training providers that are currently servicing the AFAC curriculum to see if any of the 37 learning packages listed above have already been developed.

That a brief be developed by the Training Division describing the nature of the resource development work to be undertaken and that it be circulated throughout the Agency, and advertised externally if appropriate. That expressions of interest to develop the resources be invited and appropriately skilled individuals be offered the development work on contract. This process should initially be trailed on the development of assessment packages, but could be extended to learning resources if deemed to be appropriate.

Recommendation 14 (see also Recommendation 9)

That the TFS develop a range of training policies dealing with issues such as: responsibilities and role of Brigades, responsibilities and role of training service providers (Training Division and

TAFE), assessment (see Recommendation 9), training delivery, training plans,

in preparation for the arrival of the National Training Package in 1999, and that these policies be designed to establish a strong link between training and the TFS’s Corporate Plan.

Recommendation 15

That responsibility for an individual’s training be devolved to the Brigades, with the Training Division and TAFE acting as service providers. To initiate this process, it is recommended that the RCC process be quickly finalised and that all individuals be invited to appeal if genuinely aggrieved (see Recommendation 11).

Recommendation 16

That Brigades use the outcomes of the RCC process to develop an annual Brigade Training Plan which would be subject to continuous review. That the Training Plan complement and support the implementation of the TFS’s Corporate Plan. That individual training plans for all career staff in each Brigade be developed to encourage participation within the training.

Recommendation 17

That the Training Division establish a training intranet home page to provide a transparent and easily maintained communication channel with the Brigades. That the home page become one of the principal media through which new information about training be transferred to the Brigades from the Training Division. The home page could be established so that it had read only and write to facilities as well as sections where access is restricted to selected people with security code access.

10


Recommendation 18

That the TFS be cognisant of its User Choice rights from 1999 onwards and take the opportunity of investigating the scope and quality of learning resources and training services available by other training providers servicing the AFAC curriculum. This is no reflection on the current training providers.

11


Background to the Evaluation

The Tasmania Fire Service (TFS) decided to adopt the Australasian Fire Authorities Council (AFAC) national fire curriculum in mid-1995, which was developed under a Competency Based Training (CBT) framework. Despite the fact that the AFAC national curriculum was not well resourced at the time, the TFS was anxious to implement CBT into its training. This evaluation covers the introduction of CBT into the TFS over the period mid-1995 to the present.

The AFAC national curriculum consisted of five sequential courses:

Certificate 11 in Firefighting Operations Certificate 111 in Firefighting Operations Certificate 1V in Firefighting Operations Diploma of Firefighting Management Advanced Diploma of Firefighting Management

The courses were subsequently accredited and the TFS, along with the then, Hobart Institute of TAFE, were registered by the Tasmanian State Training Authority (TASTA) to deliver the courses.

The training modules delivered by TAFE were funded through the Institute’s resource agreement with the Office of Vocational Education and Training (OVET). The TFS funded its component of the courses from its internal training budget.

Concurrent with this local training arrangement in Tasmania, the Flexible Learning Centre, a unit within the then, Hobart Institute of TAFE, was successful in receiving two national grants from the Australian National Training Authority (ANTA) to develop learning resources for a Computer Managed Learning (CML) system in order to assist in supporting the national delivery of the AFAC curriculum using flexible delivery. These learning resources were based on the AFAC module descriptors and were intended to provide training in specific modules to cater for all of the fire services affiliated with AFAC.

In addition to the Flexible Learning Centre, other organisations have also been active in developing resources, including AFAC and the Open Training and Education Network (OTEN). Because funding was limited, a decision was made to also utilise learning resources from other national courses, where they were applicable. Consequently, some learning packages from courses such as accounting and office skills were adopted to service the AFAC curriculum.

The process of developing the learning resources was at first slow to develop, and as such little material was initially available to support the flexible delivery mode. However by early 1997, there were sufficient resources in place to introduce the CML system, concurrent with distance learning via the Flexible Learning Centre.

The AFAC CBT curriculum also used a specific assessment process known as the Fire Industry Assessment Template (FIAT). This template provided a blueprint for the

12


development of assessment instruments for each AFAC module descriptor and the administration of the assessment process by course providers. The assessment instruments (known as assessment packages) were optional and were to be developed by each State/Territory Fire Service to meet its own requirements.

The TFS adopted the AFAC CBT curriculum in mid-1995, before any of the learning and assessment resources were developed. Few resources were available at that time, and it has primarily been in the last twelve months that has seen the emergence of significant amounts of learning and assessment resources.

The evaluation was established based on terms of reference developed by the TFS (see Attachment 1).

13


Aims of the Evaluation and Methodology

The evaluation aimed to provide a review of the competency based training system in the TFS. It was to be undertaken with a focus on identifying ways of streamlining the system to ensure that it met the training needs of the TFS and uniformed career staff.

The methodology involved several phases of data collection which included conducting eight focus workshops attended by 87 career staff, undertaking numerous interviews with key stakeholders , analysing training related documents and concluding with the distribution of a statewide questionnaire (see Attachment 5) to all on station career staff. For a full description of the methodology used, please refer to Attachment 2.

The methodology involved a sampling process to ensure a representative view of the whole career staff was achieved. To maximise the representativeness of the data collected, the focus workshops were conducted in all regional areas, roughly in proportion to the staff numbers. Also workshops were held during day and night shifts in Launceston and Hobart to allow as many people as possible to attend. Eighty seven career staff attended the eight workshops, representing approximately 35% of the operational workforce.

The interviews were conducted with all principal stakeholders. The staff from the Training Division and the TAFE Institute had two half day interviews.

The questionnaire was distributed to all Brigades at a time when the holiday change over period occurred. This allowed a maximum number of people to complete the questionnaire. The response rate was 59% (135 returns) based on a population of a maximum of 228 individuals (128 in the South, 64 in the North, 36 in the North West).

14


Results of the Questionnaire

The survey was administered to 228 operational career staff, most of which were on stations around the state. The total returns were 135, representing 59%. Although this return rate was lower than expected, it is still very high for surveys of this type, where typical response rates usually average 30%-50%.

The data have been sectioned into five categories in order to compare responses from specific groups:

All responses (N=135) Trainees (N=64) Trainers (N=58) Managers/coordinators of training on station (N=17) No role (N=27)

Each question has been reproduced as per the original questionnaire and the statistical information presented as a table. The data have been tabulated in two forms, as a raw figure followed by the percentage it represents of the total e.g. 26 (57%). In some columns, the percentages may not equal 100% due to rounding off, or the fact that not all questions were fully answered by all respondents.

Anecdotal comments have been summarised where appropriate. Where the same/similar comment was made by different respondents, the number of respondents making that comment appears at the end of the statement.

15


1. Regional area? South 67 (50%) North 59 (44%) North West 9 (6%)

2. Current rank? Firefighter, First Class Firefighter, Senior Firefighter 74 (55%)Leading Firefighter 19 (14%)Station Officer, Senior Station officer 36 (27%)Higher ranks 7 (4%)

3 Role in the CBT?

Trainee 64 Trainer 58 Assessor 35 Validator 2 RPL facilitator 5 Manager/coordinator of training on station 17 No role 27

(Note that the total adds to more than 135 as many respondents indicated more than one role.)

4. Have you ever attempted to undertake a CBT module?

All respondents

(135)

Trainees (64)

Trainers (58)

Managers/Coordinator

(17)

No Role(27)

Yes 88 (65%) 51 (80%) 38 (66%) 6 (35%) 13 (48%)No 43(32%) 9 (14%) 20 (34%) 11 (65%) 13 (48%)Don’t know 2 (3%) 2 (6%) 0 (0%) 0 (0%) 1 (4%)

If No, please indicate why:

There is no CBT set up for Pay Point 7. No indication there is any need to. Have not been trained under CBT, but have seen and heard from lower ranks. Currently assessed at senior firefighter level. Time constraint requirements. Waiting for the company to get it right. No reason to (3). Too old. No need to complete pay point 6. Completed Associate Diploma (5). Not really interested. Already qualified to officer ranks. They are not ready yet. No need to study any further except optional –roping, etc Not required to (2). There is little or no promotional prospects, so I can’t see the point at my age. Currently finishing TAFE. Unclear as to what is required. No one knows. Over 27 years service. No chance of promotion. At this stage it is not applicable-that is not to say what the future holds. Worked in the region for the last two years and have no reason to use CBT. Not as yet applicable to my position. Not necessary for advancement (2). Still working on the old TAFE Fire Technology course Ignorance of CBT

Italicised responses from the No Role group (Question 3)

16


5 Which of the following best describes your attitude/feelings towards CBT:

All respondents

(135)

Trainees (64)

Trainers (58)

Managers/Coordinators

(17)

No Role(27)

Comfortable 19 (14%) 9 (14%) 9 (16%) 1 (6%) 0 (0%)No opinion 26 (19%) 9 (14%) 5 (8%) 1 (6%) 13 (48%)Concerned 79 (59%) 42 (66%) 40 (69%) 10 (59%) 12 (44%)

6. From your knowledge and experience of the CBT initial recruit training, which of the following approaches do you most favour?

All respondents

(135)

Trainees (64)

Trainers (58)


(17)

No Role(27)

No recruit training at all

2 (1%) 2 (3%) 0 (0%) 0 (0%) 0 (0%)

Short block at Cambridge

4 (3%) 1 (2%) 2 (3%) 0 (0%) 1 (4%)

Short program in local region

4 (3%) 2 (3%) 1 (2%) 1 (6%) 2 (8%)

12 week block at Cambridge

71 (53%) 32 (50%) 29 (50%) 8 (47%) 0 (0%)

16-20 week block at Cambridge

51 (38%) 26 (41%) 26 (45%) 5 (29%) 10 (37%)

Another approach: please indicate

A three to four week block at either Cambridge or in the local region. Either a twelve or 16 week block at Cambridge.

7 If the roles of trainer and assessor were to be merged and all of the trainers were allowed to assess and assessors allowed to train, indicate your response to the following:

All respondents (135)

Trainees (64)

Trainers (58)


(17)

No Role (27)

The credibility of the training would be undermined:

AgreeDisagreeCannot say

39 (29%)61 (45%)28 (21%)

16 (25%)34 (53%)11 (17%)

14 (24%)31 (53%)12 (21%)

5 (29%)7 (41%)3 (18%)

10 (37%)7 (25%)8 (29%)

The assessment and training would be greatly streamlined:

Agree:Disagree:Cannot Say:

98 (73%)16 (12%)16 (12%)

48 (75%)6 (9%)8 (13%)

43 (74%)6 (10%)8 (14%)

13 (76%)1 (6%)2 (12%)

18 (66%)3 (11%)3 (11%)

17



Trainees (64)

Trainers (58)


(17)

No Role (27)

CBT would be more acceptable:


71 (53%)18 (13%)37 (20%)

37 (54%)8 (14%

14 (22%)

29 (50%)11 (19%)16 (28%)

13 (76%)1 (6%)3 (18%)

12 (44%)2 (8%)8 (29%)

8. Indicate whether you agree or disagree with the following statements:


Trainees (64)

Trainers (58)


(17)

No Role(27)

8.1 A firefighter’s role is more than just emergency response. There is an important community education function also.


93 (69%)36 (27%)6 (4%)

42 (66%)17 (27%)5 (8%)

41 (71%)12 (21%)5 (8%)

13 (76%)3 (18%)1 (6%)

15 (55%)12 (44%)0 (0%)

8.2 CBT is under-resourced.


101 (75%)4 (3%)

30 (22%)

49 (77%)1 (2%)

13 (20%)

48 (83%)1 (2%)8 (14%)

13 (76%)0 (0%)4 (24%)

16 (59%)12 (44%)9 (33%)

8.3 The TAFE CML system is flawed

Agree:Disagree:Cannot Say

68 (50%)6 (4%)

62 (46%)

38 (59%)2 (3%)

24 (38%)

30 (52%)3 (4%)

25 (43%)

10 (59%)0 (0%)7 (41%)

9 (33%)18 (66%)0 (0%)

8.4 Many of the modules appear to be out of sequence with the needs of the job


85 (63%)9 (7%)

41 (30%)

44 (69%)6 (9%)

14 (22%)

39 (67%)5 (9%)

14 (24%)

10 (59%)0 (0%)7 (41%)

14 (51%)0 (0%)

13 (48%)

18



Trainees (64)

Trainers (58)


(17)

No Role(27)

8.5 The assessment process is complicated and time consuming


95 (70%)12 (9%)28 (21%)

49 (77%)4 (6%)

11 (17%)

42 (72%)9 (16%)7 (12%)

12 (71%)1 (6%)4 (23%)

17 (62%)1 (4%)9 (33%)

8.6 The RCC/redlining process was unfair


44 (33%)40 (30%)51 (38%)

24 (38%)19 (30%)21 (33%)

21 (36%)19 (33%)18 ((31%)

6 (35%)4 (24%)7 (41%)

7 (25%)7 (25%)13 (48%)

8.7 Should be able to appeal the RCC decision, if genuinely upset by the result


119 (88%)3 (2%)

13 (10%)

56 (88%)3 (5%)5 (7%)

55 (95%)0 (0%)3 (5%)

17 (100%)0 (0%)0 (0%)

21 (77%)0 (0%)5 (18%)

8.8 There is too much training delivered by TAFE that is not directly related to my job


74 (55%)14 (10%)47 (35%)

37 (58%)10 (15%)17 (27%)

27 (47%)6 (10%)25 (43%)

5 (29%)2 (12%)10 (59%)

17 (62%)1 (4%)8 (29%)

8.9 There should be more block training at Cambridge, in addition to the recruit training

Agree:Disagree:Cannot Say

107 (79%)23 (17%)5 (4%)

47 (73%)14 (22%)3 (5%)

50 (86%)6 (10%)2 (4%)

14 (82%)2 (12%)1 (6%)

21 (77%)0 (0%)6 (22%)

19



Trainees (64)

Trainers (58)


(17)

No Role(27)

8.10 CBT overall, provides a good preparation for the job of firefighter


48 (35%)49 (36%)38 (28%)

19 (30%)29 (45%)16 (25%)

21 (36%)22 (38%)15 (26%)

6 (35%)6 (35%)5 (30%)

7 (25%)7 (25%)13 (48%)

8.11 The TAFE course coordinator is always willing to help when requested


45 (33%)5 (4%)

85 (63%)

28 (44%)3 (5%)

33 (51%)

17 (29%)1 (2%)

40 (69%)

4 (23%)1 (6%)

12 (71%)

4 (14%)0 (0%)

23 (85%)

8.12 It is very difficult to communicate with the TAFE teachers, as they are often hard to reach


30 (22%)15 (11%)89 (67%)

15 (23%)10 (16%)38 (59%)

11 (19%)6 (10%)40 (69%)

2 (12%2 (12%)13 (76%)

6 922%)1 (4%)2 (8%)

8.13 The TFS assessment process takes too long before the trainee gets the results


58 (43%)24 (17%)53 (39%)

31 (48%)15 (23%)18 (29%)

23 (36%)13 (22%)22 (38%)

10 (59%)2 (12%)5 (29%)

9 (33%)3 (11%)15 (55%)

8.14 The TAFE assessment process takes too long before the trainee gets the results


41 (30%)8 (6%)

85 (63%)

22 (34%)6 (9%)

34 (53%)

19 (33%)3 (5%

36 (62%)

9 (53%)0 (0%)8 (47%)

5 (18%)2 (8%)

20 (74%)

20


9. If you indicated in question 5, that you are concerned about CBT, indicate your greatest concern by selecting one of the statements from question 8 above, or write your own response. My greatest concern is:

Item 8.1 (1); 8.2 (6); 8.3 (3); 8.4 (2); 8.5 (3); 8.6 (1); 8.8 (4); 8.9 (4); 8.12 (1); 8.13 (1); 8.14(1); Plus: Lack of team approach. Introduction of CBT was poorly managed. Being left to own resources while other personnel are not aware of the level a person is at. Senior firefighters should be recognised at their rank regardless of CBT. Lack of education and acceptance of CBT as a whole. Too many people at different stages of training. CBT should be prioritised and work with practical on station training. Ignorance of the system (2). Too many changes to the training system. Too much emphasis on theory and not enough on practical. Main job should be emergency response. Should not be forced to do public education. Too individualistic, too cumbersome, does not meet Agency’s needs. Modules and assessment tools still not available (2) CBT is good but modules written at too higher level. CBT workload too high. Introduced when not ready. Too rigid-not reasoned well enough. Too many caught in the system that is not functioning. Too much bull shit. Failure to support the training financially, physically and emotionally.

10 Would you support any of the following changes to CBT?

All respondents

(135)

Trainees (64)

Trainers (58)


(17)

No Role(17)

Allow trainers to also assess so trainees get immediateassessment feedback on their progress

Yes:No:Cannot Say:

104 (77%)19 (14%)11 (8%)

51 (80%)7 (11%)6 (9%)

46 (79%)9 (16%)2 (3%)

14 82%)0 (0%)3 (18%)

18 (66%)5 (18%)4 (14%)

Remove the over reliance on formal written assessments

Yes:No:Cannot Say:

80 (59%)31 (23%)24 (18%)

43 (67%)10 (16%)11 (17%)

32 (55%)17 (29%)9 (16%)

9 (53%)5 (29%)3 (18%)

17 (62%)4 (14%)6 (22%)

21


All respondents

(135)

Trainees (64)

Trainers (58)


(17)

No Role(17)

Introduce more progressive assessments and reduce the use of final assessments conducted at the end of a module

Yes:No:Cannot Say:

93 (69%)18 (13%)23 (17%)

47 (73%)6 (9%)

10 (16%)

44 (76%)5 (9%)8 (14%)

11 (65%)3 (18%)2 (12%)

14 (51%)5 (18%)8 (29%)

Restructure CBT so that the core competencies of firefighting are delivered earlier in the training program

Yes:No:Cannot Say

123 (91%)9 (7%)3 (2%)

61 (95%)0 (0%)3 (5%)

56 (97%)0 (0%)1 (2%)

16 (94%)0 (0%)1 (6%)

23 (85%)0 (0%)4 (14%)

Reduce the number of modules you have to do to graduate with the Level 3 Certificate (Pay Point 6)

Yes:No:Cannot Say

68 (50%)17 (13%)49 (36%)

37 (58%)6 (9%)

20 (31%)

23 (40%)11 (19%)24 (41%)

9 (53%)1 (6%)7 (41%)

13 (48%)2 (8%)

12 (44%)

Introduce a fairer and more consistent RCC process that is managed at the station/brigade level.

Yes:No:Cannot Say

94 (70%)18 (13%)23 (17%)

47 (73%)8 (13%)9 (14%)

38 (66%)10 (17%)10 (17%)

10 (59%)4 (24%)3 (18%)

19 (70%)3 (11%)5 (18%)

Give more responsibility for the management of training to the brigades.

Yes:No:Cannot Say

84 (62%)35 (26%)15 (11%)

47 (73%)8 (13%)9 (14%)

32 (55%)20 (34%)5 (9%)

12 (71%)4 (24%)1 (6%)

17 (62%)7 (25%)3 (11%)

22


All respondents

(135)

Trainees (64)

Trainers (58)


(17)

No Role(17)

Make more use of on station training

Yes:No:Cannot Say

99 (73%)23 (17%)10 (7%)

51 (80%)9 (14%)3 (5%)

42 (72%)14 (24%)1 (2%)

11 (65%)3 (18%)1 (6%)

20 (74%)3 (11%)4 (14%)

Make more use of regular block training at Cambridge

Yes:No:Cannot Say

106 (79%)21 (16%)8 (6%)

48 (75%)13 (20%)3 (5%)

49 (84%)7 (12%)2 (3%)

13 (76%)2 (12%)2 (12%)

21 (77%)4 (14%)2 (8%)

Provide more information about the CBT modules that is easily accessible to the stations

Yes:No:Cannot Say

116 (86%)2 (1%)16 (12%)

55 (86%)1 (2%)8 (13%)

49 (84%)1 (2%)7 (12%)

16 (94%)1 (6%)0 (0%)

24 (88%)0 (0%)3 (11%)

Use TAFE to deliver more CBT modules

Yes:No:Cannot Say

13 (10%)86 (64%)36 (27%)

5 (8%)43 (67%)16 (25%)

5 (9%)37 (64%)16 (28%)

4 (24%)9 (53%)4 (24%)

1 (4%)16 (59%)10 (37%)

11 Do you feel that there are sufficient assessors available in your area to meet the needs of the on station training?

All respondents

(135)

Trainees (64)

Trainers (58)


(17)

No Role(27)

YesNoDon’t Know

31 (23%)74 (55%)28 (21%)

14 (22%)36 (56%)13 (20%)

8 (14%)39 (67%)10 (17%)

4 (24%)10 (59%)2 (12%)

7 (25%)10 (37%)10 (37%)

23


12 Which of the following statements best describes the way you like to learn?

All respondents(135)

Trainees (64)

Trainers (58)

Managers/coordinator

(17)

No Role(27)

In a structured way with the support of the class and the assistance of a face-to-face teacher/trainer

42 (31%) 21 (33%) 16 (28%) 7 (41%) 9 (33%)

In a self-paced manner where I can progress at my own pace without direct face-to-face contact with a trainer and the need to follow a set timetable

8 (6%) 0 (0%) 2 (3%) 1 (6%) 3 (11%)

In a small group of work colleagues on station where close contact witha trainer is always available

55 (41%) 30 (47%) 23 (40%) 6 (35%) 11 (40%)

Not applicable to me

7 (5%) 0 (0%) 7 (12%) 2 (24%) 0 (0%)

Some other answer. Please indicate

Options one and two above (2) Options two and three above (1) Options one and three above (10) All of the above three options (5)

24


13 Indicate if you consider any of the following features are strengths or weaknesses of the competency based training in the TFS:


Trainees (64)

Trainers (58)


(17)

No Role(27)

The service TAFE provides

Strength:Weakness:Cannot Say

21 (16%)46 (34%)66 (49%)

11 (17%)24 (38%)28 (44%)

8 (14%)20 (34%)29 (50%)

4 (24%)6 (35%)7 (41%)

2 (8%)7 (25%)18 (66%)

The service provided by the Training Division


49 (36%)52 (39%)34 (25%)

25 (39%)25 (39%)14 (22%)

18 (31%)29 (50%)10 (17%)

4 (24%)8 (48%)5 (29%)

9 (33%)9 (33%)9 (33%)

Your ability to use and understand the TAFE CML system


12 (9%)45 (33%)77 (57%)

11 (17%)20 (31%)32 (24%)

2 ((3%)24 (48%)31 (53%)

0 (0%)5 (29%)12 (71%)

1 (4%)4 (14%)22 (81%)

The on station training


77 (57%)44 (33%)13 (10%)

36 (56%)23 (36%)4 (6%)

29 (50%)22 (38%)6 (10%)

10 (59%)5 (29%)2 (12%)

15 (55%)8 (29%)4 (14%)

The block recruit training at Cambridge


94 (70%)18 (13%)22 (16%)

47 (73%)8 (13%)8 (13%)

38 (66%)10 (17%)9 (16%)

11 (65%)4 (24%)2 (12%)

20 (74%)2 (8%)5 (18%)

The CBT assessment system


25 (19%)68 (50%)42 (31%)

11 (17%)35 (55%)18 (28%)

9 (16%)36 (62%)13 (22%)

3 (18%)11 (65%)3 (18%)

5 (18%)7 (25%)15 (55%)

25



Trainees (64)

Trainers (58)


(17)

No Role(27)

The rate of progress on the development of assessmentand learning resources


4 (3%)93 (69%)37 (27%)

2 (3%)48 (75%)13 (20%)

2 (3%)48 (83%)8 (14%)

0 (0%)12 (71%) 4 (26%)

1 (4%)10 (37%)16 (59%)

The amount of learning resource material available for training on station


8 (6%)92 (68%)35 (26%)

5 (8%)42 (62%)17 (25)

2 (3%)45 (78%)11 (19%)

1 (6%)11 (65%)5 (29%)

3 (11%)16 (59%)8 (29%)

The ability of stations to release people for training


5 (4%)118 (87%)12 (9%)

2 (3%)57 (89%)5 (8%)

2 (3%)53 (91%)4 (7%)

0 (0%)16 (94%)1 (6%)

2 (8%)21 (77%)4 (14%)

The overall content in the competency based training (what you have to learn)


39 (29%)50 (37%)43 (32%)

21 (33%)25 (39%)16 (25%)

19 (33%)23 (40%)16 (28%)

4 (24%)7 (41%)5 (29%)

4 (14%)8 (29%)15 (55%)

The order in which the modules are taught


9 (7%)70 (52%)56 (41%)

4 (6%)37 (58%)23 (36%)

5 (9%)32 (55%)21 (36%)

0 (0%)7 (41%)10 (59%)

0 (0%)11 (40%)16 (59%)

The ability of CBT to provide a portable qualification


68 (50%)25 (19%)40 (30%)

31 (48%)14 (22%)19 (30%)

33 (57%)12 (21%)13 (22%)

9 (53%)3 (18%)5 (29%)

11 (40%)4 (14%)11 (40%)

26



Trainees (64)

Trainers (58)


(17)

No Role(27)

The level of information about CBT made available by the Training Division


11 (8%)69 (51%)52 (39%)

5 (8%)31 (48%)25 (39%)

5 (9%)33 (57%)17 (29%)

1 (6%)9 (53%)6 (35%)

2 (8%)12 (44%)13 (40%)

The level of information about CBT made available by TAFE


8 (6%)62 (46%)64 (47%)

5 (8%)29 (45%)29 (45%)

1 (2%)31 (53%)25 (43%)

2 (12%)6 (35%8 (47%)

1 (4%)10 (37%)16 (59%)

14 Are you aware that the CML training can be substituted with a distance learning package?

All respondents

(135)

Trainees (64)

Trainers (58)


(17)

No Role(27)

Yes 37 (27%) 26 (41%) 12 (21%) 7 (41%) 2 (8%)No 68 (50%) 27 (42%) 33 (57%) 5 (29%) 16 (59%)Not applicable 27 (20%) 10 (16%) 11 (19%) 4 (24%) 9 (33%)

If yes have you used this option in preference to the CML?

All respondents

(37)

Trainees (26)

Trainers (12)


(7)

No Role(2)

Yes 13 11 8 4 0No 29 18 7 1 4Not applicable 42 12 20 10 9

(Discrepancies in responses were because many respondents ignored the instructions and answered all questions in the sequence. Hence data are not reliable here.)

27


If no, please indicate why by ticking the appropriate box.

All respondents(29)

Trainees (18)

Trainers (7)


(1)

No Role(4)

CML is easy to use and more efficient

9 9 1 0 1

I like the fast feedback the CML gives me

24 2 3 0 0

I like to use the computer to help me with my training

4 2 1 1 0

Other answers 15 6 7 5 0

(Discrepancies in responses were because many respondents ignored the instructions and answered all questions in the sequence. Hence data are not reliable indicators here.)

15. There is a perception that there is over-training in CBT. Do you agree with this statement?


Trainees (64)

Trainers (58)


(17)

No Role(27)

Yes 60 (44%) 31 (48%) 25 (43%) 9 (53%) 11 (40%)No 28 (21%) 15 (23%) 16 (28%) 3 (18%) 2 (8%)Don’t know 41 (30%) 14 (22%) 13 (22%) 4 (24%) 14 (51%)

If yes, indicate where you feel that the over-training occurs.

Overlapping content in some modules e.g. communications stream modules. Give other examples:

Three respondents reiterated the perceived overlapping content in the communications stream Confined spaces, atmospheric monitoring Non-fire service modules Wildfire modules

Some modules have too much material not relevant to the TFS. Give examples, if any:

Computer Skills (6) Health and Fitness (2) Administration Some pay point 7 modules too advanced

Some modules delivered early to new recruits have material that should really come late. Give examples, if any:

Alarms and Alarm Systems Work Team Communication Writing Skills for Work Computer Skills (2) Fire safety Education (3) Community Fire Safety (2) OH&S

28


Dangerous Substances Hazardous Substances (2) Present Information

Other examples of over-training you are aware of, if any:

Computer Skills Fire Safety Public Education

Other general comments made by respondents

The need to return to the basics is increasing. We need to use the skills and knowledge of senior firefighters.

The training that is delivered by the TFS is not too bad and has good content. Leadership and management courses-designed for full-time managers, not leading F/F’s. Few, if any, basic modules for fire fighters to be trained in. Too far removed from practical fire fighting. CBT modules too heavy. On station training that goes on without reference to CBT and then later re-visited to meet CBT

needs. Need to strike a balance between what is reasonable and relevant. Some material taught at recruit courses is not CBT standard and therefore must be taught later. Fire fighting is a very practical skill, one which is being lost in many areas due to emphasis on

academic ability.

29


Analysis of Findings with Recommendations

The data summarised below have originated from four principal sources:

Eight focus workshops attended by 87 career staff Interviews with 22 key stakeholders associated with the training Analysis of numerous documents and curriculum materials associated with the

training Questionnaire returned by 135 career staff from all of the brigades.

The raw data findings from the focus workshops, interviews are included in Attachments 2 and 3. The summary below attempts to integrate these findings in a succinct way.

Using the analytical framework identified within the methodology, it has been possible to summarise the findings under various headings, or issues as they relate to the internal training delivered by the Training Division and the external training delivered by TAFE.

National Influences on the TFS Training

The current AFAC competency based training system adopted by the TFS is modelled on the old National Framework for the Recognition of Training (NFROT) principles. This approach to training was established in the early 1990’s and subsequently enshrined in Commonwealth and State legislation (Tasmanian Vocational Education and Training Act 1994).

The NFROT principles controlled the way training was designed, delivered and the way quality assurance was managed. The approach was course and provider driven. That is, qualifications were based on meeting course outcomes. Thus, to receive a qualification, an individual needed to be a student undertaking a qualification, usually an accredited course delivered by a registered training provider, such as a TAFE Institute, or a private training provider.

This approach to training was begun to be phased out from 1/1/98, in favour of a demand driven model known as the National Training Framework (NTF) and the Australian Recognition Framework (ARF). Under this new approach, industry sectors will develop Training Packages, which will define the workplace standards, qualifications structure and assessment guidelines required.

Discussions with Sandra Lunardi (AFAC Curriculum Development Manager) indicated that a Public Safety Training Package with a Fire stream is currently in the early stages of development, and is expected to be endorsed and available by April 1999. The Australian National Training Authority (ANTA) is driving the development of Training Packages in most industry sectors and has developed guidelines for their development. Based on these guidelines, the Training Package that will be used in the national Fire Industry from 1999 onwards will consist of six components as illustrated in Figure 2 below.

30


From this figure it can be seen that there will be major changes to the way qualification and training will be managed in 1999. The AFAC qualifications (Certificates and Diplomas) will become AQF qualifications based on meeting the competency standards, not the courses/modules. The need to enrol in a course to receive a qualification will be eliminated. The three top components of the Training Package will be endorsed. Significantly, the AFAC curriculum that controls the TFS training will become part of the non-endorsed component of the learning strategies. That is, they will become optional components of the training.

Figure 2 Six components of a Training Package

In terms of the current training arrangement, the following changes will occur once the Fire Training Package is endorsed and subsequently adopted by the TFS:

The TFS will be able to register to deliver the Training Package as a Registered Training Organisation (RTO)

The current course-based qualifications will become standards-based. This means that a firefighter will only need to demonstrate competency against the relevant standard to be deemed to be competent and ultimately receive the qualification. Hence the emphasis on assessment guidelines/materials in the Training Package.

Training Package

Australian Qualifications Framework

(AQF)

IndustryCompetency

Standards

Assessment Guidelines

Three Endorsed Components

Professional Development

Learning Strategies

e.g. training modules

Assessment

Materialse.g.

assessment

Three Non-endorsed Components

31


The passing of training modules will become optional learning pathways, but not compulsory. Work experience and on-job learning will complement structured off-job training.

Work based learning will become more common. As an RTO, the TFS will be able to exercise its User Choice rights. This means

that it could seek to control the government funding that was previously given to TAFE to deliver some of the off-the-job training component of the AFAC curriculum. This funding could then be used to establish appropriate learning/training responses that directly meet the TFS’s needs. This could include outsourcing training to TAFE, or other training providers as appropriate. The state policy for User Choice is still being developed, but should be well established by mid-1999.

This explanation of Training Packages, although brief, has been given as it informs the nature of the recommendations that follow in this report. Essentially, the TFS needs to consider two training agendas: The short term agenda between now and mid-1999 when the Training Package is

endorsed. The longer term agenda which will follow the introduction of the Training

Package.

Given this scenario, it is essential that the TFS be in a position to adopt the Fire Training Package by mid-1999, or thereabouts. To do this, it will be necessary to ramp the Agency up to a position where it will be able to adopt the Training Package and cause little disruption to the training and development arrangements. This will be possible, provided strategic approaches are taken now in the way training is managed.

The CBT Course

Looking at the training as a whole entity, it was difficult not to conclude that there really has been no accredited course at all being delivered. Rather, there was a series of independent training modules being delivered by both TAFE and the TFS all of which aggregate at various pay points to achieve a pay outcome for individuals. The notion of a holistic course outcome leading to a nationally recognised qualification was not supported by the evidence.

No person has so far graduated with a qualification since CBT was introduced in 1995, although approximately 270 firefighters are eligible to receive one (personal communication Brian Hevey 5/3/98). No formal course enrolment occurs for the TFS component of the training. Enrolment in the course is deemed to be automatic once a module is attempted.

Also, the TFS chose to restructure the AFAC course to match the pay point needs. Up until early 1998, this caused much concern within the firefighting ranks, as the number of modules required by the TFS was much larger than that required to meet the AFAC course requirements.

With the introduction of a new pay point system in March 1998, this anomaly was partly corrected although it was noted that the new pay point system was still out of synchronisation with the course structure. The Training Division indicated that the

32


pay point system and the course requirements were separate issues and that they no longer relate. Training Division will, in future, concentrate only on delivering the course requirements.

Knowledge of the course requirements was low amongst trainees and others. For example, when asked about the level of information about CBT provided by the TAFE and Training Division in the questionnaire, only 8% of trainees considered the information sharing process to be a strength, whereas 48% considered it a weakness (see q 13). Course information trainees usually require upon starting a new course is usually provided at the enrolment stage. It is not uncommon that the enrolment form triggers a process whereby the trainee receives a kit of information about the course structure, availability of resources, RPL rights, etc. Because this trigger did not exist in the TFS component of the course, it appears that this information was then provided ad hoc or when requested by the trainee.

The training has been driven by the pay point system, rather than the need to meet the course requirements. This resulted in less emphasis being placed on course management and more being placed on individual module resourcing. Both Brian Hevey and Robyn Pearce acknowledged the reality of this situation, but correctly argued that the essential reason for providing training was to maintain, or build, the trainees’ levels of competence, so that they can do their job safely and efficiently. Yet 50% of respondents to the questionnaire felt that the portability of the qualification was a strength of the training (see q. 13).

Recommendation 1.

That the TFS review its obligations as a Registered Training Provider, to ensure that it meets the requirements in relation to the delivery of the following accredited courses:

1927 Certificate 11 in Firefighting Operations1928 Certificate 111 in Firefighting Operations1929 Certificate 1V in Firefighting Operations1931 Diploma of Firefighting (Management)1930 Advanced Diploma of Firefighting (Management)

and that every effort be made to issue qualifications to eligible recipients as soon as possible.

Delivery Methods and Module Sequence

There are three distinct teaching/learning modes used within the training: the self-paced, individualised learning using CML or distance learning packages

(also known as flexible delivery at distance) the on station just in time training mode the pre-planned, highly structure block training for recruits and others at

Cambridge

33


It is a very positive feature of the training that it does embrace several delivery modes. This provides variety in the learning process and thus increases the enjoyment of learning, while concurrently providing for different learning styles.

On Station Training

On station training was one of the preferred modes of delivery by respondents to the questionnaire and participants in the focus workshops. When asked to comment on whether the TFS, should make more use of on station training, 73% of all respondents said yes, including 80 % of trainees, 72% of trainers and 65% of managers/coordinators of on station training (see questionnaire q 13). Approximately half of all trainees indicated that they preferred to learn on station in a small group with close contact with a trainer (see questionnaire q 12).

However, there was a high level of frustration amongst nearly all of those on station who had anything to do with the delivery of CBT, because of the interruptions caused to well planned training sessions, by call outs and other distractions. This, in some cases, was having a major negative affect in the way the various shifts responded to their training needs.

For example, some Brigades had well structured training schedules for delivery, say on weekends, while others relied more on being able to respond to the personal needs of individuals or platoons.

There was frustration with the perceived under-resourcing of the Training Division, which was hampering its ability to effectively service the Brigade’s training needs. Little blame was directly apportioned to the Training Division staff- all were considered to be highly competent and dedicated to their jobs. The general feeling was that these officers were doing the best they can, but the job was just too big and they were falling further behind.

For example the questionnaire indicated that 75% of all respondents felt that CBT was under-resourced, including 76% of station officers responsible for managing the local training function (see questionnaire p5).

The training room facilities on all of the major stations (Burnie, Devonport, Launceston, Hobart) were quite adequate to service the needs of those requiring a quiet area for study. Most were well resourced with appropriate teaching aids and computer support was also available. Some participants in the focused workshops indicated that the facilities on the smaller out stations were not as good as that described above, and that this affected some trainees’ ability to access learning resources and complete TAFE modules.

The availability of teaching and learning resources on station was also raised by several participants in the focus workshops and later in the interviews. Significantly, only a small minority of all respondents (6%) saw the availability of resources as a strength of the training (see q 13), whereas 68% felt it was a weak link in the training. Trainers felt the strongest on this issue with 78% indicating that it was a weakness in the training. One respondent in the Launceston Brigade commented that trainers often

34


spent more time collecting resources and preparing lesson material than they spend training.

Information provided by Brian Hevey indicated that when a teaching resource was developed for a particular module by someone on station, then it was noted in the information sheets provided to trainers. This approach assumes that the developer remembers to pass the material onto the Training Division. The above evidence suggested that this approach needed to be more structured.

Recommendation 2

That an audit of all teaching and learning resources relevant to the competency based training be undertaken across the TFS and interstate. The audit should attempt to identify all existing resources, both those commercially produced and those developed informally by TFS trainers. The purpose of the audit should be to: Identify what resources are available on each station and in each Brigade, so

that any deficiencies/gaps can be addressed The nature (brief description) of each resource The module and competency standard to which each resource relates The currency of the resource The author of the resource The availability of the resource for borrowing/purchase

To facilitate this process, it is recommended that a single page template be developed that contains each of the above items, and that all trainers be encouraged to complete this form each time a new teaching/learning resource is developed. From the material collected in this project, it should be possible to establish an electronic data base on the TFS internal network. If all trainers are encouraged to update the data base as new material is developed, it will provide the first port of call of all trainers before they begin to develop new lesson material.

The second major concern with the delivery of on-station training related to the ability of stations to release people for training. Along with untimely call outs, this was the single biggest impediment to the efficient delivery of training, both on station and elsewhere.

Eighty seven percent of respondents to the questionnaire indicated that this was a weakness of the training system (see q 13), including 94% of officers responsible for managing and coordinating on station training. Similar responses were provided by trainees and trainers.

The creation of a training crew was seen by many respondents to be a positive response to this issue. When questioned about the nature of this crew, no consistent answers were given. Most Brigades indicated that they wanted control of the training crew, whereas the Training Division also indicated that it should control this crew. Some Brigades indicated that the crew should actually exist in name, and be allocated duties accordingly, while others favoured a phantom crew, which would really be an over complement of staff to provide backfill for the release of firefighters for training

35


on a Statewide basis. Until the size and nature of this resource has been determined, it was not possible to comment further.

TAFE CML Training

The TAFE CML delivery of training was viewed negatively by many respondents. When asked whether the TAFE service was a strength or weakness of CBT, 17% of trainees indicated it was a strength, while 38% a weakness (see questionnaire q 13). Similar responses were given by trainers. From the evidence provided by participants in the focus workshops, the principal concerns with the CML training were:

Not user friendly, Numerous errors with the question bank (e.g. duplicated questions occurring on

the same test; correct answers to some questions being marked wrong ), Being timed out of tests when called out to incidents, Little or no personal interaction with the TAFE tutor, High pass marks (80% to 90%) are often difficult to reach on a theory test.

Although many of the concerns expressed above are legitimate, some have been rectified, or are in the process of being rectified and can be put down to teething problems associated with a new system. TAFE indicated that no student has been disadvantaged by any of the above problems, and that the CML system can be over-ridden manually to rectify any technical problem.

The CML approach was established by TAFE to deliver specific AFAC modules not only within Tasmania, but nationally. There are currently 80 Tasmanian enrolments with another 50 students using the system from interstate. CML was introduced in early 1997 and is a delivery process which has been endorsed by AFAC and is being used nationally via the service provided by the Flexible Learning Centre at Clarence. The system can hold up to 200,000 test questions, but currently has around 10,000 questions.

The pass mark on the CML progressive tests is high (80% to 90%) and individuals can be timed out after 90 minutes if they fail to complete the test in that time. If an individual fails a test three times he/she is compelled to contact the Flexible Learning Centre for advice. The Flexible Learning Centre is reluctant to give individuals access to the final written examination until they have passed all of the progressive tests via the CML process.

The issue here is if the trainee doesn’t need to do all of the work in a module to pass the final examination, should he/she be asked to also pass all of the progressive tests? TAFE’s response to this was that the progressive tests served to provide feedback to the trainee (which would otherwise be provided by the trainer) on his/her progress and that this was an essential feature of any valid training system. This approach is sound, but it also maintains control of the learning process with the training provider.

CML has been established, and is managed on a limited budget by TAFE. To provide the equivalent training service by more traditional teaching approaches would be far more expensive, given the fact that trainees are spread around the state, and are all at different stages in their learning. In effect, CML has been established in such a way to

36


minimise the need for direct teacher support (additional resources), provided you follow the rules.

For example, tutors are only paid a nominal amount to mark assessments. They have no direct teaching role. Thus, John Williams is the only TAFE technical person with knowledge of the course content, who is in a position to respond to module content queries. The evidence from both the focus workshops and the questionnaire (see q 8.11) indicated that John is always willing to help when available. However, John has other duties, including the management of the resource development project which is very time-consuming. In addition, he is rarely available at nights and weekends when most of the training takes place. Despite the fact that TAFE mobile phone numbers are available for after hours advice, most trainees rarely use the facility. It could be that the TAFE staff are not familiar to many of the trainees (except a voice at the end of a phone), and are thus bypassed when a learning problem occurs.

From the indirect evidence available, the TAFE Flexible Learning Centre appeared to be under-resourced in terms of meeting the AFAC curriculum delivery requirements. It appeared that approximately 130 students were being serviced by less than one full-time teacher, plus administrative support and tutors for marking assessments. Based on a normal annual teaching load of 840 hours for a TAFE teacher, each trainee would only need to undertake 6.46 hours of training per year to meet this load. Although this comparison is crude and fails to acknowledge the support provided by the CML system and the tutors, there does appear to be a case for additional TAFE resources.

No one from any of the focus workshops indicated that he/she had home access to the TAFE CML program, although this facility was available upon request, provided the trainee has the necessary hardware.

Given the reliance on computer support for the CML, it was interesting to note that only Hobart and Launceston had dedicated PC’s for training while all other smaller stations only had shared PC’s. Since no one was particularly aggrieved by this, it was assumed that this was not one of the principal issues affecting the delivery of CBT.

When the CML system was established, the Centre for Flexible Delivery offered to install some computers on stations to assist in the establishment of the new system. Apparently, the Training Division at Cambridge was the only station to take up the offer.

The TAFE staff also indicated that the CML was one of two options available for many modules. Trainees were free to use the distance learning method if this was preferable. From the comments made by many of the trainees at the focus workshops, many were not aware of this option, despite the fact that the TAFE staff claimed that this was brought to their attention early in their course. When asked about this option in the questionnaire (see q 13) 41% of trainees said they were aware of this option, while 42% said that they were unaware of this alternative. Overall, only 27% of all respondents to the questionnaire indicated that they were aware of this option.

It is obvious from these data that better communication/information about the TAFE training options needed to be provided to all station personnel, not just trainees. John

37


Williams indicated that he had not been able to visit many of the stations in recent times due to other work commitments, but that would welcome the opportunity to do so.

Recommendation 3

That TAFE staff be invited by the TFS to visit the four Brigades around the state at least twice a year, and provide a face to the voice at the end of the phone. During such visits, TAFE staff should be invited to hold small group and one-on-one meetings with station personnel to discuss any issues affecting their training. In addition these visits should be used to disperse information about the TAFE training options, availability of new resources, and generally bring everyone up to date with the TAFE component of the training. This strategy will require cooperation and logistical support from the Brigades.

Recommendation 4

That the TFS write to the Flexible Learning Centre, TAFE Tasmania executive and Margaret Thurstans, executive officer of the Public Administration, Finance and Business Services Industry Training Advisory Board, to explore the possibility of TAFE being able to allocate additional resources to support John Williams in the delivery of the AFAC curriculum to the TFS.

Recommendation 5

That the TFS negotiate with the TAFE Flexible Learning Centre about its concerns relating to the CML system and request that appropriate action be taken. Specifically: The elimination of all technical errors associated with the assessment bank

e.g overlapping/duplicated questions, incorrect answers to questions, etc Liberalising, or eliminating the time out process Eliminating the need for trainees to undertake all progressive tests before

gaining access to the final assessment (this issue is also covered later under assessment, over-training and resource development).

This recommendation, if accepted, will require additional resources be allocated to the Flexible Learning Centre (see Recommendation 4 above)

Of all the issues that caused the most debate at the focus workshops, CML topped the list. However, this finding was not corroborated in the questionnaire as any more of a concern than any other issue (see question 9), despite the fact that 59% of all trainees felt that the CML system was flawed (see q 8.3). One suspects that the concerns about CML listed above are serious for some, especially those who dislike computers and self-paced learning, but have been overlooked by others who just want to get on with their training.

The self-paced learning style is not favoured by most trainees who preferred to have contact with a trainer and the support of their peers. The questionnaire data indicated that none of the 64 trainees responding to the questionnaire favoured self paced learning over face-to-face training , either on station or in a classroom. This type of

38


finding is not unusual, given that most people prefer close trainer support when learning. However, it does serve to emphasise the difficulty that TAFE has faced in the delivery of their component of the training.

When mixed delivery modes are used in training programs, and the trainees are generally used to face-to-face teaching, then it is often recommended that the individualised learning approach be held back until the trainee was confident enough to learn with little direct assistance. The recruits normally employed by the TFS are mature age people, often with trade backgrounds. They are practical people who often find difficulty in spending long periods of time in self study mode, especially when there were so many distractions on station during passive times. When presented with this type of training situation, it may be preferable to delay the trainees’ need to do the TAFE modules as long as possible until they become more familiar with the discipline required for unassisted study.

The CML dilemma for the TFS is that the flexible learning approach at a distance is the only training option available to it, if it wants to have the training delivered and funded by the office of Vocational Education and Training (OVET). This TAFE training is fully funded under the TAFE resource agreement for 1998.

The alternative is for the TFS to fund and deliver this component of the training as well using methods other than CML. This would require additional resources. Once the national Training Package is in place in 1999, the funding options will vary, subject to the User Choice State policy. This issue is further discussed later in this chapter (see Recommendation 18).

Recommendation 6

That the option of delaying the introduction of some modules delivered by TAFE for new recruits be investigated, to see if it is feasible. This is in response to the concerns about the readiness of some trainees to commence self-directed learning early in their training.

Block Training at Cambridge

There was almost universal support from all groups for the long version (12 weeks plus) of the recruit training, provided that it focused on the competencies required by a beginning firefighter. The questionnaire (see q 6) indicated that 91% of all respondents favoured either a 12 week block or a 16-20 week block for recruit training. The current 12 week block was the most favoured. Seventy percent of all respondents to the questionnaire (see q 13) indicated that the block training at Cambridge was one of the strengths of the current training and that the amount of block training conducted at Cambridge should increase (see questionnaire q 8.9).

Many participants at the focus workshops, especially those that had experienced block training at Cambridge, indicated that the preferred model was a live in block for all participants, not just those living outside Hobart. Although more expensive, it was felt that the benefits of living and training together as firefighters prepared them well for their new life to come. It encouraged team work and networking, especially during evenings when much of their study was undertaken.

39


Most respondents indicated that the recruit program should focus only on the must know competencies, leaving others for delivery in the second and subsequent years. Must know competencies, in this context, refer to those associated with the duties of a probationary firefighter. Many respondents felt that the recruit training included too much material that related to building fire safety, fire prevention and alarms, sprinklers and evacuation systems. It was felt that this would be better sequenced to occur later possibly on station or in a second block.

For example, several officers at the focus workshops indicted that they needed to ensure that a new recruit understood how to safely work in his/her crew and was confident in being able to effectively respond to incidents without endangering any lives.

Building fire safety, fire prevention and community education were acknowledged as also must know competencies by most senior officers and lower ranks, but duties related to this important work are not normally undertaken by a probationary firefighter to any large degree. The questionnaire (see q 8.1) indicated that 69% of all respondents agreed that a firefighter’s role was more than just emergency response- there was an important community education function also, but also indicated that many of the modules were out of sequence with the needs of the job (see q 8.4). When asked if the training should be restructured to allow the core competencies of firefighting to be delivered early in the training, between 91% and 97 % of all major groups agreed (see q 10).

On station career staff, especially officers, were looking for fire safety, fire prevention and the community education aspect of the training to be introduced later in the training, say towards the end of the first year, or early in the second year, by which time the training would have more relevance and meaning for the individual. This would mean a second training block, which may cause release problems for Brigades.

It should be noted that this issue was addressed via the publication of the new Pay Point System on 16/3/98. This restructuring of the module sequence resulted in several modules that were previously taught in the recruit training program now being allocated to higher pay points.

It was difficult to determine exactly why the block training approach was most favoured. It probably related to some or all of the following:

frustration with some of the current alternatives modes of training being used, extensive up front block training obviates the need for stations to organise and

deliver extensive training on station themselves, resources for the delivery of training are shifted to the Training Division, trainees are generally practical, hands-on people, many with trade backgrounds.

They are familiar and comfortable with a more socially interactive training model (groups learning together with a teacher (see questionnaire q 12).

40


Recommendation 7

That the initial block training for recruits being delivered at Cambridge be maintained at approximately 12 weeks duration. This block should focus on firefighting competencies, but also include an awareness program of approximately two to three days on Community Fire Safety. In addition, this initial block should also include face-to-face delivery of the module, Computer Skills (see also Recommendation 10 relating to this module). Approximately 9-12 months after their initial block training is completed and consolidated on shift, trainees should be required to commence the various modules relating to Community Fire Safety on station.

Assessment Processes

The national framework (VEETAC 1993, 13) under which the CBT system was introduced stated that assessment had four specific purposes:

assist and support learning by advising the learner about the quality of performance and the learner’s rate of progress towards the achievement of the competency standard (formative assessment).

help learners and their supervisors determine their education and training needs (diagnostic assessment).

determine whether a unit of competency or a learning outcome has been achieved for the purpose of formal recognition of training (summative assessment).

determine whether a person has achieved standards of competency which have not yet been formally assessed or recognised so that they may gain entry to or credit in recognised courses (recognition of prior learning (RPL /Recognition of Current Competencies (RCC)).

Using a variety of different forms of assessment, the intention of the national policy was to encourage a balanced approach to assessment so that each of the above purposes could be achieved, where appropriate. The recommended forms of assessment were observation of practical tasks with oral questioning, practical demonstration and questioning, pen and paper tests and essays, oral tests, projects, simulations, portfolios and computer based assessments.

The data collected from the focus workshops, questionnaire and interviews all indicated that assessment practices were a continuing cause of concern by most people associated with the training.

TAFE Assessment Practices

In the TAFE component of the training, the assessment approaches, focused mainly on summative techniques (pen and paper, invigilated exams), formative assessment (progressive tests) and RPL. The formative testing was also partially diagnostic, as its

41


purpose was primarily to provide feedback to the trainee, including any weaknesses in knowledge that he/she may possess at the time.

The level of invigilation was high. This was because of the way the original AFAC assessment approach was established in 1995. TAFE claimed that it needed security and fairness in the delivery of the assessment process, and that this was best achieved by the invigilation process.

In the TAFE component of the training, while a flexible delivery approach at distance is maintained, TAFE claimed that no other assessment approach was viable without providing additional resources. The Flexible Learning Centre attempted to design the training system so that the assessment process was both a quality assurance mechanism, to maintain the integrity of the training, as well as to provide a progressive feedback process.

As indicated earlier, the Flexible Learning Centre has only one technically qualified teacher (John Williams) who could provide the technical feedback the trainees require for all of the modules in the various courses. Other staff at the Centre were available to assist, but TAFE acknowledged that they have little technical knowledge of firefighting. With only one full-time staff member available, the training and assessment approach was designed to enable the trainee to progress at his/her own pace, receive feedback via the progressive tests, and then seek technical advice from John Williams, when appropriate. The part-time tutors were only employed to mark assessments.

Because the majority of the modules taught by TAFE were theoretical in nature, the assessments tended to be dominantly invigilated written examinations or objective computer tests based on the Fire Industry Assessment Template (FIAT) developed by AFAC. The high level of invigilation has become a costly exercise in time and effort by officers on stations. Many senior officers indicated that a more streamlined approach was favoured, provided the integrity of the assessment process was not undermined.

In relation to all assessment, both TAFE and TFS, the trainees indicated in the questionnaire (see q 10) that they strongly favoured more progressive assessments (73%), in favour of final assessments at the end of a module. This was also supported by trainers (76%) and station officers responsible for training (65%). Similarly, the majority of respondents (59%) favoured the reduction in the use of formal written assessments. Comments made by participants at the focus workshops and at interviews indicated that more use of oral assessments was favoured, as it was quicker and more efficient and required no invigilation.

The use of oral assessments was supported by Sandra Lunardi of AFAC. She indicated that the FIAT approach was never meant to be taken literally. It was meant to be a guide only, and that oral assessment was a recommended assessment approach under FIAT.

The use of oral assessments, in lieu of written invigilated examinations, would be difficult for TAFE, given its investment in the current flexible learning system. It would also require more direct contact with trainees which would require more

42


resources. Informal oral assessments do occur from time to time, but only for retesting after a trainee fails a test.

As indicated earlier, the CML system was designed to reduce the need for recurrent resources (teaching). Any variation from the current approach would reduce this cost benefit and would most likely be unacceptable to TAFE.

TFS Assessment Practices

As indicated above all respondents to the questionnaire favour a streamlining of the assessment process and a reduction in the use of written invigilated examinations.

In the TFS component of the training the separation of the assessment process from the teaching/learning process was no longer supported. All of the Brigade Chiefs, seven out of eight focus groups and 77% of respondents to the questionnaire (see q 10) felt that the trainer should also be able to formally assess the trainee. The process of having both a trainer and a separate assessor has been a constant cause of frustration by all parties associated with the training. It was considered unnecessary, inefficient and generally highly bureaucratic, especially when it applied to all modules in the TFS training.

All of the Brigades interviewed, the TFS central executive representatives and the UFU representatives indicated that they favoured a relaxation of the current assessment arrangements whereby trainers were not permitted to assess. Sandra Lunardi indicated that this approach, if supported by the TFS, would be quite acceptable to AFAC.

The Training Division staff favoured retention of this AFAC model, as they claimed that it reduces nepotism. Although this comment is probably accurate, assessors can also be nepotistic. The key issue is: would allowing trainers to assess undermine the integrity of the CBT training?

Having a trainer teach a trainee to be later followed by an independent assessor and validator is not a necessary feature of a CBT program. This is a feature adopted by the firefighting industry. Some industries utilise a third party assessor, but most rely on having a quality assurance system in place which places trust in the ability of the trainer to also assess. This is managed by giving the trainer both assessor and trainer training and providing quality assurance checks such as moderation.

Given the Training Division’s and stations’ resources that are utilised in managing the current assessment system, and the acrimony that it has caused, it is problematic whether the costs are justified by the perceived benefits.

Most CBT training conducted in Australia at present assumes that trainers can also assess. Most TAFE courses that converted to CBT in the early 1990’s eliminated external invigilated examinations from their curricula and allowed their teachers to also assess.

Given these external benchmarks, it is unlikely that the TFS CBT training would be undermined. However there are risks in this approach and they would need to be

43


managed if a change was to be instigated. For example, it is appropriate that a common assessment instrument be maintained to enhance reliability of the assessment outcomes. Also, the role of the validator could be amended to include an auditing function at the Brigade level. Also, a more open grievance/appeal process administered on station would need to be developed with an assessment policy framework.

Given these quality assurance checks, enabling assessors to train and trainers to assess would greatly increase the pool of expertise available to Brigade Chiefs to manage the CBT training on station. Fifty five percent of all respondents to the questionnaire (see q11) felt that there were still insufficient assessors available on stations, three years after the introduction of CBT.

Additional Workplace Trainer Category 1 training and Assessor training may be required to enable assessors and trainers to fulfill both functions.

To create greater efficiencies, all Brigade Chiefs and their senior officers supported the retention of all assessment instruments in a secure location within the Brigade. It was felt that having to continually request assessments be sent from Cambridge was inefficient and delayed the administering of the assessment process.

The UFU representatives indicated that the other area where the risk needed to be managed was in those modules where the health and safety of the firefighter was of prime importance. It may be appropriate to nominate certain modules where an external assessment process was warranted.

The questionnaire focused on merging the roles of assessor and trainer (see q 7). While 29% of all respondents felt that the credibility of the training would be undermined, 73% agreed that the assessment process would be greatly streamlined and 53% thought that CBT would be more acceptable. A similar pattern of responses was given by trainees, trainers and officers coordinating training on station.

Complementing this trainer-as-assessor concept was the reduction of invigilated, sit down, paper and pencil assessments. The need to seek an assessment instrument from Cambridge, administer the assessment through invigilation and then have it passed onto the validator was seen as too bureaucratic. All of the Brigades agreed that the process needed to be streamlined.

The approach favored involved the Brigades taking more responsibility for the assessment practice, with the Brigade Chief, (or his nominee) overseeing the process. Theory assessments could be administered as a written assessment, or be undertaken orally if appropriate. The need for an invigilator would be greatly reduced, as the trainer would assess during the training sessions on a needs basis. Once the trainee was deemed to be competent, the results would be signed off by the Brigade Chief, after consultation with the trainer and trainee. If the trainee is aggrieved by the results, then he/she would have a right to appeal to a panel appointed by the Brigade Chief. Auditing of the process would be systematic and be managed by the validators. Once the results are finalised, the Brigade Chief would advise the Training Division of the results.

44


Recommendation 8

That the roles of assessor and trainer be merged to allow trainers to also provide final assessments, and if appropriate, allow assessors to also train. To support this action, it is likely that additional training programs (Workplace Trainer Category 1 and Workplace Assessor ) be conducted in the three regions as soon as possible.

To maintain quality assurance, the following actions should be initiated:

All assessors use a common assessment instrument developed and maintained by the Training Division

Grievance/appeal procedures within the Brigade structure be available The role of validator be amended to include a systematic auditing function

within the Brigade structure That all assessment instruments be held in a secure location within each

Brigade under the authority of the Brigade Chief.

Recommendation 9

That the TFS develop appropriate policy on assessment, similar to its RPL policy, to provide advice and guidance to Brigade Chiefs in the administration of the assessment process within their respective Brigades. The policy should address the general principles of competency based assessment based on the evidence2 model, and among other things, identify the role and function of the key stakeholders and elaborate on quality assurance procedures and appeal procedures available within Brigades.

Although not supporting the trainer-as-assessor concept, the Training Division staff acknowledged that there was too much emphasis on hard assessment, rather than recognition. That is, the Training Division supported the notion that trainer/assessors should be encouraged to identify the competency gap in the trainee before attempting any training. In this way, unnecessary training would be eliminated and the criticism (focus workshop feedback) that there was too much over-training would be reduced.

This issue is addressed later in the RPL/RCC section.

Suitability of Module Content

There was a strong view held by all of the focus workshop groups that there was some over-training in both the TAFE and TFS components of the course. This criticism of over-training took several forms:

1. There was material in the front end of the course which only applied to senior firefighters and above

2. The front end of the course contained too much non-essential material which was not related to the core business of fighting fires

2 The evidence model is based on the assumption that assessment is a process of collecting evidence about a person’s level of competency and then judging the adequacy of that evidence.

45


3. Some material was just not relevant to a beginning firefighter (e.g. public speaking)

4. A number of modules were perceived to have overlapping content.

5. Some AFAC modules could be too generic and provide too much broad based material that did not exactly suit the TFS

6. TAFE teaching a module using an off-the-shelf package from another course ( e.g. Computer Skills)

Items 1-3 and in part, item 4, were addressed by the introduction of a new pay point system dated March 16th 1998.

When asked about the perception of over-training in the questionnaire (see q15) 44% of all respondents agreed there was over-training, while 21% disagreed. However when asked to be specific and identify the nature of any over-training, the vast majority of respondents failed to answer the question. The questionnaire listed all responses given, and indicated that Computer Skills was perceived to be the module causing the greatest concern. These data were also reflected in the responses given by participants in the focus workshops and some interviewees.

Apart from Computer Skills, it is problematic whether it is worth attempting to investigate any further over-training, in relation to overlapping module content. With the impending new Training Package on the horizon, and the work being undertaken by AFAC, it is doubtful whether much would be gained in the short term. Once the AFAC modules become part of the non-endorsed component of the Training Package, it will be possible for trainees to only study those components of a given learning package that are directly relevant to their needs. Non-relevant material will just be ignored.

AFAC, the TFS and TAFE all acknowledged that there was some overlapping material in some modules, especially in the communications stream. TAFE indicated that it was given the contract of developing resources to meet the AFAC module descriptors, not what it felt was the appropriate balance of training. AFAC’s curriculum review panel was addressing overlapping content within some modules at the time this report was written. The TAFE resource development project was designed to service the needs of all Fire Services, each of which has slightly different needs.

If there was unsuitable, or excess material in any of the modules, then TAFE indicated that the principal cause laid with the AFAC module descriptors which were adopted in full by the TFS. No attempt was made by TAFE to amend any of the module’s content, as it was perceived that this would affect the accreditation of the various courses and their national portability.

This issue of national portability was important, but also needed to be put in perspective. If the curriculum has faults, then surely it was better to fix these faults

46


and then address the portability issue, which only affects very few people, rather than ignore the faults which affects the vast majority of trainee firefighters.

In response, Sandra Lunardi of AFAC indicated that she was under the impression that resource developers such as OTEN and the Flexible Delivery Centre had a brief to identify overlapping content in any modules, so that corrections to the module descriptor could be made prior to the resource being developed.

In 1999, when the new Fire Training Package is available, the qualifications will transfer from the modules to the competency standards. When this occurs, the AFAC curriculum will lose its accreditation as a course and become an optional learning pathways for trainees as part of the non-endorsed components of the Training Package (personal communication Sandra Lunardi, 16/4/98). This will provide the opportunity for the TFS to amend any training module without losing the portability of the qualification, as this would be linked to the competency standards.

The nature of the Computer Skills module needs to be highlighted, as it was mentioned on numerous occasions by both trainees and other TFS staff at the focus workshops and was thus seen as symbolic of the type of positive action that the TFS workforce was looking for from this evaluation.

The learning package for Computer Skills was extracted from another suite of national courses servicing the accounting and office skills area. This package was a significant cause of concern by all of those trainees who attended the focus workshops and who have been required to undertake this training.

Statistics provided by TAFE indicated that of the 26 total all time enrolments in this module, only three trainees have so far successfully completed, with another three being granted RPL. Comparative figures for other modules could not be provided by TAFE.

The TAFE staff’s response to these concerns was that they did not acknowledge that there was a problem with the learning package, and that the TFS employees needed to be able to use computers in the way presented in the learning package, to not only support their work but also their learning via the CML process.

In relation to this module, the TFS has several options, if it supports the views of its own people:

It can request that TAFE redevelop the learning package so that it better meets the TFS’s needs,

It can request that a learning guide be developed which provides a map through the learning package, based on the TFS’s needs,

It can decide to teach the module itself in-house and develop its own learning guide/learning resource,

It can use another distance learning provider, such as OTEN which apparently has already developed a learning guide based on the learning package (personal communication with Sandra Lunardi),

It can deliver the training in a face-to-face mode as part of the recruit training block.

47


Given the nature of the discipline, many of the problems the trainees are having with the module probably related to the issue of not having close trainer support. TAFE also acknowledged that learning computing by flexible delivery at distance would probably extend the duration of the training substantially. Teaching this particular module in a face-to-face mode therefore seemed a more practical and efficient option.

Recommendation 10

That the TFS advise TAFE that in future, it intends to have the module Computer Skills delivered in a face-to-face mode, as part of the initial block training for recruits. That the TFS consult with TAFE Tasmania, or another appropriate computer training provider, or its own internal IT training personnel, to arrange for the delivery of the training.

For existing trainees, it is recommended that the TFS write to the Flexible Learning Centre and request that the Computer Skills module be delivered as a face-to-face short program (up to two days) within each Brigade’s region to ensure that the 20 current trainees successfully meet the learning outcomes of the module descriptor as soon as possible.

For other career staff who may wish to enrol in this module in future, it is recommended that the TFS negotiate with TAFE to deliver the module face-to-face within the local region, subject to viable class sizes being established.

Recognition of Prior Learning (RPL) and Recognition of Current Competencies (RCC)

(Note RPL and RCC are two terms used interchangeable here.)

The RPL/RCC comments made by many of the respondents at the focus workshops were inconsistent, both for the TAFE’s RPL process and the TFS’s RCC process.

From the TAFE viewpoint, many individuals claim they were not offered RPL while an equal number claimed they were. Others indicated that the process was not encouraged due to its bureaucracy. The evidence provided by TAFE indicated that all trainees were given adequate information about their RPL rights, both in the module material and in general information provided upon initial enrolment, such as the course diary, where RPL was further explained. A simple explanation for these inconsistencies was that some individuals may have enrolled in a course some time ago, say in 1995, and probably forgot about the level of information provided about RPL. Alternatively, it was possible that the benefits of RPL were not appreciated by some recruits until after they had completed some modules.

RPL is an integral component of all TAFE training and all teachers have an obligation under the State Training Act to provide suitable RPL information and support. There was no evidence to suggest that this did not occur in the case of the fire courses. For example, in the Certificate Level 2 course, TAFE has so far provided eight students with RPL for between one and four modules each.

48


The RCC process currently being administered by the Training Division received a mixed response from the focus group respondents. A minority felt that it was fair and reasonable and welcomed Brian Hevey’s approach. However, the majority of respondents at the focus workshops were generally negative to the process and felt that it lacked validity and consistency. The responses to the questionnaires indicated evenly mixed views. While 33% of all respondents to the questionnaire (see q 8.7) felt the RCC process was unfair, 30% felt it was fair and 38% could not comment.

Some focus workshop participants felt that it would be much fairer and objective if the process could be managed at the local level, provided a set of protocols were established. They commented that the local Fire Chiefs and Station Officers are the only ones who really knew the competency levels of their firefighters. Many respondents at the focus workshops indicated that they were aggrieved, but felt that they had no recourse to appeal the result, despite the fact that the Training Division indicated that this was not the case. The results from the questionnaire (see q 8.7) indicated that 88% of all respondents felt that the RCC process should be capable of appeal for those genuinely aggrieved. The Training Division officers who administered the process claimed that all participants were verbally advised of the appeal rights. Although not doubting this fact, it appears that this verbal reassurance was insufficient for many.

Instigating the RCC process, in principle, should be applauded. The work involved in the process used was considerable, and the staff in the Training Division should be congratulated for initiating the process. Technically, it should have been undertaken in mid-1995 at the introduction of CBT, but the administrative system in place at the time probably couldn’t have managed the process and the data, given the fact that the CBT system was just commencing and unfamiliar to most.

Recognition is a trainee’s right and with this right it is an obligation of the TFS to meet the policy guidelines established for RPL. The TFS RPL policy was based on the TASTA policy and was generally consistent with this state policy, except in the grievance procedure area, where the protocols did not allow the appeal to extend beyond the TFS to TASTA. After examining the TFS RPL policy and comparing it to the RCC process, it was apparent that the RCC was done in haste, and probably did not strictly follow all of the RPL processes as outlined in the policy statement. For example, the RPL policy required a separate assessment panel made up of an RPL facilitator, a benchmark expert and a support person to be available to assess and review, if required, the application.

Breach of this policy leaves the TFS open to challenge by any individual to the Training Authority, which would not only be embarrassing, but could undermine all of the good work so far completed as part of the RCC process. It is better to treat the RCC process so far completed as the first two stages in the broader RPL process and to now take the RPL process to its next logical phase-give all genuinely aggrieved individuals the chance to appeal.

For an appeal to be appropriate trainees need to be given guidance on what constitutes grounds for appeal and an appropriate appeal panel needs to be established. An appropriate panel could consist of the Brigade Chief’s nominee, an RPL adviser and a union representative, or support person.

49


Given the fact that all of the Brigade Chiefs strongly indicated a preference for a more devolved model of training, it would be appropriate that this appeal process be administered in-house within the Brigades, following the establishment of appropriate guidelines. Since most individuals now have their results, the approach would be to issue them with a communiqué outlining their rights of appeal. Brigades could then deal with the appeals internally and advise the Training Division of their decision. At worst, individuals will receive no more than they have already been given; at best they may receive credit for more modules based on the quality of their appeal and the evidence that they can provide.

Recommendation 11

That an appropriate appeal process, consistent with the TFS’s RPL policy, be established to conclude the RCC initiative. That this appeal process be managed by the Brigades as an in-house activity and that appropriate guidelines be prepared for Brigades to follow.

In relation to RPL and the transparency of the policy, the Training Division indicated only two applications for RPL have ever been received by the Training Division. This suggested that most career staff were unaware of their rights under this policy and that maybe some people have undertaking training on station in the mistaken belief that this was the only way that they could be credited with a pass in the module. Of the ten RPL advisors, the Training Division indicated that most have never dealt with a formal RPL request.

RPL was established under the NFROT principles for several reasons. One of the more pragmatic reasons was that it was far smarter, and cost effective, to not train someone than to train that same person in something that is not required, because the competency is already held. Given that so few internal RPL applications were received since 1995, it appeared that career staff were unfamiliar with their rights under the policy. There was strong evidence to suggest that the RPL policy needed to be translated into plain English and made available to all career staff.

Recommendation 12

That the RPL policy be summarised into a simple easy to read brochure and be distributed to all personnel on all stations as part of Recommendation 11. In addition, it is recommended that copies of the full policy be distributed to all Brigades as part of Recommendation 11.

50


Relationship with the Training Providers (TAFE and Training Division)

TAFE

From the comments provided by those attending the focus workshops, the relationship between TAFE and the trainees was very mixed. Some had positive views of the training and accordingly had a good relationship with the TAFE administration. Others however, had a very negative view of the TAFE training. This, in part, could be attributed to the theoretical nature of the training that TAFE was asked to deliver, most of which was not favoured by trainees. Also, the times when trainees were most likely to need a technical support person (nights and weekends) were the times when John Williams was least likely to be available.

Several responses to questions on the questionnaire illustrated the relationship between TAFE and the on station personnel. When asked if there was too much training delivered by TAFE that was not directly relevant to the job, 58% of trainees indicated yes, while only 15% disagreed (see q 8.8). When asked if TAFE should be invited to deliver more CBT modules (see q 10), 64% of all respondents said no, with similar responses from trainees and trainers. When asked if the service TAFE provided was a strength or weakness (see q 13) of the training, 16% of all respondents indicated that it was a strength, while 34% indicated it was a weakness. Approximately half the respondents were undecided. Conversely, when asked if the course coordinator was always willing to help when requested, 44% of trainees agreed, with most undecided.

These data strongly suggested that TAFE as an organisation (not John Williams) had a poor image amongst the trainees in particular, and amongst the majority of station personnel., in general. The reasons for this were not exactly clear and appeared to vary from individual to individual. If Recommendation Three is accepted by the TFS and TAFE takes up the offer to become more visible on station, the situation would most likely improve.

Training Division

Within the TFS, the on station training and the way it was locally managed was generally considered to be very good, despite the constant frustration of call outs to incidents during training.

The relationship between the Training Division staff and the on station personnel was mixed. Many felt that the Training Division didn’t service their needs very well, by not providing regular on station training, while others took the view that such training was the Brigade’s responsibility, and that the Training Division’s role was to provide the teaching/learning and assessment resources. Concerns were also expressed in relation to the time it took some assessment results to appear on an individual’s personal file (see questionnaire q 8.13). There were conflicting views of what role and function the Training Division was expected to play in regard to the training.

The biggest concern expressed by most respondents related to the lack of transparency in the delivery and management of the TFS component of the training. Many

51


respondents had little understanding of the course structure, as the training was driven by the pay point system. Similarly, many appeared to be unaware of their RPL rights, and grievance procedures rights.

Most of the respondents had the deepest respect for the competence of the staff of the Training Division. Most respondents felt that the Training Division’s staff were hard working, but that the job of managing the training across the whole State was beyond their resource capabilities.

When asked to comment on the service provided by the Training Division in the questionnaire (see q 13), 36% of all respondents indicated it was a strength of the training, while 34% indicated it was a weakness. Similar responses were given by trainees. However, 48% of officers on stations responsible for coordinating training felt the Training Division’s service to be weak, while 24% felt it was a strength of the training. Fifty percent of trainers also indicated the Training Division’s service was weak.

One suspects that these comments probably have something to do with the frustration felt by the slow rate of progress of resource development over the past three years and the normal parochialism that exists between the North and the South in Tasmania.

Resource Development

Training Division

Resourcing of the training has been a recurring issue since the introduction of CBT in 1995. When the TFS adopted CBT, comments made by senior officers interviewed indicated that it probably didn’t fully understand the resource implications of its decision. This view was also supported by the participants in the focus workshops.

There has been a high turnover of staff through the Training Division since the introduction of CBT, which has not assisted the development of resources. Much progress was made in 1997/98 however, in comparison to the previous two years. The Training Division claimed that there was a massive increase in the workload since the introduction of CBT, but the staffing level were not increased proportionally.

When comparing the output of assessment packages from the Training Division, it was obvious that Tasmania has made more progress than any other State/Territory (AFAC 1998).

The table below summarises the work still to be completed on assessment and learning packages by the Training Division. This does not include TAFE modules.

52


AFCAQF

Level 1Level 2

Level 2Level 3

Level 3Level 4

Level 4Level 5

Level 5Level 6

Total

AssessmentPackages 4 11 1 6 8 30

Learning Packages 2 19 3 5 8 37

Approximately 32 assessment packages and one learning package were developed by the Training Division since 1995. All of these assessment packages were developed in 1997 and 1998.

TFS management indicated that in the current funding environment, the TFS did not have the flexibility to put additional resources into the Training Division. CBT must be delivered within the current resource base.

Given the commendable rate of progress on the development of assessment packages since early 1997, Brian Hevey has been frustrated by his inability to retain staff for any duration in order to get the job done. A small CBT project team was established, originally to be made up of three workplace assessors, assisted by a part-time administrative assistant. This team never reached full strength and on several occasions its members were required to assist in validation processes, rather than concentrate on assessment packages.

Because of the nature of this type of resource development, and the way the TFS managed its personnel, it was probable that the 30 assessment packages and the 33 learning packages still to be developed would take many years to develop. Many organisations with this type of problem choose to outsource the work to appropriately skilled people, so that the work can be done concurrently by many individuals. Appropriate employment arrangements would need to be instigated to avoid any conflict of interests and double dipping, especially by any TFS personnel.

It is likely that the cost of paying for the work to be done on contract would not exceed the salary costs of the current CBT development team. To facilitate the outsourcing of this activity, the Training Division would need to carefully select all appropriate people and provide them with clear guidelines and some basic training in the process.

Recommendation 13

That if not already done so, the Training Division investigate interstate training providers that are currently servicing the AFAC curriculum to see if any of the 37 learning packages listed above have already been developed.

That a brief be developed by the Training Division describing the nature of the resource development work to be undertaken and that it be circulated throughout the Agency, and advertised externally if appropriate. That expressions of interest to develop the resources be invited and appropriately

53


skilled individuals be offered the development work on contract. This process should initially be trialed on the development of assessment packages, but could be extended to learning resources if deemed to be appropriate.

TAFE and AFAC

Funding provided to the Flexible Learning Centre to support the development of learning resources originated from The Australian National Training Authority (ANTA) through an Interstate Cooperative Project. Two grants totaling $220,000 were received in 1996 and 1997 to develop and deliver nationally, those AFAC modules designated for TAFE. AFAC acted as a clearing house of information regarding resource development to avoid duplication. It also coordinated the development of other learning packages by other organisations.

The table below summarises the progress to date on resource development for the TAFE modules by the Flexible Learning Centre.

Qualifications (AQF)

Learning resources

Assessment packages

CML

Level 2 9/9 9/9 7/9 Level 3 13/13 11/13 5/13Level 4* 11/15* 6/15 1/15Level 5* 2/10* 2/10 0/10Level 6 2/2 2/2 0/2

* At Level 4 three learning packages are completed in electronic form, but are not yet printed* At Level 5 three learning packages are completed in electronic form, but are not yet printed

The Fire Services around Australia are driving the development of all resources at the present time. Resources are developed according to an agreed formula which involves careful review by content experts.

With flexible learning courses, especially those that have geographically diverse students, it is imperative that adequate learning resources are readily available to support the isolated learner.

The quality of the dedicated fire learning resources developed by AFAC, the Centre for Flexible Delivery and others is of the highest quality. The graphics, art work and general text lay out is very good. These resources could be used in a variety of ways. For example, they could be studied in depth to meet a particular learning need, or they could be treated as mini-textbooks, to be dipped into from time to time for specific learning needs.

All of the learning resources (those developed by AFAC, TAFE and others) were criticised by respondents for containing material that was irrelevant to the job, or out of sequence with the needs of the job (for example, see questionnaire q 8.4 and 8.8).

Indirectly, this was a criticism of the scope and depth of the AFAC module descriptors developed in 1995 to service all State/Territory Fire Services. It is not unusual for learning resources developed to service a national system to provide an

54


excess of material. All Fire Services have their similarities and differences, so in order to accommodate all, the resources often need to provide say, 130% of which only 100% is directly relevant to any given Fire Service. It thus becomes appropriate from time to time to filter the resource and extract only that material relevant to the needs of the Fire Service. This filtering process is conducted informally when a trainer uses the learning resource in a training session and refers the trainee to only specific elements of the package, or when a learning guide is developed to complement the learning package, say in a distance learning or CML program.

Compounding this issue was the CML system which was designed to meet the national need. Consequently, participants in the service being provided by the Flexible Learning Centre were being required to do all of the work in each learning package, because all State/Territory Fire systems needed to be serviced. When that work was done on station, using say, an AFAC learning package, it is naturally filtered by the trainer and trainee so that only that material relevant to the needs of the TFS was used. The progressive testing system built into each learning package reinforced this point.

This issue is not a criticism of the nature of the Flexible Learning Centre’s learning packages. As indicated earlier, they have been developed to a very high standard to support a national curriculum. The concern lies in the requirement to learn all of the material, rather than only those portions deemed to be relevant.

In past years, most organisations would have thought it normal to educate the individual by providing a broad, well rounded curriculum which covered more than the minimum essentials. Today, with a diminishing budgets, this luxury was no longer available. The new paradigm is the provision of just in time training, not just in case training. Any broadening of the competency levels needed can be achieved on the job by job transfer, job rotation and promotion. This is the essence of the new National Training Framework.

The issue was covered in the earlier sections on Module Content and the TAFE CML Training. If Recommendation 5 dot point three is accepted, then this issue will probably resolve itself, as has been the case with the on station use of the learning packages.

Strategic Planning and Training Policy Development

Few training policies to support the implementation of CBT were found within the Training Division. Given the investment that the TFS has in CBT, this finding was surprising. Apart from a comprehensive policy on Recognition of Prior Learning (already discussed), there appeared to be no policies on assessment practice, delivery of training, quality assurance and resource development.

For training to be effective, it needs to be linked to an organisation’s corporate plan. This puts the training in perspective and provides it with a focus. More importantly, it encourages accountability as outcomes and outputs can be measured against goals. In addition, it encourages a culture of accepting training as being a service, funded to assist the organisation to meet its mission statement. When considered in this light, training thus becomes demand driven. That is, the training providers, both internal and

55


external, provide a training service to meet the needs of the users. Resources can then be justified and allocated to meet those needs.

All of the TFS senior management interviewed (Brigade Chiefs and senior executive members ) indicated that they would prefer to see this strategic link established between the CBT training and the function of the Brigades. They favored devolution of many of the current training processes to the Brigades, and an opening up of the training so that it was more transparent.

All Brigade Chiefs supported the idea, in principle, of allowing them to manage their own training needs, using both TAFE and the Training Division as service providers. The translation of this concept to effective policy and training practices is in part, an expression of the changes highlighted in the earlier Assessment and RPL/RCC sections of this report (see Recommendation 9)

Under the new National Training Framework, training is being converted from a supply driven model to a demand driven model. In essence, this means that training providers are now repositioning themselves to provide a training service to their customers, based on the needs of the customer. Within the TFS, the users are the Brigades, whereas the principal suppliers are the Training Division (internal) and TAFE (external). This notion of supply driven training is extremely important for the TFS, as it will become the basis for the way the new National AFAC Training Package is presented in 1999. Therefore, it is important for the training infrastructure within the TFS to gradually realign over the next twelve months so that when these new national training arrangements are introduced, they will be accepted as a logical extension of the existing arrangement.

All of the Brigade Chiefs supported the concept of the Training Division and TAFE acting as training providers. The issue was how were the Brigades to identify their training needs? This was explored with the Brigade Chiefs and all agreed that the RCC process should become the foundation for the development of Brigade Training Plans, based on individual needs. The Training Plans would then be linked to the TFS Corporate Plan, so that the role training plays in assisting the TFS in meeting its corporate goals became more apparent.

The Brigade Training Plans would grow out of the RCC process once the appeal stage was concluded. This RCC process was seen as a defacto training needs analysis. All Brigade Chiefs would then have a clear indication, of the competency levels of each of their staff. This could then be linked to the new pay point system and course structure so that every individual could see what training was required to meet certain immediate goals. This approach would also help to reduce the number of firefighters who have not participated in CBT. Twenty percent of all respondents (see q 3) indicated that they felt that they had no role to play in CBT. When asked why, the comments were varied:

No indication there is any need Waiting for the company to get it right Too old Not really interested Not required (2)

56


Ignorance of CBT Over 27 years of service. No chance of promotion

These responses and others indicated that there appeared to be a reluctance on the part of a core group of middle-ranking firefighters to participate in CBT. By having their competencies benchmarked against the AFAC competency standards/modules, then their skill gaps could be quickly identified and their training needs outlined in a personal training plan.

This would encourage all career staff to participate in the training. Currently only 57 career staff have completed a TFS module since 1995, representing about 24% (data provided by the Training Division). In addition, only 29% of all career staff are enrolled at TAFE mostly in the three lower Certificates. These figures further indicated that many of the middle ranking career staff either opted out of CBT, or never commenced it.

Within the region, the Brigade Training Plan would aggregate all of the data and priorities so that an achievable training strategy could be developed within the budget and expertise available. The Brigade Training Plan would not just list a training schedule and invite people to attend. This type of training tends to be opportunistic (like to participate), rather than strategic (need to participate). Although better than no training schedule at all, it still falls short unless it is linked to the Brigade’s training needs analysis data. The Brigade Training Plan could consist of some, or all of the following:

Training needs by individual, in order of priority Aggregation of individual training needs to identify viable class groups Strategies to deliver the training within the budget available. These strategies

could identify local on station trainers, as well as trainers in other Brigades with appropriate expertise. Appropriate TAFE modules would be identified and arrangements for training could be made. Where size permits, local study support groups could be encouraged so that trainees are not always required to learn in isolation. Ways to utilise the new Training Crew could be identified.

The process for developing the Brigade Training Plan would be consultative and involve as many members of the Brigade as possible. For a Training Plan to be accepted and implemented, ownership is essential. All career staff need to be part of its development. To facilitate this process, it is advisable that a single person in each Brigade take responsibility for coordinating the overall training function (Training Officer). In this way a consistent approach could be adopted across the platoons and this person could also become the key link person with the Training Division. In addition to other duties, the Training Officer would take responsibility for coordinating quality assurance, developing and implementing the Brigade Training Plan and monitoring the implementation of the personal training plans of station personnel.

The Training Division would act as a service provider under this model, providing advice and guidance to the Brigade Chiefs. Its role could include some or all of the following:

57


Providing information on where to access the best internal trainers and external training providers, resources, etc

Liaison with TAFE for access to the TAFE modules Providing an auditing function to maintain quality assurance between Brigades Assistance in the provision of resources Delivery of on station training and Cambridge based training (recruit and

promotion courses) in response to the Brigade’s needs Assist in the integration of the Brigades’ Training Plans to avoid duplication and

provide economy of scale for the training Maintenance of records for the issue of qualifications and for pay point

recognition Management of resource development

To assist in the evolution of the Training Division as a service provider, it was suggested that the Training Division establish and maintain an Intranet Home Page onto which all relevant training information could be placed. This home page would become one of the principal media through which the Training Division communicated with the Brigades. If a Brigade Chief needed to find information about the availability of assessment packages under development, or the modules in Pay Point 5, or the competencies of an individual in the Brigade, then the home page would provide this information. If necessary, it could be designed to enable limited access to sensitive material.

The suggestion of establishing a home page is a response to many of the concerns expressed in the data collected for a more transparent training system. In essence, many of the respondents at the focus workshops and interviews indicated that they didn’t understand the CBT training system, because they couldn’t easily get the information they wanted in a easy to read format. For example, only 8% of respondents felt that the level of information about CBT made available by the Training Division was a strength of the training system (see q 13), with approximately half of respondents indicating it was a weakness. The Brigade Chiefs and their senior officers indicted at the interviews that they needed to know the competencies of their staff at any given time and that a database available on an intranet home page was seen as the most efficient and cost effective solution.

This intranet facility would need to be developed over a period of time and could be established to convey a wide range of information about CBT to the Brigades. This should include: The personal training records of all career staff in a given Brigade, including skill

gaps, The names of all trainers and assessors in the TFS, together with their relevant

module expertise, The pay point system linked to the modules, The course structure linked to the pay point system, The full array of learning resources available across the State, based on the results

of the audit (see Recommendation 2), Copies of learning packages that are available on disc, Details of current training policy,

58


General training information, such as updates on the development of the national Training Package.

Complementing this supply driven approach to training would be the development of a resource database (refer to Recommendation 2). Because of the heavy reliance on training resources, especially on station, the Launceston Brigade indicated that it would strongly support the establishment of a project to map the full array of appropriate learning resources available in the State, and elsewhere, to support the on-station training.

Because this search for appropriate resources is being repeated every day around the State, it was logical to attempt to streamline the process and create a common database. An outcome of a project of this type would be a database on the intranet home page, which would list all available resources, published and unpublished, by author and description, all cross referenced to the modules. Where appropriate, some resources could be placed on the home page for down loading to a station, subject to copyright laws. Some AFAC manuals are available in electronic form for this purpose.

In conclusion, the results of the evaluation indicated that the Brigades were looking for an enhanced training service from both the Training Division and TAFE. The Federal and State Governments, in establishing the new National Training Framework and associated Australian Recognition Framework on 1/1/98, have indicated that creating more competition within the training market place will help achieve this enhanced performance. These new national training policies would allow training users (TFS Brigades) to make better use of the available training market.

In this context, the fire training market consists of all of the training providers that service the various needs of all of the Fire Agencies as well as the Agencies themselves. It therefore, is in the TFS’s interests to explore the availability of accessing other training services to complement those services that it currently provides itself.

The following services were recommended by Sandra Lunardi as providing services that may be appropriate to the needs of the TFS:

OTEN in Sydney which provides distance learning packages for many AFAC modules (contact Mike Olsen 02-97158490),

Open Learning Institute of Queensland (contact Kathy Bannister 07-32594099), and

Queensland Fire and Rescue Service (contact Mike Dwyer 07-3283052)

59


Recommendation 14 (see also Recommendation 9)

That the TFS develop a range of training policies dealing with issues such as: responsibilities and role of Brigades, responsibilities and role of training service providers (Training Division

and TAFE), assessment (see Recommendation 9), training delivery, training plans,

in preparation for the arrival of the National Training Package in 1999, and that these policies be designed to establish a strong link between training and the TFS’s Corporate Plan.

Recommendation 15

That responsibility for an individual’s training be devolved to the Brigades, with the Training Division and TAFE acting as service providers. To initiate this process, it is recommended that the RCC process be quickly finalised and that all individuals be invited to appeal if genuinely aggrieved (see Recommendation 11).

Recommendation 16

That Brigades use the outcomes of the RCC process to develop an annual Brigade Training Plan which would be subject to continuous review. That the Training Plan complement and support the implementation of the TFS’s Corporate Plan. That individual training plans for all career staff in each Brigade be developed to encourage participation within the training.

Recommendation 17

That the Training Division establish a training intranet home page to provide a transparent and easily maintained communication channel with the Brigades. That the home page become one of the principal media through which new information about training is transferred to the Brigades from the Training Division. The home page could be established so that it had read only and write to facilities as well as sections where access is restricted to selected people with security code access.

Recommendation 18

That the TFS be cognisant of its User Choice rights from 1999 onwards and take the opportunity of investigating the scope and quality of learning resources and training services available by other training providers servicing the AFAC curriculum. This is no reflection on the current training providers.

60


Attachment 1Terms of Reference

In consultation with TFS senior management, the UFU, uniformed career employees and TAFE:

1. identify the issues impacting on the success of the CBT system;

2. identify the suitability of the range of modules, the modules themselves, the learning materials, the assessment packages and assessment processes;

3. recommend ways that the development and delivery of competency-based training can be improved; and

4. make recommendations on the optimum methods (i.e. training providers and modes) for providing CBT training to uniformed career staff.

Reference Group

A reference group was established to provide direction, guidance and feedback on project performance. The reference group consisted of:

Robyn PearcePeter Alexander Brian Hevey

Richard WarwickChris ArnolGraham Campbell

Marcus SkellyDavid CastellerJohn Kelleher

Timeframe

The project was to commence from 26th January 1998. A report on items 1 and 2 was to be provided by 27 March 1998, with items 3 and 4 were to be completed by 1st May 1998.

61


Attachment 2Methodology

The methodology used in this project involved a multiphase approach which began with a global scoping phase to identify the key issues affecting the training, followed by a more detailed analysis of these issues using interviews with key stakeholders, surveys and document analysis. To initiate the scoping phase, a flier describing the evaluation process was distributed throughout the TFS.

Underpinning the methodology was the analytical framework encompassed in the, Framework for the Implementation of a Competency Based Vocational Education and Training System, (VEETAC 1993). This national policy document was the basis for all national vocational curriculum development and delivery over the period 1993-1997. Therefore, it is the most suitable ‘official’ document against which a training program can be legitimately compared, when issues such as CBT are being considered. The strength of the 1993 Framework was that it placed much emphasis on flexible learning and assessment, two issues which were likely to impact on this review.

Phase 1 Global scoping of key issues impacting on the success of the current CBT-based system

Purpose

To identify the nature of the current training arrangements

To identify the key issues that are currently affecting the Tasmania Fire Service training

To identify broad reactions to the current training services in order to identify key strengths that can be value added and weaknesses that can be addressed

To examine the quality and quantity of outputs/outcomes of the current training arrangements

Process

Three complementary approaches were used in this phase.

1A Statewide Focus Workshops

Eight two hour workshops will be conducted in four locations: Burnie x 1 Devonport x 1 Launceston x 2 Hobart x 4.

62


These site visits consisted of informal meetings held in the fire stations with as many TFS personnel who could attend. Their purpose was to ascertain the real issues that were most concerning career staff associated with the training.

The results of these focus workshops were then be used to inform the remainder of the evaluation. In essence, the data collected in this phase one activity was used to inform phases two and three.

1B Document analysis

Relevant training and development documents pertaining to the outputs/outcomes of the Tasmania Fire Service training were examined. Documents targeted for analysis included current accreditation course documents, learning materials, assessment packages/materials, national policy documents relating to AFAC and relevant TFS training policy documents.

1C Analysis of emerging national and local training reforms that might impact on the development of the current training in the fire service

This analysis involved having several telephone discussions and a face-to-face meeting with Sandra Lunardi, the AFAC Curriculum Manager to ascertain the immediate impact that the 1998 training reforms would have on this project. In particular, emphasis was placed on obtaining data on the development of the endorsed components of any national training package envisaged for this sector and the background to the development and resourcing of the AFAC curriculum.

Phase 2 Interviews with Key Stakeholders

Purpose

To ascertain the views of key stakeholders with respect to the current training arrangements.

Process

Interview schedules were prepared based on the data collected in Phase one. Interviews were held with:

TFS management (Robyn Pearce, Peter Alexander and Chris Arnold) UFU (Richard Warwick and Wayne Seabrook) TAFE teachers/managers (John Williams, Des Vernon, Peter Higgs) TFS Training Division staff (Brian Hevey, Gavin Freeman; Gary Baylis,

Wayne Richards, Ian Hoggett) Fire Station management

Hobart Tom Davidson, Hugh JonesLaunceston Mike Brown, Andrew Comar, Graham Campbell, David

Peck)Devonport Mark Porter

63


Burnie John Streets

Interviews were either one-to-one or small group depending upon the availability of staff and each lasted approximately one to two hours.

Phase 3 Statewide Questionnaire

Purpose

To provide documentary evidence of the collective opinions of all TFS staff directly affected by CBT.

To identify any inconsistencies with the training in relation to CBT principles

To identify areas where the training could be streamlined for greater efficiency and effectiveness

Process

Based on the data collected from Phases one and two, one questionnaire was developed and distributed via the TFS internal distribution network. Respondents were given approximately ten days to return the questionnaires which subsequently computer analysed using a Microsoft Access data base.

The questionnaire were first validated by the Reference Group before being distributed.

Phase 4 Data Analysis and Report Writing

Purpose

To analyse and aggregate all data collected and provide an informed report which addresses the terms of reference

To identify any inconsistencies with the training in relation to CBT principles

To make recommendations where the training could be streamlined for greater efficiency and effectiveness

To make recommendations on the optimum methods for providing CBT training to uniformed career staff

Process

All data were analysed and reported progressively to the Reference Group via two interim reports (after Phase 1 and then Phases 2). These interim reports were also made available to the TFS executive and the TAFE Institute.

64


Feedback from these groups on any technical/inaccurate facts and proposed recommendations subsequently was incorporated into this final report.

65


Attachment 3Data Collected from the Focus Workshops

Delivery Methods of the CBT Training

Comments made on the TAFE Training

Comments made on the TFS Training

Senior firefighters and above haven’t been properly briefed on CML, hence they are unable to adequately assist trainees doing their TAFE studies (x 3).

Trainees have no time to do their CML studies, as it regularly interferes with structured on station training and other duties (x 2).

There is very little CML material currently available above Pay Point 5. Trainees are thus frustrated in having to mark time while waiting for modules to be developed.

It is too difficult in trying to work alone under the CML system. No one likes working alone doing the CML.

Platoon leaders do not like individuals doing their own personal training, as they are supposed to be part of a team.

There is a high level of dissatisfaction with the CML TAFE training, by both trainees and their work colleagues.

Sometimes difficult to reach John Williams who is the only TAFE person at TAFE who knows anything about the course.

The TAFE training appears to be disorganised. The modules appear to be out of sequence and don’t relate to the type of work being done by the trainees. Many modules are too advanced for the trainees which causes frustration, slows their learning and enhances frustration with the training.

It is sometimes difficult to access the CML on station, because there is no dedicated computer.

CML is slow, unstimulating and backward. The approach does not fit the TFS needs.

On station training is basically hands on and covers most of the practical competencies required by a trainee.

Manning levels are negatively impacting on a station’s ability to deliver timely and effective training on station. (x 3) Younger trainees with pay point progression needs are given priority which means that the more mature trainees (third and fourth year) and others are missing out.

The Training Division often changes the dates of training programs which reduces the ability of a station to release staff to attend.

The general view was that on station training, although difficult to organise, was much preferred to the TAFE CML training (x 4). Provided that the resources were available, most trainees preferred to do their training on station or at Cambridge. This form of training is flexible, responsive to local needs, and provided immediate feedback.

There is no time and no manning to do effective on station training.

Constant disruption to on station training by call outs frustrates the officers’ ability to deliver quality training (x3). This is especially applicable to the longer duration modules.

Cannot access Training Division training because they are too busy with volunteer training.

Many career staff (pay point 7 and above) have had no formal training for several years.

Best training was that done at Cambridge during the recruit training block (x 2). The live in nature of the block training encourages team building and peer support.

Trainees graduating from the 12 week recruit course were more competent more quickly, and progressed through their course quicker due to their greater level of learning during the block.

66


Some who did not do the longer (12-18 week) block recruit course felt unprepared for the lifestyle.

The longer version (12-18 weeks) of the recruit course was preferred, focusing only on the core business of a young firefighter. The non-essential modules such as Community Fire Safety could be introduced later in the course (x 2).

Most felt that their was little trust in the ability of the regions to manage their own training.

General Comments Relating to Both Forms of Training

Continual frustration at having to jump from Pay Point 2 to 4 then back to 3 because the modules haven’t been developed (x 2).

Course was introduced before the modules were developed (x 2).

There is a perceived level of under-resourcing associated with the training in general.

Access to learning materials (library) difficult, especially for non-Hobart based trainees.

Assessment Processes



The average time on feedback for assessment results is three weeks, but could extend to two months (x 3).

There is a serious bug in the CML program which shuts down the system if you take more than 45 minutes to complete a progress test. This often occurs when called out during a test, so a failure is recorded.

The CML allows no flexibility, no personal interaction.

The turnaround on final assessments has generally been good. Most come back promptly (x 2).

The lack of access to dedicated PC’s for CML work is frustrating. It means you can only do the CML work when the PC is free, and this often does not coincide with passive time for training.

Some modules (e.g. Wildfire Behaviour 1), if failed, require a complete resit of the examination, rather than just the portion that you failed (x 2).

75% pass mark is difficult to reach on a written external exam.

There are insufficient assessors to run the system efficiently (x 2). We have four on our shift (Hobart) and it is still not enough. Whenever someone needs to be assessed, we need to transfer staff around the town in order to provide adequate coverage. The whole process is time consuming and inefficient.

Had to wait two months for the assessor to assess me after completing training on shift.

Having the assessment process separated from the teaching process is inefficient. Trainees don’t get immediate feedback from their instructor because he/she has no assessment data (x 6).

Prefer all instructors to be able to also assess the trainees concurrent with the instruction (x 7). This is more efficient and would enable the instructor to tailor the training to suit the speed of learning of the individual, rather than assuming that all of the material is understood.

The assessment process is very bureaucratic, and difficult to resource for the smaller stations (x 2). It is a logistical nightmare for smaller stations. When an on station assessment needs to be done,

67




Assessments have been known to be lost. You then have to do them again (x 2).

Every module has at least a two hour formal exam under the CML. Finding time to supervise someone for two hours without interruption is very difficult- the process does not fit with the TFS work flow.

It is frustrating to pass a final exam from TAFE and then find out you also need to be assessed again on station.

Assessment is unfair- the use of percentages is more like high school.

Sit down, closed book exams are inconsistent with the principles of CBT.

it first needs to be ordered from the Training Division by phone, then set up with the assessor and trainee, followed by validation and then marked by someone in the Training Division. Finally the trainee gets the results. It can take up to three months.

The use of verbal assessments and reassessments was common at the Cambridge recruit training. This was welcomed because it was efficient and didn’t require the whole module to be reassessed (x 2).

People are being held back with progress to their pay points because they cannot get assessed.

People at the top end of the pay scale (e.g. 6+) are not progressing because of the unavailability of the assessment packages and the learning resources.

The assessment approaches on station are much better than that given by TAFE. With internal assessments, you only have to redo the work you failed. This is much fairer than having to do the lot again.

Some of the practical assessments seem to be too involved and time consuming.

There are not enough assessment packages ready (x 3).

Through the assessment process, there is too much power in the hands of the Training Division at Cambridge.

General Comments Relating to both Forms of Training

There is an over-reliance on written assessments at the expense of verbal questioning. Many of the trainees, because of their background, find it easier to explain an answer orally, rather than having to write it down

Assessors are not always technically competent, or confident, to assess all modules in the course. Often, they have to rely on the instructor to see whether the trainee’s response is correct. This makes a mockery of the assessor as being independent of the instructor.

With 9 assessors across 6 stations in Hobart, there are insufficient assessors to efficiently run the system.

General feeling is that firefighters are over-assessed.

The assessment system is intimidating for both the trainee and the assessor because of the bureaucracy.

68




Suitability of Module Content



There is gross-over-training in many TAFE modules (x 2).

Many of the TAFE modules are not relevant to being a firefighter e.g. public speaking.

TAFE training is not job-specific.

Focus on the core business of fighting fires first in the training, and the other material can come later.

The induction training should only contain the basic essential competencies- the must know material that a recruit needs to survive as a firefighter (x2).

On station training mainly focuses on practical content. Most respondents were generally happy with the material delivered on station (x 3).

Group learning and peer support has been lost as most trainees now study in isolation.


There is generally too much material to learn in the course.

A trainee with an electrical trade background commented that there is a huge work load in this course compared to the electrical apprenticeship he undertook and passed [Author’s comment: the electrical apprenticeship is considered one of the most difficult of all apprenticeships, given the level of mathematical/electrical/electronic theory required].

The content needs to relate to the core business of being a firefighter, especially in the early years of training (x 3). The sequence of the learning is wrong. Need to do the must know material first.

There are too many modules not yet fully developed.

Over-training is rife at the base grade for both the TAFE and on station training (x 3). For example, material that goes beyond the basic needs of the module (Computer skills) or material that is only required by leading firefighters and above (e.g. Occupational Hazards, Dangerous Substances) (x 2).

The whole training system is driven by the pay point system.

There must be more consultation with operational staff on the content of the base grade training modules.

Some modules have material in them that is given to a trainee, when it is only required by senior firefighters and above.

There is a perception that there is little or no training available to senior and leading firefighters. The recruits and volunteers get all of the resources.

AFAC modules have been well received, (x 2), but there are not enough of them available, and some go a little too deep at times.

69


There is too much overlap with some modules: Urban Fire Suppression 1 & 2 Wildfire Suppression 1 & 2 Workplace Communications, Work Team Communications, Present Information, Workplace

Trainer Cat 1.

Recognition of Prior Learning and Recognition of Current Competencies



RPL was offered to new recruits upon initial enrolment (x 3).

Some were only offered RPL a year or more after starting the course.

No RPL was offered at the enrolment process stage (x 3).

The RPL process is complicated and bureaucratic.

RPL was not encouraged because it was quicker to do the module than follow through with the complicated RPL process.

Two people still waiting six months for their RCC results.

Some have been waiting several months for their RCC results.

Unimpressed with the RCC process. Have no confidence in the results.

RCC process was too subjective and open to personal opinion of the RCC assessor.

RCC process too fast, no time to prepare. No follow up.

RCC was introduced because the TFS might lose its training provider registration.

The RCC process places too much responsibility on Brian Hevey. It is presumptuous to suggest that Brian should be the final arbiter.

RCC process dependent on how well you can articulate your competency level (x 2). Those who were highly assertive and capable of articulating their competency levels got more recognition.

Since Brian didn’t know me, you could easily pull the wool over his eyes.

Most were satisfied with the RCC process (x 3).

Having an independent person manage the RCC was favourably accepted.

Reluctant to give RCC for people with qualifications gained outside the TFS.

RCC process was inconsistent. Some felt that they got too much recognition (senior officer) while others felt that they got too little.

The introduction of the RCC process is discouraging people from doing their module training, because many are anticipating that they might be given recognition.

70




Some people are being held back from doing their TAFE modules because it is taking too long for the RCC results to be finalised.

RCC process was ad hoc and varied from a 20 mn interview to one hour. If you don’t like the results, you have to prove Brian is wrong.

Most RCC interviews took less than 15 mn.

RCC needs to be managed by the station/regional management; by people who really know the competency levels of the firefighters.

RCC does not adequately assess your competencies.

RCC favoured senior ranks who could ignore the results whereas the lower ranks could not progress their career without doing all the extra training (x 2).

Relationship with the Training Providers (TAFE and TFS Training Division)



Trainees had a good relationship with John Williams (x 3).He seems to be the only one who has any idea about the course (x 2).

It is difficult to get information when John Williams is unavailable.

Have a neutral attitude to TAFE.

Rarely get direct feedback on work submitted.

Hard to reach part-time TAFE teachers for feedback because they are never available.

Officers and other senior ranks have little comprehension of the CML process and thus cannot help the trainees (x 2).

The relationship between trainees and trainers/assessors on station is very good (x 6).

Training Division support is minimal because there is insufficient budget to release people for training.

On station training is mixed-some very good and some very poor.

Some assessors are also doing training themselves, so they are very busy.

71


Training Facilities



John Williams doesn’t realise how difficult it is for firefighters to access information, because he and his staff are not available on night shift and weekends.

John Williams is only available about 60% of the time, during business hours.

Cambridge facility is excellent, but under utilised.

It is difficult to access the Hobart TFS library at night and on weekends.


Learning facilities at Hobart No. 1 are good. Most study has to be done at night. Because Hobart No. 1 is so busy, it is easier to do personal study at the out stations, but many don’t have good training facilities. Outside training facilities at Hobart No. 1 are now very cramped.

At Launceston, we have only one PC on site in the training room, so trainees sometimes need to queue up. Learning resources are fairly scant.

Burnie has a dedicated training room on station, but trainees need to share a PC with others. Most of the learning resources are dated. It is hard to get resources and information on relevant material.

Officers and trainees at Devonport are generally satisfied with the training facilities on station. It is sometimes difficult to access material and information from Hobart.

Strengths of the Current System



Portability of the qualifications (x 2).

John William as the key contact person.

Flexible

CML concept is good but the practice is poor.

CML feedback on progressive tests instantaneous.

Senior officers have a strong commitment to training their firefighters.

Willingness of firefighters to train in their own time if necessary.

Training is job specific and directly relevant to the job (x 3).

Every time you respond to an incident you are always learning.

On station training encourages team building.

The hands on nature of the training (x 2).

Training is timely and conducted when it is required.

Credibility of the trainers (x 2).

Small group training encourages group dynamics, peer support (x 2) and helps motivate people to

72


succeed.

You need to know one another’s strengths and weaknesses and this helps you when you get to the sharp end of the job. This can come from good on station training.

On station training must be retained and balanced with an injection of new training, maybe off-site at Cambridge, to provide a balance.

The recruit course is a major strength (x 2), and it must be live in for at least three months to encourage team building and peer support, focusing on practical, must know aspects of the job at the entry level.

One-on-one nature of the on site training.

You retain more by learning as a group.


AFAC modules are an excellent resource.

Weaknesses of the Current Training System



A hands on job needs a hands on training course, not a theory based CML course.

Only person who knows anything is John Williams.

Senior officers know very little about CML.

Too much self-paced learning too soon. It should be left for a later part of the course.

Because of the CML training, it is hard to know what the competency level of each person is at any given time.

Most of the TAFE training is irrelevant ( x 2)

Gross over-training.

Limited to doing the training at work unless you have a PC connected to the TAFE e-mail.

CML not interactive.

Insufficient time to do the TAFE work.

Learning only for the exam.

Insufficient trainers with qualifications at Category 1.

Lack of workplace assessors.

Lack of training for officers to teach firefighters.

No designated training time on station.

It is an assessment driven system.

Training records are slow to be updated.

No time for training.

Insufficient manning levels for quality on station or Cambridge training (x 3).

Cannot get the time to train-always being interrupted with call outs (x 5).

Lack of expertise on station to deliver training.

If people don’t want to progress beyond pay point three, then they don’t have to. Hence, their skills will not improve.

73


As many modules don’t yet have assessment packages, you have to wait several months or longer to be assessed..

Computer marking is constantly failing people who are competent.

Assessment process being separated from the trainer.

With the trainer not assessing, there is no guidance on the progress of the trainee.

No standardised approach to on station training.

Impossible to get trainers from Training Division to deliver training on site (from NW).

Impossible to send more than one person to a training session at Cambridge.

Recruit courses need to be at least 12 weeks or longer.

Perception that there is too much training done in Hobart that could be done in the north e.g. First Aid, Foam Workshop, Incident Management.

In many out stations, there is no designated places to study and many have inadequate CML computer support.

Lack of access to Cambridge based training rare for most senior officers. Many indicated that they hadn’t received training in two to three years.


Most of the core modules should be TFS not TAFE.

Just too long and frustrating to pass all of the modules.

The whole training system is a mystery to most people.

Held back getting to a pay point because the modules are not ready (x 2).

Suggestions to Improve the Current Training System



Do basic training and then a Fire Technology-type course. Leave CML until later.

Drop CML, at least in the first year or two.

Make the TAFE modules more relevant to the work of the firefighters.

Operational staff should be offered at least one full shift per year to undertake training, preferably at Cambridge. The training week release could be linked to each person’s annual leave, if appropriate (x 3).

Need better utilisation of the Cambridge facility.

Need a training system that allows people to be trained without being called out to incidents.

Need more off station training to avoid the call out problem.

74


To pay point 6, structure the course so that trainers can manage the assessments on station.

Let trainers also assess.

On station training needs to be better structured.

Employ more staff to enable more training to be undertaken off station.


Need a better data base of personal training files to be accessed from each station.

Establish a home page where training information can be distributed throughout the agency.

Provide more training for career staff who get none at the moment. Trainees and volunteers get all of the training resources.

Need to re-redline people again to reduce the tension in the system.

Want a training system with an end to it. There is a strong view that no one will ever graduate.

Streamline the training. Cut it to the bone and make it relevant.

Pause the training so that it can be managed and not just repaired.

Make the training more accessible.

People who do training in their own time should be given TOIL.

Make more modules available to eliminate the pay point log jam.

Ensure that the material in the modules does not go beyond the needs of the job.

75


Final General Comments About the Training System



No Cambridge based trainers have delivered training on site in recent years.


Officer development training is very thin and poorly supported by the Training Division.

No one has been given a list of course modules to see what they have to do.

Unless this review is acknowledged and followed up, it will be counterproductive, as it won’t be the first time our advice has been ignored.

Since manning levels have dropped, skill levels of firefighters have dropped because we cannot release people for training.

Cannot get training on special equipment.

Need a wider view of what base level training is all about.

Fully support CBT but it was implemented too early, is under-resourced and many people have been reluctant to accept mistakes made in the past.

All people need to be re-redlined to provide appropriate recognition.

For someone who is a senior firefighter, there is no incentive to try to progress, because it is just too hard now.

The qualification is not appreciated by the service, therefore the training is driven by the pay point system.

There is a strong perception that the volunteer training is better than the career staff training.

It is inappropriate to train someone and then immediately expect them to train others in the same technical area.

Need more outside instructors, especially for First Aid (x 2).

There is no TFS training being provided to career staff in Burnie by the Training Division.

Cannot get external trainers to deliver training in Hobart.

The Cat 1 Instructional Design module is inadequate to meet the training needs of officers.

Pride in the job is the key motivator for training.

The whole training system is driven by gaining pay rises.

76


Attachment 4Questionnaire

MEMORANDUM

To: Brigade Chief, Devonport Fire BrigadeBrigade Chief, Burnie Fire BrigadeDistrict Officer (Human Resources), Hobart Fire BrigadeDistrict Officer (Human Resources), Launceston Fire Brigade

From: Robyn Pearce, Manager (Human Resources)

Date: 24 March 1998

Subject: Survey Questionnaire – TFS Training Review

As you are aware, John Kelleher has been engaged to conduct a review of TFS competency-based training systems.

The first two stages of the review involved focus group sessions with career brigade employees and interviews with other stakeholders, such as supervisors, managers, UFU and TAFE. These stages have enabled the development of a questionnaire aimed at gathering information from all uniformed career employees rather than only those able to attend the focus group sessions. This questionnaire forms the third stage of the review.

In order to get the questionnaire to as many people as possible, it is intended to have it available for completion over the annual leave changeover period from March 26 to April 10, 1998. I would appreciate if you could ask all Station Officers to have employees on their shift complete the questionnaire over this period. I should note that it is not intended for the questionnaire to be completed by groups of employees. We need each individual person’s views rather than that of the peer group. To achieve this and to ensure a high return rate of questionnaires, I ask that the Station Officers facilitate this being done in a structured manner while on shift.

The survey period finishes on April 10, 1998. However, there will be a large number of questionnaires to be analysed by John Kelleher. To help as do this efficiently, could you please return to me at the end of each shift (ie 4 days) the questionnaires that have been completed. I will then forward them to John.

The final thing I ask is that, if a person is absent from shift over this period for some reason, could you please mail them a copy of the questionnaire and ask that they return it to me by April 10, 1998. The cover sheet for the questionnaire contains some additional details.

Thank you for your assistance and please contact me if additional information is required.

Regards

77


Robyn Pearce

Tastrain

Evaluation of the TFS CBT Training

Questionnaire for all Career Staff

This questionnaire forms part of a major review of the competency based training that the TFS is currently undertaking. The review commenced in late January when I conducted a series of workshops in all regions of the State. This questionnaire is a follow up to those workshops, and will give all career staff the opportunity to have their say. This is a real opportunity for you to assist the TFS to make your training more efficient and effective. Please take the trouble to complete the questionnaire as soon as possible.

It is not recommended that the survey be completed as a group. We need to know what you think, not what your work group thinks, just in case you have a different opinion. Try to answer each question from your own personal experience of CBT.

It is not necessary to indicate your name. However, if you are concerned by confidentiality, please place your completed questionnaire in a sealed envelope and give it to your station officer, or Brigade Chief.

Please return this questionnaire to your station officer as soon as possible, but no later than April 10th . If possible, return it the same day that you receive it.

For staff who are absent from work between 26th March and 10th April, please ensure that a copy of this questionnaire is mailed to them with instructions to return it to their station, or Robyn Pearce in Hobart by 10th April.

All questionnaires are to be returned to Robyn Pearce in Hobart by 10th April.

Thank you for your assistance in participating in this review of the TFS’s competency based training.

John KelleherEvaluator

23/3/98

78


List of Terms with which you may be Unfamiliar

This questionnaire uses a number of terms and acronyms which you may not have seen before. So, to ensure that every one can fully understand the questionnaire, some definitions and explanations are given below. Please note that this questionnaire only refers to the competency based training. It does not relate to the old Fire Technology courses delivered by TAFE.

CBT This is the competency based training introduced by the TFS in mid-1995. It consists of several modules taught by the TFS on station and at Cambridge and by TAFE using flexible delivery.

Trainee Person who has undertaken structured competency based training.

Trainer Person who delivers, or teaches the trainee as part of the structured competency based training.

Cambridge Location of the training centre in southern Tasmania.

Under-resourced Insufficient funds or personnel to do the job adequately.

TAFE Technical and Further Education. TAFE delivers part of the competency based training from the Flexible Learning Centre at Clarence in Hobart.

Module A subject in the competency based training.

Progressive assessments Assessments conducted during the module.

Final assessment Assessment conducted at the conclusion of the module.

CML Computer Managed Learning. This is the system used by TAFE do enable trainees to access progressive assessments via a computer.

RCC/redlining Recognition of Current Competence. The redlining process enables TFS career staff to have their individual competencies recognised against the CBT modules.

Appeal Process whereby the individual asks for the results of the assessment or RCC to be reviewed.

Certificate The qualification the trainee receives from the TFS and TAFE after completing the required number of Modules e.g. Level 3 Certificate.

Training Division The training unit based at Cambridge that is headed by Brian Hevey.

Learning resource material Manuals, books, videos, etc available to assist trainees to learn better.

79


Please indicate your response by ticking the box that best represents your answer. In some questions, it will indicate if you are free to tick

more than one box or give your own alternative response.

1. Regional area? South North North West

2. Current rank? Firefighter, First Class Firefighter, Senior Firefighter Leading Firefighter Station Officer, Senior Station officer Higher ranks

3 Role in the CBT? (You may need to tick more than one box if you have more than one role, e.g. trainee and assessor)Trainee Trainer Assessor Validator RPL facilitator Manager/coordinator of training on station No role

4. Have you ever attempted to undertake a CBT module?

Yes No Don’t know

If No, please indicate why_______________________________________________________________________________________________________________________

5 Which of the following best describes your attitude/feelings towards CBT:I am comfortable with the nature of CBT within the TFS I have no particular opinion of CBT within the TFS at present I am concerned about the nature of CBT within the TFS Other ______________________________________________________________________

6. From your knowledge and experience of the CBT initial recruit training, which of the following approaches do you most favour?

No initial off station recruit training at all

A short three to four week block at Cambridge

A three to four week program delivered in the local region

A 12 week block at Cambridge

A 16-20 week block at Cambridge

Another approach: please indicate __________________________________________________________________________________________________________

80


7 If the roles of trainer and assessor were to be merged and all of the trainers were allowed to assess and assessors allowed to train, indicate your response to the following:

Agree Disagree Cannot say

The credibility of the training would be undermined The assessment and training would be greatly streamlined CBT would be more acceptable

8. Indicate whether you agree or disagree with the following statements:Agree Disagree Cannot say

8.1 A firefighter’s role is more than just emergency response. There is an important community education function also. 8.2 CBT is under-resourced 8.3 The TAFE CML system is flawed 8.4 Many of the modules appear to be out of sequence with the needs of the job 8.5 The assessment process is complicated and time consuming 8.6 The RCC/redlining process was unfair 8.7 Should be able to appeal the RCC decision, if genuinely upset by the result 8.8 There is too much training delivered by TAFE that is not directly related to my job 8.9 There should be more block training at Cambridge, in addition to the recruit training 8.10 CBT overall, provides a good preparation for the job of firefighter 8.11 The TAFE course coordinator is always willing to help when requested 8.12 It is very difficult to communicate with the TAFE teachers, as they are often hard to reach 8.13 The TFS assessment process takes too long before the trainee gets the results 8.14 The TAFE assessment process takes too long before the trainee gets the results

9. If you indicated in question 5, that you are concerned about CBT, indicate your greatest concern by selecting one of the statements from question 8 above, or write your own response. My greatest concern is:Item 8._____________________________ or ______________________________________________________________________________________________

81


10 Would you support any of the following changes to CBT?Yes No Cannot say

Allow trainers to also assess so trainees get immediateassessment feedback on their progress Remove the over reliance on formal written assessments Introduce more progressive assessments and reduce the use of final assessments conducted at the end of a module Restructure CBT so that the core competencies of firefighting are delivered earlier in the training program Reduce the number of modules you have to do to graduate with the Level 3 Certificate (Pay Point 6) Introduce a fairer and more consistent RCC process that is managed at the station/brigade level. Give more responsibility for the management of training to the brigades Make more use of on station training Make more use of regular block training at Cambridge Provide more information about the CBT modules that is easilyaccessible to the stations

Use TAFE to deliver more CBT modules

11 Do you feel that there are sufficient assessors available in your area to meet the needs of the on station training?

Yes No Don’t know

12 Which of the following statements best describes the way you like to learn?

In a structured way with the support of the class and the assistance of a face-to-face teacher/trainer In a self-paced manner where I can progress at my own pace without direct face-to-face contact with a trainer and the need to follow a set timetable In a small group of work colleagues on station where close contact witha trainer is always available

Not applicable to me

Some other answer. Please indicate ________________________________________________

______________________________________________________________________________

82


13 Indicate if you consider any of the following features are strengths or weaknesses of the competency based training in the TFS:

Strength Weakness Cannot sayThe service TAFE provides The service provided by the Training Division Your ability to use and understand the TAFE CML system The on station training The block recruit training at Cambridge The CBT assessment system The rate of progress on the development of assessmentand learning resources The amount of learning resource material available for training on station The ability of stations to release people for training The overall content in the competency based training (what you have to learn) The order in which the modules are taught The ability of CBT to provide a portable qualification The level of information about CBT made available by the Training Division The level of information about CBT made available by TAFE

14 Are you aware that the CML training can be substituted with a distance learning package?

Yes No Not applicable If yes have you used this option in preference to the CML?

Yes No Not applicable If no, please indicate why by ticking the appropriate box.

CML is easy to use and more efficient

I like the fast feedback the CML gives me

I like to use the computer to help me with my training

Other, please indicate:_________________________________________________________

83


15. There is a perception that there is over-training in CBT. Do you agree with this statement?

Yes No Don’t know If yes, indicate where you feel that the over-training occurs. Tick one or more boxes.

Overlapping content in some modules e.g. communications stream modules. Give other examplesyou are aware of, if any ________________________________________________________________________________________________________________

Too many modules in the CBT module structure Some modules have too much material not relevant to the TFS. Give examples, if any:______________________________________________________________________________ Some modules delivered early to new recruits have material that should really come laterGive examples, if any:______________________________________________________

Other examples of over-training you are aware of, if any________________________________________________________________________________________________________________________________________________________________________________________________

Thank you for assisting us to make the competency based training better.

84