ALERT™ Education Outcomes and Impact on Clinical Practice Chris Parr Senior Information Analyst 1

ALERT™ Education Outcomes and Impact on Clinical Practice

Chris ParrSenior Information Analyst

1

What we hope this will do for you…• Provide you with evidence of the impact the training

has on patient care and justify your investment by demonstrating the value of the course.

• Enable you to quickly and easily report this evidence to your stakeholders (Patients, Trust Board, various departments of your trust, LETB, CCG).

• Contribute to your compliance to the Education Outcomes Framework.

The HEE Mandate requires that there is a direct line of sight between education and training and the patient. 2

What we hope this will do for you…• Allow you to identify any issues with the training

whether they are local or course based.

• Allow you to compare the training before and after any changes are made.

• Instantly feedback to us so that we can make improvements/adjustments to the course.

3

The Surveys• On The Day survey• Post Course survey

• Both surveys were put together by a group made up of clinicians, L&D staff, and external survey experts.

• The questions are given weighted scores based on their importance and once collated give an overall score out of 100.

4

Results: On The Day Survey

5

Green Bars: ≥63Programme exceeding standards

Amber Bars: 46 - 62Programme only just meeting standards, or falling slightly

below standardsRed Bars: ≤45

Programme falling below standards

Data from 6 ALERT courses run between May 2014 to August 2014

100

25

25

25

25

86 /

23 /

22 /

22 /

19 /


6

25

8

8

5

4


7

25

4

5

8

8


8

25

16

3

3

3


9

25

5

5

5

10

10

11

12

Tick all that apply

13

Tick all that apply

14

Tick all that apply

Post Course survey

15

• Key to measure the actual impact of the course.

• Designed to gain honest answers from the respondent focussing on what they do rather than what they know.

• Questions are predominantly frequency based (Never / Sometimes Usually / Always).

• Each score indicates the return on investment as the number of pence in the pound. For example, a category score of 89 indicates that the trust is getting a return on investment of 89 pence in the pound for that category

Results: Post Course survey

16

100

31

44

25

Overall

Candidate

G ≥ 80A 61 - 79R ≤ 60

Programme only just meeting standards or falling slightly below standardsProgramme exceeding standards

Programme falling below standards


17

31

22

9

Sub-section 1 – Confidence and Capability

Candidate


18

31

16

28

Sub-section 2 – Safe Practice

Candidate


19

25

11

14

Sub-section 3 – Underpinning Knowledge

Candidate

Collection Process• On The Day survey collected on the day of the

training course.• Post Course survey collected via e-mail

approximately 3 months after the training course.

• E-mail addresses of the trainee and their manager to be collected on the register and sent to PHT.

• Please promote the Post Course survey on the training day.

20

Collection process: On the Day Survey

21

+ Everyone can complete a survey

- Manual data entry required

PaperPaper

Manual Data EntryManual Data Entry

QR CodeQR

Code

Survey WebsiteSurvey Website

+ Data feeds directly into system

- Candidates need to have access to a smartphone / tablet

- QR code reader app required

- Possible difficulty typing correct web address

Web Address Web

Address

www…www…

::::::::::::::::::::::::::::::::::::::::

::::::::::::::::::::::::::::::::::::::::

- Accessible Wi-Fi or internet signal required

Option 1 Option 2

22

On The Day Evaluation Survey

http://www.elesurvey.co.uk/f/612318/e847/

23

Collection process: Post Course Survey

24

Candidatee-mail

Candidatee-mail

PHT system

PHT system

Candidate’s Manager

e-mail

Candidate’s Manager

e-mail

Registration to include…Registration to include…

Link to Candidate survey sent to

candidate e-mail

Link to Candidate survey sent to

candidate e-mail

www…www…

Link to Manager survey sent to

manager e-mail

Link to Manager survey sent to

manager e-mail

www…www…

3 months later

?

25

Downloading the results

We will send you an EleSurvey link specific to your Trust.

26


You can filter by date to report on a specific training day, a month,a quarter, or the year in full.

27


Then export the scores to Excel.

Using the Excel Templates

28

• We will send you 2 templates (1 for On the Day and 1 for Post Course) along with simple instructions on how to use them.

Basically select all, copy and paste into the template. Then Save As. Then Save As PDF.

Survey WebsiteSurvey

Website Excel ExportExcel ExportPHT

ExcelTemplate

PHTExcel

Template

PDF in printable

format

PDF in printable

format

29On the Day survey print preview

30

Post Course surveyprint preview

Candidate

Candidate

Candidate

Candidate

Faculty / Peer Assessment

• This was one of the issues that we discovered and have subsequently resolved at PHT.

• This will be covered in Developing ALERT™ Faculty at 14:15.

31

Summary• Identified benefits of using this evaluation

process.

• Taken a look at example results from PHT.

• Mapped out the collection processes for both On The Day and Post Course.

• Talked about extracting the results.

• Shown what the print outs will look like.32

Discussion• Issues you foresee. e.g. Manager’s e-mail – best way to collect this?

• Possible solutions to these issues.

• Where would this data be best presented and to whom.

• How we can help make the process easier.33

Documents

ALERT™ Education Outcomes and Impact on Clinical Practice Chris Parr Senior Information Analyst 1