Executive summary. - DSS Web viewxxviii. 10/11/2016des youth mental health trial - evaluation report . xxviii. xxviii. 10/11/2016des youth mental health trial - evaluation report

Department of Social ServicesDES Youth Mental Health Trial

Evaluation

Issue Date: 16 September 2016Project # DSS0004

Twitter: ColmarBruntonAUwww.colmarbrunton.com

1

Contents.

1. Executive summary..............................................................................................41.1. Introduction.............................................................................................................................................4

1.2. Findings in brief.......................................................................................................................................5

1.3. Conclusions.............................................................................................................................................9

1.4. Recommendations................................................................................................................................13

2. Introduction.........................................................................................................162.1. Background...........................................................................................................................................16

2.2. Evaluation objectives............................................................................................................................17

2.3. Methodology in brief..............................................................................................................................17

3. Findings..............................................................................................................223.1. Employment, Outcomes and Success..................................................................................................22

3.2. The Career Account..............................................................................................................................28

3.3. Journey mapping for trial participants...................................................................................................40

3.4. Providers experience with the trial........................................................................................................47

3.5. Areas of potential improvement............................................................................................................52

4. Appendix A: Methodology in detail.....................................................................57

5. Appendix B: Ethics Application...........................................................................65

6. Appendix C: Initial Participant Discussion Guide................................................77

7. Appendix D: Follow up Participant Discussion Guide.........................................83

8. Appendix E: Initial Provider Discussion Guide....................................................89

10. Appendix F: Follow up Provider Discussion Guide..........................................95

11. Appendix G: Trial participants and control group matches............................101

12. Appendix H: Power Analysis.........................................................................114

2

Index of Figures

Figure 1: Program logic framework 18


Figure 3: Outcomes for trial participants and the control group 25

Figure 4: Employment journey map 27

Figure 5: Average spend by 13 week Outcomes 31

Figure 6: Spend categories 33

Figure 7: Mapping the Youth Mental Health trial 40

Figure 8: Trial participation decision making 42

Figure 9: Employment journey map 44

Figure 10: Deliverables and timings 58

Figure 11: Research approach 59


Figure 13: Trial participants and control group matches 101

Figure 14: Power Analysis calculations 114

3

1. Executive summary.

1.1. Introduction

Colmar Brunton Social Research (Colmar Brunton) was engaged by the Department of Social Services (the Department) to conduct an evaluation of the Disability Employment Services (DES) Youth Mental Health trial.

The DES Youth Mental Health trial was a one-year trial designed to test an individually tailored - and participant-driven - employment support model. The trial was conducted in Employment Services Areas in Queensland, Victoria and Western Australia. In the trial, up to 200 participants aged 24 years and under with a variety of diagnosed mental illness conditions were provided access to $5,000 worth of funding through a ‘Career Account’ administered by their local DES provider. Trial participants were afforded a high degree of choice in how the funds were spent as long as the intent was to overcome vocational and non-vocational barriers and meeting employment goals.

The objectives of this evaluation were to:

• Assess the extent to which the trial participants have and exercise greater control and/or

choice;

• Identify whether (and how) the flexibility of a Career Account benefits trial participants through

employment outcomes;

• Identify any challenges or concerns raised by trial participants and DES providers;

• Identify key risks in potentially further expanding any subsequent model and mitigation

measures to address these risks; and

• Draw out possible implications for design of support for employment outcomes for young

people with mental health conditions in the future, which do not significantly increase the risk

of inappropriate use of Government funding.

The evaluation was conducted over several stages:

• The first stage involved n=33 face to face interviews with trial participants and n=9 face to

face interviews with DES providers participating in the trial. This initial qualitative stage was

conducted in three locations (Brisbane, Melbourne and Perth) between 9 and 18 February

2016.

• The second stage involved n=23 follow-up telephone interviews with trial participants and n=9

follow-up telephone interviews with DES providers (all of whom participated in the initial

stage). The research was conducted in between 11 May and 15 June 2016.

4

• The third stage involved detailed quantitative analysis of program administrative data supplied

by the Department for both trial participants and a control sample. This administrative data

was provided in July 2016 after the trial had reached completion.

1.2. Findings in brief

In order to evaluate the Youth Mental Health trial, Colmar Brunton conducted qualitative research with trial participants and DES providers and also examined administrative data that was provided by the Department. The following points highlight the key findings from the evaluation.

Employment, Outcomes and Success:

• Administrative data provided by the Department allowed for a control group to be created to

match against trial participants. The control group was chosen by matching as closely as

possible each trial participant with two equivalent individuals from the pool of control subjects.

Each participant was matched on the basis of age, gender, funding level, education level,

months unemployed, CALD status, Homeless status, Refugee status, ex offender status,

ATSI status, mental health conditions and geographic areas with similar unemployment rates.

• The administrative data showed that trial participants did not perform as well as the control

group for both 13 week Full Outcomes (6% compared to 18% respectively) and for 26 week

Full Outcomes (0% and 11% respectively). Trial participants and the control group did

however perform similarly with regards to 13 week Pathway Outcomes (4% and 4%

respectively) and 26 week Pathway Outcomes (0% and 1% respectively).

• It was observed that the nature of the trial may have been a contributing factor with regards to

the differences in employment outcomes (as outlined above). The trial provided participants

with additional funds (i.e. the $5,000 Career Account) and time (i.e. the Career Account could

be used until the end of the trial at 30 June 2016) to address the vocational and non-

vocational barriers to employment that they themselves identified. Conversely, those not

participating in the trial (i.e. the control group) were not afforded these opportunities and as

such the assumption is that their focus would have been finding immediate employment (as

per the usual arrangement). Additionally, as trial participants were not randomly selected, it is

possible that providers selected job seekers to participate in the trial that they felt would be

harder to place into employment. Unfortunately it is not possible to identify whether, or the

extent to which, this occurred from the data available.

• The qualitative interviews with trial participants and DES providers suggested that another

key reason as to why so few trial participants were employed at the conclusion of the trial was

that they did not feel mentally ready for employment and were still focusing on addressing

their mental health issues. For several trial participants this meant that they needed or were

still undertaking specialist counselling or assistance and/or needed medication to assist with

addressing their mental health needs.

5

The Career Account:

• The qualitative research highlighted that some trial participants experienced feelings of

trepidation over the amount of money they were being given access to through the Career

Account. The primary reasons for these feelings of anxiety and nervousness were not having

ever accessed such a large sum of money, finding the concept of self-agency daunting (as it

was vastly different from the usual situation) and/or self-imposed pressure to not waste the

opportunity of these funds being made available to them. It was noted that these feelings

were principally eased by reassurance provided by both their DES providers and family.

• Administrative data provided by the Department included details of all expenditure of Career

Account funds split by trial participants. This data was examined in detail and it was found

that total spend of funds from the Career Account was $480,022, with trial participants

spending an average of $2,540 – which was approximately half of what each participant had

access to ($5,000). This supports the findings from the qualitative stage of the study (noted

above) that found participants commonly struggled to develop plans to spend the full

allocated amount.

• The details of expenditure were coded into twelve categories, with the largest areas of

expenditure being:

education and education related expenses ($177,178), which accounted for just over a

third (37%) of total spend;

travel and transport expenses ($75,764), which accounted for 16% of total spend; and

specialist mental health assistance expenses ($71,419), which accounted for 15% of total

spend.

• The 6% of trial participants that reached a 13 week Full Outcome generally spent their Career

Account funds on the same areas as the total trial group. Education and education related

expenses was the top spend category (41% of spend compared to 37% for all trial

participants), mental health assistance was the second (19% of spend compared to 15% for

all trial participants) and travel and transport assistance was the third highest spend (15% of

spend compared to 16% for all trial participants).

• Both the administrative data and the qualitative research with participants and providers

suggested many participants struggled with the degree of self-agency the trial afforded them.

This tended to be reflected in participants taking longer to engage in the trial overall (e.g.

developing a plan of how to spend the monies in addressing vocational and non-vocational

barriers to employment) and in monies not being fully spent during the trial. Of note, DES

providers mentioned that they felt constrained by the amount of guidance that they could give

participants concerning how the monies could be used specifically because the trial was

aimed at encouraging participants to exercise control over this process.

6

• Trial participants and providers also noted logistical challenges with regards to how payments

were made for goods and services purchased with Career Account funds. Under the

governance arrangements of the program, participants were required to spend Career

Account funds via their DES provider (who would physically make the payments on the

participant’s behalf). While this approach was important for the governance, integrity and

ultimate accountability of the program, participants themselves noted this process was

sometimes a barrier to spending the funds. Some participants noted that they were reluctant

to ask their providers to pay for things due to feelings of shame and embarrassment, which

was counter to the programme’s intention of fostering self-agency among participants

themselves. DES providers also noted challenges in regards to use of credit cards not

specifically set-up for the program including card limits below Career Account entitlements

and the need for such expenditure to be subsequently reconciled against the Career Account

itself. Both trial participants and DES providers saw the main benefit of the Career Account to

be the ability for the participants to direct (within reason) how the funds would be spent, as

opposed to the amount of monies made available to each participant. Reflective of the

average spend being around half of the monies available, both participants and providers

suggested a smaller amount of money may have the same impact on self-esteem and

feelings of self-agency while potentially allowing a larger number of people to access such

funding.

Journey mapping for trial participants:

Across the two phases of depth interviews conducted with trial participants and DES providers, it became apparent that there was a general sequence of key stages that participants followed in relation to the trial. The journey map developed for trial participants is summarized below.

• Awareness of the trial – For the vast majority of trial participants, their DES provider was the

primary source of awareness and information with regards to the trial. Few participants

recalled receiving or reading the letter that was distributed by the Department during the early

stages of the trial.

• Joining the trial – For most of the trial participants interviewed for the evaluation the decision

to join was not a difficult decision. However, there were a few participants who were reserved

and cautious about the trial (as they were anxious about trying something new and were

apprehensive about the pressure and responsibility that would come with having control over

such a large sum of money). For these participants, speaking to others (such as their provider

and their family) for advice and guidance encouraged them to participate.

• Making decisions – Some trial participants immediately had ideas as to how they could

spend the Career Account money, while others needed more guidance and help to make

decisions. It was noted that family and friends influenced and gave guidance to some trial

participants with regards to how they should spend the funds, however providers were

7

observed as the biggest influencers and supporters. Importantly, participants indicated that

they themselves ultimately decided how to spend the money, with approval from their

provider.

• Changes and Impacts – The biggest change noted by both trial participants and DES

providers was that the vast majority of participants experienced improvements in levels of

confidence, self-esteem and motivation. Trial participants also suggested that their

relationship with their DES provider had improved as a result of the trial, while some also

commented that they had experienced better-quality relationships with their family due to the

trial.

• Employment/Education – Only a few of the trial participants interviewed as part of the

evaluation noted that they were in employment at the time of the follow up interviews. This

was consistent with the findings from the administrative data that showed very few trial

participants had achieved an employment outcome. However, analysis of what the trial

monies were spent on (as discussed above) shows that education and education related

expenses was the largest spend category of the Career Account, indicating that many

participants were using the trial as a stepping stone to future employment through gaining

additional qualifications or skills (as opposed to addressing more short-term barriers to

employment).

• Next steps – The majority of participants and DES providers felt that the trial had enabled

this cohort to address vocational and non-vocational barriers to employment. Trial participants

identified several benefits of the trial that would assist them in securing employment moving

forward, including: exercising power and responsibility and learning to make decisions;

improved confidence, self-esteem and motivation; feeling healthier mentally; a sense of

progression and moving forward; and improved personal relationships.

Providers experience with the trial:

• The DES providers interviewed as part of the evaluation were all very positive about the trial.

They saw that the model used for the trial had led to a number of benefits for the job seekers

including greater confidence and composure; improved communication skills and abilities; a

greater willingness of participants to attend scheduled meetings and stay in touch; and

improvements to mental health conditions (commonly including a more positive outlook about

the future).

• Providers also noted the trial had increased their personal capacity and the capacity of their

organisations in relation to a greater ability to assist job seekers with complex needs; staff

being invigorated and seeing improvements in relationships; having more leverage, incentive

and funds to help job seekers; and experiencing a rewarding feeling from achieving

meaningful progress towards employment for participants.

8

• In addition to the benefits noted above, providers also outlined a number of key challenges

that they faced during the course of the trial:

As noted above, some trial participants struggled to engage with the trial and many found

it difficult to spend all of the Career Account funds;

The perceived lack of detailed guidelines on how Career Account monies could be spent

was noted to result in some participants attempting to take advantage of the funds being

made available to them and providers having limited scope to provide an evidence-based

argument as to why such expenditure could not be approved – it should be noted that the

trial principles were intentionally kept broad, given the participant directed approach of the

trial, and that providers were made aware of this prior to the trial commencing;

The time allowed for the trial was considered too short, with participants under pressure to

finalise plans to spend the monies within an overly constrained timeframe;

A number of administrative irritants also posed challenges, including difficulties in

recording details of how monies were to be spent in the Job Plan.

• It was also evident from the qualitative research conducted with both trial participants and

DES providers that providers adopted a broad range of approaches with regards to the trial.

These different approaches included:

A very laid-back and hands off approach compared to a quite involved and directive

approach;

Categorically leaving it up to the trial participants to consider how to spend the Career

Account funds compared to providing strong guidance on how the money could be spent;

and

A willingness to find a way to approve goods and services compared with a more risk

adverse approach and a reluctance to approve suggestions for spending of Career

Account monies.

1.3. Conclusions

Based on the above findings, Colmar Brunton makes the following conclusions in relation to the degree to which the overall goal and the objectives of the Youth Mental Health trial were achieved.

Overall goal: Improved employment outcomes

It appears from the quantitative findings that trial participants did not perform as well as the control group with regards to employment outcomes – as the control group experienced higher rates of positive employment outcomes than the trial group.

This could potentially indicate that a participant-driven employment assistance model does not necessarily provide greater employment outcomes for young people with mental health issues in the employment services space. However, due to some limitations (i.e. the length of the trial, the amount of funds available to trial participants, the nature of how trial participants were selected etc.) the trial did not allow us to draw a definitive conclusion.

9

Objective 1: Assess the extent to which the trial participants have and exercise greater control and/or choice

It was clear from feedback from both trial participants and DES providers that participants were afforded a high degree of choice and control over how the Career Account funds were used during the trial period – as per the trial Guidelines. However, it was also evident that the level of self-agency experienced by individual participants varied widely across trial participants based on their own levels of comfort with the program and whether they had ideas on how they wanted to use the funds or not.

A number of trial participants were immediately engaged by the idea of being able to decide how to spend the Career Account money and were easily able to think of ways to spend the funds – for these participants the notion of greater control was well received. For others the process was not as fluid, with many trial participants reporting that they needed a substantial amount of guidance and help from their DES provider on how the funds could possibly be spent. For some this was due to simply not knowing how to spend the money or not being familiar with a process that gave them power and choice, while for others it was more about feelings of anxiety and nervousness that came from not previously having access to and control over such a large sum of money.

This view was supported by providers who very much saw their role as one of support and guidance. They noted that there were some trial participants who welcomed the idea with open arms, while there were others that needed more support and encouragement. The providers that were interviewed remarked that while they did try to give these trial participants help by asking questions and making suggestions they ultimately wanted to provide them with the opportunity to come up with ideas on their own (as per the intent of the trial program). The providers were very supportive of the fact that the trial gave participants the opportunity to influence and (largely) control the money and ultimately what steps would be taken in order to get them more job ready.

It was also apparent from the qualitative research that the nature of the trial had a positive effect on participants. The vast majority of trial participants noted feeling empowered by the trial and the fact that they could choose to spend the Career Account as desired (within reason and with approval). DES providers also suggested that encouraging trial participants to make decisions and giving them greater control was a key contributor to the positive changes that they observed in trial participants.

In summary, while the trial did empower many participants to exercise far greater control and choice in addressing what they perceived as the key barrier/s to employment (or progressing to a more ‘employment ready’ stage), there was a significant proportion who realistically did not have the capacity to make such decisions – especially in a context where limited advice or guidance is provided as a framework for such decisions.

Objective 2: Identify whether (and how) the flexibility of a Career Account benefits trial participants through employment outcomes

The ultimate goal of the trial was for more young job seekers with a mental health issue to find employment. In order to evaluate the trial on this basis, the Department provided Colmar Brunton with administrative data that included information on employment outcomes for both trial participants and a matching control group (e.g. a group with a similar profile to trial participants by factors including age, gender, location and types of mental health conditions).

As previously noted, the administrative data (as of 30 June 2016, the official end date of the trial) somewhat counter intuitively showed that the control group actually experienced greater rates of employment outcomes than the trial group – potentially indicating that the trial was not completely

10

successful in reaching its goal. However, it is critical to review such a result within the context of the trial itself. Rather than the shorter term focus of finding and sustaining employment under the current DES service provision model, participants in the trial were afforded both additional monies (in the form the of the $5,000 Career Account) and time (with monies able to be spent up until the end of the trial at 30 June 2016) to address the vocational and non-vocational barriers to employment that they themselves identified. A review of expenditure data showed that close to 70% of monies made available in the trial were spent on activities that spanned a significant proportion of the trial period (such as education/training courses, addressing transport/mobility barriers such as obtaining driver’s licences, and specialist mental health counselling or support). Both trial participants and DES providers noted that such activities – while not immediately resulting in an employment outcome – had significantly increased the ‘job readiness’ of participants and increased their appeal to potential employers into the future.

Overall, we believe that judging the benefit of the trial on employment outcomes for participants solely on a comparison of trial and control group employment outcomes at the end of the trial period does not sufficiently reflect the value the trial has delivered for both participants and government in terms of enhancing the capacity for future employment. To this end, we would suggest revisiting both cohorts 12 months after the trial conclusion as a more appropriate quantification of the trial’s ultimate impact on employment outcomes. We would also suggest that consideration is given to how participants are selected (i.e. ensure they are randomly selected) for any further attempts to evaluate programs such as this one. Participants for this trial were not randomly selected and as such it is possible that there was bias present when providers selected job seekers for participation. It is therefore possible that this potential bias could have impacted the results of the trial and the sustainability of employment outcomes.

Based on our observations in the qualitative parts of the evaluation, we believe the flexibility of the Career Account has delivered significant positive benefits well beyond the additional financial capacity the Career Account monies afforded trial participants. While the concept of self-agency and exercising control was daunting for some, the majority of participants appear to have felt empowered by their trial participation precisely because they were given a voice and power to pursue their own courses of action – often for the first time in their lives. This in turn has resulted in a number of key transformational outcomes including increased self-esteem and improved personal relationships for participants – key factors associated with sustaining good mental health. While such outcomes are not immediately measurable on Departmental systems, we believe they need to be considered when making an appropriate assessment of the value of the trial overall (given such outcomes are likely to support increased levels of workforce participation over the medium to longer term).

Objective 3: Identify any challenges or concerns raised by trial participants and DES providers

While both trial participants and DES providers were largely positive about the trial and the outcomes it has produced, there were several challenges identified over the life of the trial.

The first challenge identified related to the novelty of the program itself. The fact that the trial allowed for such individual freedom in decision making was difficult for both participants and their DES providers to understand and successfully engage with in a timely manner. The process of understanding the nature of the trial and seeking answers to the myriad of queries and questions of both participants and providers was commonly observed to take several weeks, which in turn impacted on the time left to develop the plan to spend the monies and execute these plans within the timeframe of the trial. Being a new approach, these delays were perhaps foreseeable but nonetheless still needed to be worked through – any second pilot or potential expansion of the program in future

11

would be unlikely to face such challenges to a similar degree (or will at least benefit from the lessons observed in this trial).

As noted above, a reasonably common challenge observed was the amount of funds provided per participant ($5,000). Many trial participants found the amount quite overwhelming – especially in a context where they are given limited direction as to how such funds could be used most effectively to address barriers to employment. Both providers and participants alike indicated that a smaller amount of money would still provide for the key positive transformational effects observed through these young people with mental health conditions being invited to exercise greater choice and control in their lives.

Other challenges observed included the perception that the time period for the trial was too short for the benefits of the program to be sufficiently realised by participants, providers not having appropriate systems or infrastructure in place to manage and track Career Account spending, and constraints within Departmental systems that did not allow for sufficient detail for recording details of expenditure for governance and accountability purposes. It was noted that the intent of building self-agency in determining how monies are to be spent can be undermined if in spending the actual monies the participants are subject to undue or unreasonable controls (e.g. needing the DES provider to physically accompany them when shopping for work clothes, or having to phone the provider at a point of sale for credit card details to be given over the phone).

In summary, a number of key challenges were experienced across the trial. However, given the nature of the trial, we do not believe these to be in excess of similar scenarios where a new concept – particularly one that is almost diametrically opposite to the current service model – is being trialled for the first time.

Objective 4: Identify key risks in potentially further expanding any subsequent model and mitigation measures to address these risks

Feedback from providers suggested there was a small minority of participants that sought to take advantage of their participation in the trial by using monies to purchase goods or services that were either not related to reducing barriers to employment or were at best only marginally related to this. Any future iteration of this program should therefore seek to mitigate this risk by providing clearer guidelines on what monies can and cannot be used for.

While there was no evidence of such behaviour observed during the evaluation, another risk is the potential conflict of interest a DES provider may face if seen to be encouraging the client to spend Career Account monies on services the DES provider itself offers (such as specific training courses). This risk is readily mitigated by potentially excluding any services provided by the DES provider as being eligible items for expenditure.

A further risk evident from the evaluation was the scope for significant underspend if the Career Account allocation remains at $5,000. This risk is able to be mitigated in one of two ways – either through greater guidance on how monies of this magnitude can be spent, or possibly through reducing the amount available per person in the program (which may also allow program participation to be afforded to more people).

12

1.4. Recommendations

The following recommendations have been created to assist with answering the final objective of the research – to draw out possible implications for design of support for employment outcomes for young people with mental health conditions in the future, which do not significantly increase the risk of inappropriate use of Government funding.

Recommendation 1: Provide greater communication to participants at the start of the program

One of the key objectives of the trial was to empower participants to feel a much higher degree of self-agency in addressing their specific barriers to employment and workforce participation due to their mental health condition. While this was achieved for most participants, some struggled to understand why they had been invited to participate in the trial or exactly what they were allowed to spend the funds on within the scope of the trial. This generally delayed the development of the Job Plan and finalisation of how the Career Account funds were to be spent, and was also a factor behind participants commonly not spending the full amount available.

To address this issue, we recommend that communication to participants and providers be reviewed to address fear and uncertainty around the program and to build confidence in the expenditure of the allocated funds per participant. Specifically, these improved communications would need to better articulate the following key points:

• A description of what the program entails;

• A brief explanation as to why the Department has decided to implement a new and different

model;

• How the system will be different for participants and for providers;

• Some case studies or examples of how the program has benefited similar others;

• All rules, regulations and guidelines that both participants and providers must follow; and

• What to do if more information is required.

While the communications to participants deliberately avoided examples of what the Career Account monies could be spent on, feedback from both participants and providers suggested that such a ‘blank slate’ approach actually made it more difficult for the participants to determine how they wanted to use the monies available. To this end, providers also suggested that communications from the Department to participants could include examples of how the money could potentially be spent or examples about how spending the money in a certain way has helped a job seeker to better manage their mental health issues and find employment.

The fact that many participants did not recall receiving the letter inviting them to participate is possibly reflective of this age group’s preferred communication channels. While the letter (with improvements as noted above) could still be used, we would recommend augmenting this with online information such as a dedicated Facebook page for the program and a YouTube video that explains the program in a clear and relatable manner (potentially leveraging previous participants telling their stories).

13

Recommendation 2: Improve the administrative arrangements for the program

If the program is to be expanded, we would recommend a number of key administrative improvements to both ensure funds are spent efficiently and effectively, while concurrently ensuring the integrity of the program is maintained.

• Improving the spend process – Providers noted that the trial Guidelines did not provide a

great amount of detail on the process for how Career Account funds could or should have

been processed. For some of the DES providers interviewed this ambiguity was a challenge

as they felt that they did not fully understand how the money could be distributed and

processed. It was suggested that if a similar program were to be delivered in the future that

the Guidelines should provide more clarity and information on this area. Another possible

solution that was raised by a provider would be to provider DES providers a credit card with

the dollar amount loaded on it for each participant. This card would still be kept in the

possession of the provider; however it would afford each participant a feeling of using their

own money and would make it easier for providers to make purchases for participants. It was

also noted that would provide a good mechanism for record keeping. There was also a

suggestion around the process including a “plan creation phase”, meaning that the participant

and provider would create a plan in the beginning for possible ways the funds would be spent.

This would then be sent to the Department for endorsement and approval. Once the plan was

approved it would make it easier for participants and providers to have a good idea on what

the money could be used for.

• For this cohort, ensuring at least a proportion of funds are spent on optimising the

mental health of participants – A number of the DES providers interviewed also suggested

that if a similar program is to be implemented in future that a key component should be on

improving the mental health of participants. While these providers were very positive about

the program increasing the self-agency of participants and the resultant positive

transformational effects this brings, having a proportion of the funds needing to be spent on

mental health improvement would ensure this potential barrier to employment is being

proactively addressed.

• The information given to DES providers – Many of the providers interviewed mentioned

that while they appreciated the high degree of flexibility around how the funds could be spent,

this aspect of the trial also presented difficulties with regards to them determining what they

could suggest to trial participants and what they could ultimately approve. If the Department

employs a similar model in the future it would be beneficial to provide both participants and

DES providers with greater guidance and information on how best to spend the money, while

still allowing for flexibility and tailoring spend to individual needs and preferences.

• The recording of details in the system – DES providers were required to “complete

quarterly updates of the trial participant’s Job Plan as outlined in the Job Plan Guidelines”. It

14

was noted by some providers that in practise this was difficult to do, as the current set up for

the Job Plan did not allow for the flexibility that was required for the trial. They noted that it

would have likely been too costly to make such a systems change for a trial, but that such a

change would be warranted if the program was expanded into the future.

Recommendation 3: Running the program over a longer period of time

Several DES providers interviewed for the evaluation felt that the timeframe to spend the money was not long enough. Similarly, some trial participants also noted that the time period for spending the Career Account funds felt quite rushed and that a longer time period to spend the money could have been beneficial.

As such, we would recommend that should a similar model be used in the future the timeframe should be extended beyond 9-12 months, especially considering the complex situations commonly faced by this cohort. Some providers suggested that the ideal situation would be for the funds to be available for the “DES lifetime” of participants or until it runs out (whichever comes first). While such an approach would certainly alleviate these time pressures, we acknowledge such a model may also reduce the incentive regarding achievement of employment outcomes.

Recommendation 4: Reviewing the funding for future programs

It was clear from the qualitative research with both trial participants and DES providers that both parties felt that the key driver of success for the trial was the fact that it was participant-driven – and it was not necessarily about the amount of money. It was noted by both trial participants and DES providers that $5,000 was potentially too much money for a job seeker in this cohort (i.e. a youth with complex mental health needs). This was somewhat supported by the administrative data that showed the average spend to be $2,540 amongst the trial participants. However it should also be noted that for the trial participants on a 13 week Pathway Outcome, their spend was on average approximately $1,000 higher than the average for all trial participants ($3,552).

Many trial participants suggested that they would not have been disadvantaged by having access to less money, with a number of them also noting that fewer funds per person could suggest that more people would have access to the assistance.

We suggest that the model of providing funds per participants be reviewed for any future programs of this nature. It could be appropriate to move from a ‘one size fits all’ model to a tiered approach with funds potentially being somewhere between $1,500 and $5,000 – however this would need to be looked at in conjunction with providing additional guidance and communication (as per recommendation 1), as it could be hypothesised that with additional guidance from the Department and providers that the average spend could have been potentially higher.

15

2. Introduction

Colmar Brunton Social Research (Colmar Brunton) was engaged by the Department of Social Services (the Department) to conduct an evaluation of the Disability Employment Services (DES) Youth Mental Health trial. This report presents the findings of this research.

2.1. Background

The Department of Social Services (the Department) is responsible for the Disability Employment Services (DES) program, helping job seekers with a disability to gain and keep employment1. Through DES, job seekers with a disability, injury or health condition may be able to receive assistance to prepare for, find and keep a job. The Department - through the DES program - runs a Youth Mental Health Initiative which is a part of the Government’s Youth Engagement Strategy, as announced in the 2015-16 Budget2. The initiative was designed to assist young people with a mental illness who are disengaged from education or employment to transition to work and become less reliant on long-term income support through a range of targeted support measures.

The Department understands that young Australians need the right assistance and encouragement to learn new skills, become job ready, get a job, and stay in a job. This is particularly true for young Australians experiencing a mental illness, who are more likely to be unemployed than those who do not report a mental illness3. According to the Australian Bureau of Statistics (ABS) in 2007 26% of people aged 16-24 reported experiencing some form of mental illness4.

In early 2015, then Assistant Minister for Social Services Senator the Hon Mitch Fifield emphasised the need to increase workforce participation among people with a disability in an address to the Disability Employment Australia Leaders’ Forum. In the same address, the Minister also flagged the need for an improved and innovative support system for job seekers with a disability ready for 2018 when current service provider contracts will be reviewed.

2.1.1. The Youth Mental Health trial

The DES Youth Mental Health trial was a one-year trial (run from July 2015 to June 2016) designed to test an individually tailored - and participant-driven - employment support model, the results of which will inform the design of the policy framework for employment services beyond 2018. The trial operated in Employment Services Areas in Queensland, Victoria and Western Australia. In the trial, up to 200 participants aged 24 years and under with a mental illness were provided access to $5,000

1https://www.dss.gov.au/our-responsibilities/disability-and-carers/programmes-services/disability-employment-services , accessed 25 September 20152http://budget.gov.au/2015-16/content/glossy/sml_bus/html/sml_bus-14.htm , accessed 25 September 20153http://www.abs.gov.au/ausstats/[email protected]/Latestproducts/4159.0Main %20Features12014?opendocument&tabname=Summary&prodno=4159.0&issue=2014&num=&view =, accessed 29 September 2015.4http://www.abs.gov.au/ausstats/[email protected]/Latestproducts/4326.0Media%20Release12007? opendocument&tabname=Summary&prodno=4326.0&issue=2007&num=&view, accessed 29 September 2015

16

http://www.abs.gov.au/ausstats/[email protected]/Latestproducts/4326.0Media%20Release12007?opendocument&tabname=Summary&prodno=4326.0&issue=2007&num=&view


http://www.abs.gov.au/ausstats/[email protected]/Latestproducts/4159.0Main%20Features12014?opendocument&tabname=Summary&prodno=4159.0&issue=2014&num=&view



http://budget.gov.au/2015-16/content/glossy/sml_bus/html/sml_bus-14.htm

https://www.dss.gov.au/our-responsibilities/disability-and-carers/programmes-services/disability-employment-services

of funding through a ‘Career Account’ administered by their local DES provider. Trial participants were afforded a high degree of choice in how the funds were spent as long as the intent was to overcome vocational and non-vocational barriers and meeting employment goals.

An evaluation was required to measure the effectiveness of such a model, including how participants used the funds and how well the model assisted young people with mental illness overcome barriers to their vocational and non-vocational goals compared to standard approaches.

2.2. Evaluation objectives

The specific objectives of the evaluation were to:

• Assess the extent to which the trial participants have and exercise greater control and/or

choice;

• Identify whether (and how) the flexibility of a Career Account benefits trial participants through

employment outcomes;

• Identify any challenges or concerns raised by trial participants and DES providers;

• Identify key risks in potentially further expanding any subsequent model and mitigation

measures to address these risks; and

• Draw out possible implications for design of support for employment outcomes for young

people with mental health conditions in the future, which do not significantly increase the risk

of inappropriate use of Government funding.

2.3. Methodology in brief

The evaluation of the DES Youth Mental Health trial was split into two key research areas that each contained multiple phases:

• Qualitative research with trial participants and DES providers involved in the trial; and

• Quantitative analysis of Departmental administrative data.

The final methodology used for this research project was chosen based on the Departments original desired approach and suggested inclusions (as outlined in the RFQ provided to market), as well as the additional detail given to Colmar Brunton during a subsequent briefing on the background and context to the project at a meeting on 20 October 2015. The final approach agreed to and used for the evaluation is briefly discussed below.

2.3.1. Project planning and scoping

The initial project planning and scoping meeting was conducted on 3 November 2015. The meeting allowed for finalisation of the proposed methodology, creation of a program logic framework (discussed further below) and agreement on the key research questions that needed to be answered through the project.

17

All subsequent phases of the study were then undertaken within the context of this agreed project plan framework, including the ethics application and the development of all research instruments (i.e. interview guides).

2.3.2. Program logic framework

In order to properly evaluate the program, Colmar Brunton recommended conducting a program logic workshop, which was completed as part of the initial planning and scoping meeting held on 3 November 2015. Figure 1 below outlines the program logic framework that was agreed upon by Colmar Brunton and the Department.

Figure 1: Program logic framework

2.3.3. Ethics approval

Colmar Brunton worked with the Department to secure ethics clearance for the project through Bellberry Limited. Bellberry Limited is a national, private not-for-profit organisation providing streamlined scientific and ethical review of human research projects across Australia.

The final agreed specifications that received formal ethics clearance are included at Appendix B. Final ethics approval was granted on 4 January 2016.

2.3.4. Baseline data analysis (test vs. control)

The Department supplied Colmar Brunton with administrative data to be used to examine employment outcomes for both trial participants and a control sample of job seekers (i.e. a matched sample of non-participants).

18

The initial data sets for both trial participants and the control group contained a variety of details per jobseeker, including but not limited to the following data items:

• Age;

• Gender;

• Disability/Vulnerability;

• Commencement date;

• Length of unemployment;

• Future work capacity with intervention; and

• JSCI score.

This data was used by Colmar Brunton in terms of both familiarising ourselves with the data set and with regards to defining the control group to be used for the final analysis.

Creating the control group

Being an observational study, a key part of this research involved the construction of a comparable control group. In recent times, matching has been seen as an increasingly popular methodology for assigning robust causal inference in a range of fields including statistics, medicine, econometrics and political science.

The broad concept of matching is to match as closely as possible each individual in the trial cell (i.e. the trial participants) with an equivalent individual from a pool of control subjects. This process ensures that a control group can be considered more closely comparable across certain characteristics that may confound causal inference (for example, for each Male aged 18 in the trial we would like to assign another Male aged 18 to our control group).

Each trial participant was matched to two possible subjects to act as a control group using a methodology based on Mahalanobis distance optimised to achieve a balance between key variables (GenMatch)56. Each participant was matched on the basis of:

• Age;

• Gender;

• Funding level;

• Months unemployed;

• Education level;

• Aboriginal, Torres Strait Islander (ATSI) status;

• Culturally and linguistically diverse (CALD) status;

• Homeless statues;

5 Diamond, Alexis and Jasjeet S. Sekhon. 2013. “Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies.” Review of Economics and Statistics. 95 (3): 932–9456 Jasjeet S. Sekhon. 2011. “Matching: Multivariate Matching with Automated Balance Optimization in R” Journal of Statistical Software, May 2011, Volume 42, Issue 7

19

• Refugee status; and

• Ex offender status.

The pool of control subjects were based on geographic areas that had similar employment outcomes to the trial locations (with these being provided by the Department). Further exclusions were made on the basis of an employment outcome occurring before the beginning of the trial and trial participant who had drop out or been excluded.

Results and details of the trial to control groups can be found in Appendix G.

2.3.5. Qualitative fieldwork Phase 1

To better understand the administrative data, Colmar Brunton conducted a series of 1-hour, face to face depth interviews with both trial participants and DES providers participating in the trial. Colmar Brunton conducted a total of n=33 depth interviews with trial participants (n=12 Melbourne, n=11 Brisbane and n=10 Perth) and n=9 depth interviews with DES providers who have job seekers participating in the trial (n=3 Melbourne, n=3 Brisbane and n=3 Perth), between 9 and 18 February 2016.

2.3.6. Interim analysis and reporting

An interim summary report/presentation was provided to the Department after these initial interviews with trial participants and DES providers were complete. This report was provided in MS PowerPoint format and summarised the core findings from the research conducted up to that point in time and was not a full analysis of the situation. The report was not intended to be circulated as an output document to a wider audience but was purely for the informational use of the Department project team.


Having interviewed trial participants and DES providers during the early stages of their participation in the trial, Colmar Brunton re-engaged these people for a second round of depth interviews towards the end of the trial (May 2016). These interviewers were conducted by telephone between 11 May and 15 June 2016. Every effort was made to try and arrange a second interview for each of the trial participants interviewed in Phase 1, and in total n=23 follow up depth interviews were completed with trial participants (n=9 Melbourne, n=9 Brisbane and n=5 Perth).

In addition to interviewing trial participants, Colmar Brunton was able to arrange follow up depth interviews with all 9 DES providers interviewed in Phase 1 (n=3 Melbourne, n=3 Brisbane and n=3 Perth), which also occurred via telephone. As per the initial round of interviews, the follow up depth interviews were audio-recorded, with participant permission, to facilitate detailed qualitative analysis.

2.3.8. Quantitative analysis of administrative data (trial participants vs. control group)

In July 2016 the Department provided Colmar Brunton with updated administrative data sets for both the trial participants and the control group. This data was used to run a series of statistical tests to look at differences in the employment outcomes for both trial participants and the control group.

20

The Department also provided administrative data for each trial participant with regards to how the Career Account funds were spent during the course of the trial period. This information was provided to the Department by the DES providers and was listed at an individual participant level. This data was examined to determine average spend across trial participants and to take a deeper look at what the money was being spent on.

2.3.9. Final analysis and reporting

This final report presents the findings of all stages of the study, with the key findings and themes identified in this report reflecting a detailed analysis of all qualitative findings observed across all interviews conducted for the study, as well as the quantitative findings.

21

3. Findings

3.1. Employment, Outcomes and Success

Trial outline

The guidelines for the Youth Mental Health trial outline that the intent of the trial was to test a participant-driven employment assistance model whereby eligible job seekers with a mental illness aged 24 years and under would have control over funds for the purpose seeking and gaining employment.

The Employment Service Areas (ESAs) chosen for the trial were:

• Maroondah (Victoria);

• North Brisbane (Queensland); and

• North Metro (Western Australia).

These ESAs were selected due to their viable labour markets, to ensure employment opportunities were available for participants. In particular, these ESAs were chosen as there was a reasonable, or at least neutral, possibility of achieving employment outcomes for participants. The other two main factors for choosing these locations were to ensure that there was good coverage of states and territories, and relatively high caseloads of eligible DES participants.

Trial placements within these ESAs were allocated by the Department, with providers needing to opt-out by 31 August 2015 if they did not wish to participate. The DES providers that chose to participate in the trial then canvased the interest of eligible job seekers as per the placements they had been allocated by the Department.

According to the Guidelines distributed to DES providers, eligibility was determined by participants being required to;

• be aged 24 years old or under; and

• have a mental illness (either primary or secondary); and

• have commenced in the DES program and be in the Employment Assistance phase, but not

yet having achieved an Employment Outcome; and

• have already left school.

It was then the role of providers to explain to eligible participants how the trial and the Career Account would work – i.e. that trial participants would have choice and control over how to spend the Career Account funds. Once an eligible participant chose to take part in the trial the provider made a claim for payment from the Department. The Department also wrote to the trial participants to advise them that

22

their provider would distribute the funds and provide further explanation about how the Career Account could be used.

The role of the evaluation in terms of employment, outcomes and success

In order to properly evaluate the program, Colmar Brunton and the Department created a program logic framework for the trial. This framework identified a hierarchy of outcomes where specific actions and mechanisms are linked to the desired changes in behaviour, with the premise being that these changes would ultimately lead to the achievement of defined success and end-states. The program logic framework that was agreed upon is outlined in Figure 2 below.


As outlined in the logic model above, the ultimate goal of the trial was for more young job seekers with a mental health issue to find employment.

Administrative data supplied by the Department allowed for analysis between the trial participants and a control group. The control group was chosen by matching as closely as possible each trial participant with two equivalent individuals from the pool of control subjects. Each participant was matched on the basis of age, gender, funding level, months unemployed, education level, ATSI status, CALD status, Homeless status, Refugee status, ex offender status, mental health conditions and geographic areas with similar unemployment rates. This process ensured that the control group can be considered more closely comparable across certain characteristics that may confound causal inference.

The data in Figure 3 below demonstrates the differences between the trial participants and the control group for a range of factors, including:

23

• Full Outcomes (both 13 weeks and 26 weeks) – these are paid in recognition of a job

seeker achieving sustainable employment that equals or exceeds the minimum of their

Employment Benchmark hours, on average, each week for a 13 or 26 week period, or (for

certain job seekers) remaining in a qualifying education course. A Full Outcome occurs

where, for the duration of a 13 or 26 week period, an eligible participant remains in

employment, unsubsidised self-employment, an apprenticeship, a traineeship or a qualifying

education course.

• Pathway Outcomes – these recognise progress towards the achievement of sustainable

employment, such as through education or substantial part-time work. The requirements for

achieving a Pathway Outcome are less than those for a Full Outcome and recognise that the

job seeker is making progress towards achieving sustainable employment or education. The

payments associated with Pathway Outcomes are less than the payments for a Full Outcome.

• Job Placement Fees – these are payable when job seekers are placed in a job and their

hours worked over 10 consecutive working days equal or exceed the minimum number of

hours in their assessed work capacity bandwidth. DES providers may claim a Job Placement

Fee for placing a job seeker into a Job Placement, in which the participant achieves their Job

Placement hours within the required timeframe.

It is evident from the data below that Job Placement Fees were similar for the trial participants and the control group, however the trial participants appear to have underperformed on the Full Outcomes and Pathway Outcomes compared to the control group.

For trial participants only 6% had reached a 13 week Full Outcome at the conclusion of the trial, which was significantly lower than the control group, where 18% reached a 13 week Full Outcome. Similarly, trial participants did not perform as well as the control group with regards to 26 week Full Outcomes, with no trial participants reaching this milestone. Of the control group, 11% did reach a 26 week Full Outcome during the trial period. The data shows that the trial group performed similarly to the control group with regards to 13 week Pathway Outcomes (4% and 4% respectively) and 26 week Pathway Outcomes (0% and 1% respectively).

24

Figure 3: Outcomes for trial participants and the control group

Trial participants Control group

N= 184* 368

DES 13wk Full Outcome* 6% 18%

DES 13wk Pathway Outcome 4% 4%

DES 26wk Full Outcome 0% 11%

DES 26wk Pathway Outcome 0% 1%

DES-DMS Job Placement Fee 8% 9%

DES-ESS Funding Level 1 Job Placement Fee 8% 8%

DES-ESS Funding Level 2 Job Placement Fee 5% 4%

*Please note that n=5 trial participants are missing from these calculations due to missing data – the total sample size for the trial participant group should be n=189.

Note 1: A Power Analysis was also run on the DES 13wk Full Outcome to determine that there was an appropriate number of subjects within the trial. The results of the Power Analysis can be found in Appendix H of this report.

Note 2: The arrows in the table above represent a significant difference between the groups at the 95% confidence level, which is standard for social research. It indicates that the difference or relationship would be likely to be observed again if the observations were made again under the same conditions.

The qualitative research conducted with trial participants and DES providers suggests that one possible explanation for this difference could be the nature of the trial itself and the fact that participants viewed it as something that would assist them for the long-term. The trial gave participants a defined period of time in which they could spend the Career Account funds however they desired (within reason) for the purpose of assisting them in overcoming any vocational or non-vocational barriers, and to further their employment goals. As such, many trial participants focused on areas such as education, mental health and personal improvement (i.e. travel and transport), all of which take time and are discussed in greater detail within this report. As the control group were not participating in the trial it is assumed that their attention would have been more focused on immediate employment (as per the usual arrangement). This focus on immediate employment could be a contributing factor as to why a higher proportion of the control group had reached a Full Outcome or Pathway Outcome at the conclusion of the trial period.

Additionally, as trial participants were not randomly selected, it is possible that providers selected job seekers to participate in the trial that they felt would be harder to place into employment. Unfortunately it is not possible to identify whether, or the extent to which, this occurred from the data available.

25

One area that was not able to be examined through the administrative data was the quality or potential longevity of the employment outcomes of both groups. If possible, it would be beneficial to review the outcome results of the two parties in twelve months’ time.

From the depth interviews conducted with DES providers it was evident that while providers agreed that finding employment was the fundamental goal of the trial, they also believed that such an outcome wasn’t necessarily achievable for all trial participants within the limited timeframe of the trial (with most participating in the trial for the 9 months from September 2015 to June 2016). DES providers suggested that in evaluating the success of the program, a number of other indicators highlight positive progress towards an employment outcome including:

• Engaging in formal study or education pathways that are directly related to future employment

opportunities (e.g. TAFE courses);

• Increased engagement with the DES provider (including attendance at appointments and

participating in activities that have been paid for through the Career Account);

• Completing volunteer work and/or work experience;

• Engaging with mental health, counselling or psychiatric services; and

• Noticeable improvements to mental health as a consequence of participating in the trial,

including increased confidence and communication skills.

“The job seeker has gained several jobs since the commencement of the trial (including two Job Placement Fees claimed), however has been unable to maintain consistent work at his benchmark. Whilst the employers have not disclosed the specifics of the reasoning behind the lack of call-backs for more work, his Employment Consultant has identified several barriers to ongoing work such as motivational, memory and communication issues. The job seeker has therefore been referred to

opportunities for improving his communication and mental health-based barriers; however the job seeker declined to participate; rather, within the context of the trial funds the job seeker preferred to focus on work outcomes and to participate in job search activities. The job seeker has been assisted with job search help, career path mentoring, interview preparation (including the provision of clothing

and haircut), and advocacy to businesses in person, over the phone and online.” (Comment from administrative data)

Challenges faced by those that did not reach an outcome

There were a number of factors that contributed to why not all trial participants were employed at the time of the follow up interviews and as such were not in Stage three of the employment journey (Figure 4 below).

“I don’t feel more job ready right now but it (the trial) has definitely set up a foundation, so in the long term it's a big positive, because I'm planning to do a course.” (Male, Melbourne)

26

Figure 4: Employment journey map

For some of those not yet employed at the conclusion of the trial the main reason they gave was that they were not mentally ready for employment and were still focusing on addressing their mental health issues – meaning they were still in Stage one. For several trial participants this meant that they needed or were still undertaking specialist counselling or assistance and/or needed medication to assist with addressing their mental health needs. This was supported by the majority of providers as they mentioned that they had witnessed varying levels of engagement with the trial and varying degrees of confidence. For instance, some trial participants were very clear about how they wanted to use the funds, however for others it was a lot more difficult. In addition to this, for those that found it more difficult to engage with the trial it was taking them longer to think about what type of job/career they wanted, as they felt the trial now afforded them the opportunity to really consider the possibilities and not just focus on the short-term.

“For example we have one job seeker who hasn't accessed it much, this person's health is at a stage where they are not ready to get vocational assistance, but also not ready to get health assistance.

The job seeker needs to be ready.” (Provider, Melbourne)

“The job seeker is suffering from ongoing depression which represents a major barrier to gaining employment; in his own words he has difficulty getting out of the house. He has stated that his

psychologist has advised that he is unfit for work at this point as his mental health condition has recently deteriorated. The job seeker is in the process of applying for an exemption from job search

activities based on medical reasons.” (Comment from administrative data)

Many other trial participants had progressed to Stage two and as such were completing education courses and qualifications, which for many meant that their focus was on their studies and not on job searching (at least for the short-term).

“I should definitely be able to get employment once it (TAFE course) is over.” (Male, Perth)

27

3.2. The Career Account

The guidelines for the trial that were distributed to DES providers participating in the trial outlined how trial participants and DES providers were to use and interact with the Career Account.

They stated that trial participants would have a high degree of choice and control over how the Career Account funds would be used, with providers helping to identify appropriate supports consistent with the Guiding Principles (as outlined in the same guideline document and explained in detail below). Trial participants and providers were to then agree on appropriate supports, which were to be recorded in the participant’s Job Plan.

To encourage innovation and maximise flexibility in service delivery, in respect of trial participants only and only for the trial period, providers were not be required to:

• comply with clause 89.1 (minimum number of Contacts) of the Deed; or

• complete quarterly updates of the trial participant’s Job Plan as outlined in the Job Plan

Guidelines.

Providers were asked to use their discretion with determining how many times they should have contact with each trial participant during the course of the trial and how frequently the participant’s Job Plan should be updated. With the caveat that Job Plans were to be current and remain relevant at all times. The guidelines also stipulated that trial participants with Mutual Obligation Requirements were to be supported to meet their requirements.

For the trial to provide optimum flexibility the Career Account used a principles-based approach, known for this trial as being the Guiding Principles. These principles that providers were required to adhere to when making purchases using Career Account funds were as follows:

1. The expenditure of funds was to align with a trial participant’s future employment goals, or

show a pathway towards identifying a trial participant’s employment goals.

2. Funds were not to be paid directly to the trial participant. Only in the exception of a paid work

experience placement could DES providers use the Career Account funds to directly pay

participants for their placement. This payment was to occur following the completion of the

placement, and it was expected that providers retained evidence of all spent funds for

auditing purposes.

3. Funds were to be used to purchase goods or services that would address a trial participant’s

vocational or non-vocational barriers to employment, including integrating with mental health

treatment services. Goods or services that could not be reasonably expected to help improve

a trial participant’s employment prospects were not to be funded through the Career Account.

4. The expenditure of funds was to be consistent with the Deed and all Commonwealth and

State or Territory laws (including, for example, any relevant work health and safety

legislation).

5. The expenditure of funds was to represent value for money. Providers were required to

confirm that no other source of funding was available to fund the goods or services before

expending Career Account funds. Providers were also advised to actively seek the best

28

possible rate for goods and services, but balance these against the specific needs of the trial

participant (for example, the timeliness of supports or interventions).

6. It was necessary that the expenditure of funds and any goods or services purchased was

defensible and capable of withstanding a reasonable level of public scrutiny so as not to bring

DES into disrepute.

In essence, providers were required to ensure that all purchases made through the Career Account met the requirements under the Deed and the trial Guidelines. If DES Providers were unsure about the appropriateness of the purchase of goods and services through the Career Account, they were able to contact the Department through the Youth Mental Health trial mailbox to make an enquiry.

Initial feelings

Providers were the first point of information on the trial, as they explained to eligible participants how the trial would work. Once an eligible participant chose to join the trial the Department also wrote to them to provide further explanation about the trial.

The letter briefly outlined the purpose of the trial and that the trial participant would have access to a pool of funding of up to $5,000, known as a Career Account. It also explained that the trial participant would have the opportunity to make choices about how the funds would be spent in order to assist them in overcoming any vocational or non-vocational barriers, and to further their employment goals.

There was also a section in the letter that described what trial participants and their DES provider were required to do, which covered the following areas:

• Trial participants and their DES provider were to discuss and agree on how the funds were to

be spent.

• The trial participants DES provider was required to assist them with identifying appropriate

goods and services that could be purchased through the Career Account.

• Examples of goods and services that could be accessed through the Career Account were

given (counselling, training, transport, work experience, or clothing and equipment that would

help them to overcome barriers to employment).

• The three guiding principles that were to be followed by trial participants and their DES

provider: 1) The expenditure of funds was to align with the trial participants future employment

goals, or show a pathway towards identifying their employment goals; 2) Funds were not be

paid directly to trial participants, however they would be available for trial participants to use

with support from their DES provider who was required to manage the funds; 3) Funds were

to be used to purchase goods or services that would address trial participants vocational or

non-vocational barriers to employment, including integrating with mental health treatment

services. Goods or services that could not be reasonably expected to help improve their

employment prospects were not to be funded from the Career Account.

• Trial participants were informed about trial-related evaluation activities and advised that

participation was voluntary, but the Department would greatly appreciate feedback. Trial

29

participants were also notified that the Department may contact them during the trial period, to

ensure that they were satisfied with how the funds were being used.

The last section of the letter advised trial participants that if they had any questions or concerns they should first discuss these with their DES provider and then if needed they could contact the Department.

There was also an example attached to the letter, which was the same example that was given to providers in the Guidelines document.

From the qualitative research conducted with trial participants it was evident that the majority had not read the letter provided by the Department and were not aware that they could contact the Department directly for more information. If a program such as this is to be made available in the future it could be beneficial for direct communication from the Department to be repeated in order to receive more cut through.

It was also apparent from the qualitative research that some trial participants felt anxious about or daunted by the large amount of money, that they essentially had control over. For some of these trial participants, these feelings of trepidation came from not having ever had access to and control over such a large sum of money, and for others it was more about the concept being so different to their usual situation and difficult for them to fully understand and appreciate in the beginning. There was also some nervousness around making the most of the opportunity as a few trial participants considered the trial “their one big chance”. This self-imposed pressure meant that these few trial participants felt that the stakes were so high that they went past thinking carefully about how to best use the money and actually became scared to use it.

“Client was unable to decide on where and how to spend funds.” (Comment from

administrative data)

These feelings of consternation were principally eased by their providers and family, in so far as they helped the trial participants to think about how they might spend the money. They did this by providing trial participants with ideas and suggestions that they thought might assist with overcoming vocational or non-vocational barriers to employment and by asking trial participants questions that would guide them in their thinking about how to spend the funds.

“It is just such a large sum of money. It did seem a little excessive and it was a bit daunting but my provider helped me with that and I know that I don’t have to spend all of the money.” (Male, Brisbane)

How the funds were spent

Administrative data provided by the Department contained information on all of the expenditure made by trial participants – as recorded by their DES provider. The data outlined how much was spent by each trial participant and what the funds were spent on.

For the 189 trial participants included in the data file, the total spend of funds from the Career Account was $480,022, with trial participants spending an average of $2,540 – which was approximately half of what each participant had access to ($5,000).

When looking at total average spend split by whether trial participants had reached an outcome at the conclusion of the trial or not, the results were fairly similar. Figure 5 below shows that the average spend for trial participants that had reached a 13 week Full Outcome ($2,447) was similar to the

30

average spend of those that had not reached this milestone ($2,593), with the difference being just $146. The difference in average spend for a 13 week Pathway Outcome was greater, at $1,007, with trial participants that reached this milestone spending on average $3,552 and those not having reached this milestone spending an average of $2,545. It should be noted that the number of trial participants that had reached the 13 week Full Outcome and 13 week Pathway Outcome were very small and so these results should be viewed with caution.

Figure 5: Average spend by 13 week Outcomes

Reached Outcome Did NOT reach Outcome

DES 13wk Full Outcome$2,447(n=11)

$2,593(n=173)

DES 13wk Pathway Outcome$3,552(n=7)

$2,545(n=177)

The administrative data showed that the Career Account funds were spent on a variety of goods and services. To facilitate with analysis twelve categories were created, with each cost type that was recorded by the DES providers being coded into one of the following categories:

• Education and education related expenses (including TAFE courses, First Aid courses,

textbooks and laptop computers etc.)

“The job seeker had career goals of lock smithing - successfully completed the course.”

(Comment from administrative data)

• Travel and transport assistance (including driving lessons, public transport card top up,

bicycles and related accessories, car registration, car purchase etc.)

“The job seeker required suitable transport to assist in getting to and from training and also for

future employment purposes, as bakery work is generally shift work.” (Comment from

administrative data)

• Mental health assistance (including counselling, psychiatry sessions, anxiety workshops

and psychological consultancy sessions etc.)

“Specialist Intervention designed to motivate and challenge negative mindsets.” (Comment

from administrative data)

31

• Technology and office supplies (including laptop computers, printers, computer

accessories, paper, notepads and software packages etc.)

“Job seeker had no means of at-home job search or development, so after consultation

appropriate technology was sourced.” (Comment from administrative data)

• Physical assistance (including gym memberships, yoga classes and physiotherapy

appointments etc.)

• Clothing, accessories and personal grooming (including clothing for interviews, job

specific clothing, haircut, make-up and glasses etc.)

“Job seeker had limited clothing (professional and personal) - purchased suit with self-care

items (deodorant, soap, shaving items).” (Comment from administrative data)

• General career and interview assistance (including social casework sessions, career

counselling and resume and interview coaching etc.)

• Licenses, checks and specific work related accessories (including police checks, security

license, forklift license and tool kits etc.)

“Job seeker required security licence to work in lock smithing.” (Comment from administrative

data)

• Work experience (i.e. paid work experience placement)

• Telephone and internet assistance (including purchase of a mobile telephones, credit top

up, internet top up and phone cards)

“The job seeker had no way of being contacted; phone purchased to allow contact with

treating team, DES provider and vocational services.” (Comment from administrative data)

• Workshop/Training room hire and catering

• Other (including advertising for a small business, Coles vouchers, proof of age cards and

other miscellaneous goods and services)

32

“The job seeker required (a Proof of Age (18+) Card) as he had no photo ID and is not able to

seek (a driving licence) at this stage due to illness.” (Comment from administrative data)

Figure 6 below demonstrates how the total $480,022 was divided by the 12 category types. Education and education related expenses ($177,178) accounted for just over a third (37%) of total spend, and was a strong spend type across all disability types. Spend on travel and transport accounted for 16% ($75,764) of total spend, with spend on mental health assistance similar at 15% ($71,419).

Figure 6: Spend categories

$177,178

$75,764

$71,419

$52,143

$29,544

$21,344

$18,139$12,849 $6,514 $6,474 $829 $7,825

Education and education related expenses

Travel and transport assistance

Mental health assistance

Technology and office supplies

Physical assistance

Clothing, accessories and personal grooming

General career and interview assistance

Licenses, checks and specific work related ac-cessories

Work experience

Telephone and internet assistance

Workshop/Training room hire and catering

Other

Trial participants interviewed for the evaluation also discussed spending the Career Account funds on these category types, demonstrating consistency between the qualitative phases and the findings from the administrative data.

“(The benefit has been) being able to do things that I felt were important for my education but I wasn't able to afford.” (Female, Melbourne)

“I got a bike and it was awesome, I can whip around easier… it has been a win-win... it means I can save my money for other things.” (Male, Perth)

“(The biggest benefit has been) having the driving lessons and the fact that I feel that if I wanted to I could go for my driving licence tomorrow. I could ask if the trial could pay for the test, because

obviously having a licence would greatly benefit me in being able to get a job.” (Male, Brisbane)

33

There were some interesting observations noted during the course of interrogating the administrative data that related to how the Career Account funds were spent, these were as follows:

• The 11 trial participants that reached a 13 week Full Outcome generally spent their Career

Account funds on the same areas as the total trial group. Education and education related

expenses was the top spend category (41% of spend compared to 37% for all trial

participants), mental health assistance was the second (19% of spend compared to 15% for

all trial participants) and travel and transport assistance was the third highest spend (15% of

spend compared to 16% for all trial participants).

• There were 13 trial participants that spent the full Career Account and a further 15 that spent

between $4,900 and $5,000. The ways in which these trial participants spent the Career

Account funds were similar to the total, however with a few key differences. Education and

education related expenses was the top spend category (39% of spend compared to 37% for

all trial participants), however spend was far greater for travel and transport assistance (26%

compared to 16%) and was slightly lower for mental health assistance (12% compared to

15%).

• There were 20 trial participants that recorded not spending any of the Career Account funds.

For 11 of these participants it was noted that this was because they had exited the trial, while

for the remaining 9 the reasons varied (or were not recorded). There were mentions of long-

term medical suspension, not being able to decide how to spend the funds (despite

encouragement from the provider), not requiring funds due to being linked to free community

based services and being admitted to an acute mental health unit.

“Client was encouraged to use the funds and DES was assisting with ideas and utilising the account. Client was going to but due to health did not follow through with usage of his career

account.” (Comment from administrative data)

• One provider had 5 job seekers participating in the trial, with all 5 spending the full $5,000

available to them through the Career Account. This was unique in that no other provider

participating in the trial had the same outcome for all of their participants. The spend patterns

for these 5 trial participants were largely in line with the spend patterns of all 28 participants

that spent at least $4,900 of their Career Account funds.

• There was one provider that appeared to have organised one or more workshops for their 11

trial participants with the costs for the room hire and catering being split equally amongst

them – with the total cost for these expenses being just over $800.

34

Challenges with spending the money

Interviews with both trial participants and DES providers with clients participating in the trial indicated there were a number of challenges faced in terms of the actual expenditure of the Career Account funds.

As previously discussed, the process for using the Career Account funds was that trial participants and DES providers were to discuss and agree on how the funds were to be spent, and where necessary providers were to assist trial participants with identifying appropriate ways to use the Career Account. This process was distinctly different from the usual model that trial participants were used to (where DES providers determine the best mix of supports for their job seekers), and for some trial participants it took them a while to become comfortable with this new way of thinking. Some participants mentioned that despite encouragement from their provider and family they still found it difficult to think of ways to spend the Career Account money. This was a shock to some providers as they expected that trial participants would not struggle so much to think about how to spend the money.

“If we had a new participant today, we would do it a little differently. We would be able to use our learnings from the first 6 months of the trial and know that participants would come in and not know what they want to do... That was a real shock for us. We thought participants would be like “yep, I

want to do this, this, this and this” and start it straight away, whereas we were really having to tease out of them what they wanted to do, as well as ensure there was value for money.” (Provider,

Brisbane)

The majority of providers also mentioned that while they did try to give trial participants assistance by asking them questions and making suggestions they were hesitant to do too much too soon as they really wanted to give the participants a chance to come up with ideas on their own – which was clearly aligned with the strategic intent of the trial. The challenge for providers was finding the right balance between affording job seekers the autonomy to make decisions (as per the trials intent) and providing guidance and suggestions to ensure the opportunity was not wasted.

“Clients can at first be a bit confused or overwhelmed but that is not necessarily a negative because we put a plan together and get through it.” (Provider, Melbourne)

There was an example given to both trial participants and DES providers early in the trial, however, it was suggested by both parties that further case studies and/or a list of ideas would be very helpful for participants, especially in the early stages of thinking about how to spend the money. From analysing the administrative data and speaking with trial participants and DES providers it is evident that further information on education, travel and transport and mental health assistance would be of the greatest benefit.

“We were surprised at how hard it was to spend the money. It might have been better if we had a list of what we could spend the money on.” (Provider, Perth)

From an administrative perspective, a number of trial participants and providers made mention of logistical challenges associated with making payments using Career Account monies. Under the governance arrangements of the program, participants wanting to spend Career Account funds needed to do so via their DES provider, who would physically make the payments on the participant’s behalf (as opposed to the participant having direct freedom to spend the monies independently). While this approach was important for the governance, integrity and ultimate accountability of the program, participants themselves noted this process was sometimes a barrier to spending the funds,

35

with some reluctant to ask DES providers to pay for things due to feelings of shame and embarrassment, especially in circumstances where the participant may be in a shop and need to have the person serving them to contact the DES provider to facilitate the transaction. From the perspective of the DES providers this also posed as a challenge, especially where there were limits on credit cards within the organisation. One possible solution that was noted by a provider would be to give DES providers a credit card with the $5,000 on it for each participant. This card would still be kept in the possession of the provider; however it would afford each participant a feeling of using their own money and would make it easier for providers to make purchases for participants. It would also be a good mechanism for record keeping.

“Something that we probably need to work on is building our accounts and relationships with businesses. We were banking on a lot of businesses taking credit cards over the phone, which would have been great, but probably about 90% of businesses don't do that anymore. So because of that we

had to send executives to the shops with job seekers, as they were the only ones with high enough credit card limits.” (Provider, Brisbane)

In terms of other barriers to full expenditure of the $5,000 Career Account balance, some participants noted that the scale of the cost of things that they wanted to purchase was also sometimes an issue. For many participants, they had no previous experience with spending such large amounts of money (e.g. a TAFE course costing thousands of dollars). For a number of the trial participants interviewed as part of the evaluation they really were daunted by the thought of not using the money appropriately or spending too much on one good or service. This was possibly amplified for trial participants experiencing anxiety – which from examining the administrative data appeared to be quite common.

“I felt a bit guilty using Government money to buy things for myself… but it was nothing compared to the feeling of knowing that there is one thing I no longer have to struggle to pay for or to have to ask a

friend for help with. I don’t want to have to ask a friend for help, and I don’t want to have to use the Governments money either, but knowing that it was there was good.” (Male, Brisbane)

Across the interviews with both trial participants and DES providers it was evident that there was a significant degree of variation in how providers interpreted the program guidelines in terms of how the Career Account monies could be spent. For example, in one instance a trial participant wanted to use some of the Career Account funds to assist with purchasing a car, however this request was denied by the DES provider, yet in another instance a provider did assist a trial participant with purchasing a car. This highlights the different interpretations of the guidelines and the different ways providers were willing to assist participants. Information provided by the Department suggested that over 400 emails were sent by providers during the course of the trial, with the majority of the queries relating to the appropriateness of Career Account expenditure (i.e. seeking approval of particular items).

Overall benefits

The main benefit of the Career Account as seen by trial participants was the ability to do things they otherwise might not be able to do – as one participant described it – the Career Account has “opened up a lot of opportunities” (Male, Brisbane). This was a view shared by many of the providers interviewed as they felt that through the Career Account trial participants had opportunities to access goods and services that they would not have otherwise been able to access.

“Now you've got a lot more options you can plan a lot easier. There are a lot of people who can't get a job and feel hopeless; this program helps you move forward and plan.” (Male, Melbourne)

36

“I can say this with 100% certainty that I very much doubt that we would have been able to pay for that sort of stuff, if we didn't have this money. Or at the least we might not have been able to get him

that good a quality of stuff.” (Provider, Brisbane)

“There would have been courses with a small price tag and some referrals to specialists but not to this extent it would have been a much smaller scale.” (Provider, Melbourne)

Providers also strongly suggested that the way the money was being administered (i.e. the trial participants leading how the money was being spent) was playing a big part in the positive changes that they were seeing in trial participants (even more so than the amount of money). DES providers noted these benefits included greater confidence and composure, improved communication skills and abilities, and improved mental health. Providers mentioned that for some job seekers $500 or $1,000 would be enough to provide some practical assistance and elicit these benefits. It was suggested that these benefits being seen by providers were less about the amount of money and were more about the process being participant-driven.

“It’s helping out a lot and it’s good to spend the money in any way. Having the power to make my own decisions has been good, my motivation has increased a lot and learning to drive is good. Doing a

course has also given me something to look forward to. I don't feel that depressed anymore.” (Female, Perth)

Amount of funds

From the depth interviews with trial participants it was evident that most knew that they had access to $5,000, however very few got close to spending all of the Career Account funds – which was consistent with the administrative data that showed average spend to be $2,540. The reasons for not spending the full amount varied, however most centred around trial participants not knowing exactly what to spend the money on and not wanting to take advantage of the program by spending the money on things that were unnecessary.

“I could have spent more but I didn't know what to get, I was encouraged to spend by (provider) so I did.” (Male, Melbourne)

Many trial participants felt that they still would have benefited from the trial if there was less money to spend, with one trial participant in Brisbane noting that “any amount of money is going to be useful somehow”. At the time of the follow up interview this participant had only spent approximately $3,000 and did not plan to spend the remaining funds (at the time there was approximately 2 weeks of the trial remaining). A number of other trial participants also felt that less money still would have been useful, with many also mentioning that less money for themselves would have meant more people receiving assistance.

“It was maybe a bit too much, it may be better if it was available to more people, maybe $2,000 to $2,500 would have been better.” (Male, Melbourne)

“$3,000 would have been good. $2,000 would have been good, you could have done less with $2,000 but it is still a lot of help. With $5,000 it is a big number, it is a big scary number, whereas I think I

could have gotten a lot more done with $2,000, because it would have been less to fill.” (Male, Brisbane)

Some of the DES providers interviewed as part of the evaluation also suggested that $5,000 was potentially too much money per job seeker. They felt that for some of their trial participants $5,000

37

was overwhelming, however many also noted that it is difficult to say what amount would be ideal, as each job seeker is different and has different needs.

“I think it ($5,000) is an overkill… I think $3,000 would still be a lot of money… $5,000 is an unbelievable amount of money and it can be quite overwhelming… It also comes down to the fact that you don’t want to go out and buy them the fanciest of the fanciest clothes because that is not realistic

for them to upkeep, so you don’t want to set them up for failure.” (Provider, Brisbane)

“$5,000 is too much anyway. A lot could be achieved with half the funds.” (Provider, Perth)

The role of providers

As previously outlined, the DES providers role within this trial was to work with trial participants to identify appropriate goods and services that could be purchased through the Career Account. Providers were also to ensure that all purchases made through the Career Account met the requirements under the Deed and the trial Guidelines.

With regards to the Career Account, providers very much saw their role as one of support and guidance. The fact that the trial gave participants the opportunity to influence and (largely) control the money and ultimately what steps would be taken in order to get them more job ready was strongly supported by the providers interviewed. Providers saw this approach as empowering for the trial participants and noted that it was more likely the approach (as opposed to the amount of money) that was influencing the positive changes they were witnessing in their trial participants.

However, most DES providers noted that the majority of trial participants struggled to make decisions on how the Career Account monies were to be spent and that they did need to provide substantial guidance on plans to use these funds. This was most commonly achieved by either brainstorming with the participant on things they felt could be useful to them in getting a job, or by asking a series of more focussed questions and then offering suggestions based on the participant responses. It was evident from DES provider feedback that participants were often overwhelmed by the amount of money available to them and were looking for support from providers in determining how the monies were to be spent.

“Using suggestions or using examples from other job seekers participating in the trial... or doing a skill gap or accreditation gap or a bit of vocational planning and then working backwards from that, so

teasing out of their mind where they are, where they want to be and what would help get them from A to B.” (Provider, Brisbane)

DES providers indicated that where participants sought guidance on developing a plan to spend the monies available, participants were most commonly encouraged to consider the following items or activities:

Education – Providers saw education as a key pathway to employment for many of their trial participants, with numerous participants using some of the Career Account for education (e.g. courses/qualifications) and education related expenses (e.g. laptop computers). This was also evident from the administrative data provided by the Department, as this category experienced the highest amount of spend across all trial participants. o For some trial participants this meant being able to complete short courses to assist with

immediate employment. For example, one provider in Brisbane mentioned that a participant was offered a job after commencing the trial, however needed to have completed a specific first aid course to be able to accept the position. This trial participant

38

only had one week to make this happen and was able to use the Career Account funds to cover the cost of this course at short notice. In this instance the provider believes that this would not have been possible in such a short time frame had it not been for the trial.

o For other trial participants education was about increasing their employability for the longer term. They were enrolling in and completing formal courses and qualifications to make them more employable and to continue to increase their confidence and mental wellbeing. It should be noted that under normal circumstances job seekers are able to participate in formal education and achieve an education outcome, provided it is an approved Registered Training Organisation course or a University course and meets the criteria of a ‘Qualifying Education Course’ (meaning that it is a) approved for Austudy or Youth Allowance (Student) or Abstudy purposes; b) normally of two or more Semesters in duration within a 12 month period; and c) Full-Time Study). Providers mentioned that while they do see this as an option for job seekers it can be challenging assisting with this as they have limited funds for each job seeker, and as a result these types of education are not readily accessed.

Travel – Providers mentioned that for many people with a mental illness, public transport is a horrific experience - as it creates anxiety and induces stress - and many don't (or feel they can’t) use it. In addition to this, some participants lived in geographic areas that were not very well serviced by public transport, making it difficult for them to be able to commute for work and/or access education and training. Given these barriers, providers were generally very supportive of trial participants using some of the money to help pay for travel related expenses such as driving lessons, license fees and learners exemptions – which was also evident from the administrative data with this being the second biggest spend category at 16% of total spend (or $75,764). Providers noted that such supports not only addressed the practical barriers identified above, but also gave trial participants a sense of freedom and independence. It also gave them greater confidence and motivation to get a job, as they may wish to earn money to purchase a car (having obtained their licence).

Mental health – Providers were excited by the fact that the trial allowed participants to get quicker and more specific access to mental health services (e.g. specialist psychologists), compared to the types of support DES providers can fund under the standing administrative arrangements. Many of the providers mentioned that their trial participants have very complex needs and that many wouldn’t be able to get a job until they received the appropriate level of tailored support to address these conditions – and providers seemed to be of the opinion that they don’t normally have sufficient funding to get the tailored level of assistance that these job seekers require. To this end, the administrative data demonstrated that 15% of total spend was on mental health assistance, which was the third highest spend category. The existence of the trial seems to have given both providers and job seekers added motivation to address mental health barriers and to look for ways to do this. However, some providers noted that there were job seekers that continued to resist engaging in services to address their mental health barriers, even though they had the opportunity through the trial.

Work Experience – Providers from all three locations did mention that for job seekers with a mental illness one barrier to employment can be that some employers are simply reluctant to hire them. Some providers noted that the trial presented them with an opportunity to assist job seekers to overcome this barrier through paid work experience. This involved finding employers willing to take the job seeker on for a work experience placement for a short initial period (such as 2 weeks), and the job seeker being paid an appropriate wage out of their Career Account funds. These providers noted that such placements allow the job seeker to demonstrate their skills and abilities to the employer, with the hope that this will lead to continued employment – and ideally change perceptions towards hiring those with a mental illness more broadly. This is a technique that these providers have used in the past, however

39

not with the speed or flexibility that was available through the trial. It was noted by other providers that were interviewed that they believed the funds were not able to be used in this way, which suggests that communication/guidelines would need to be improved for future programs of a similar nature.

3.3. Journey mapping for trial participants

Across the two phases of depth interviews conducted with trial participants and DES providers, it became apparent that there was a general sequence of key stages that participants followed in relation to the trial. The journey map developed for trial participants had six key stages, which are depicted in Figure 7 below. These stages collectively provide an overall journey map for trial participants and provide a generalised description of the process, noting that not all trial participants experienced all stages.

Figure 7: Mapping the Youth Mental Health trial

3.3.1. Awareness of the trial

The initial round of face to face depth interviews with trial participants and DES providers signalled that in the vast majority of cases DES providers were the primary source of awareness and information for participants of the trial. Trial participants indicated that they were made aware of the trial through one of the following ways:

• Some trial participants were made aware of the trial during their usual fortnightly catch up with

their DES case manager.

• Others trial participants were called specifically by their case manager and told about the trial

outside of their usual meeting times.

40

• A few trial participants were informed by someone other than their case manager, who was

involved with organising the trial for their DES provider.

• Just one trial participant mentioned that the first time he heard about the trial was in a letter

from the Department and that he shortly after spoke to someone from his DES provider about

participating.

“My case manager told me about it in about October. He told me the specifications, that there is $5,000 at our disposal, but at that stage I think he was a bit unsure about what we could use it on. He

was just saying that potentially I might get picked to use this grant.” (Male, Brisbane)

There were mixed emotions from trial participants when they first find out about the trial. Many were excited by the idea and straight away saw it as a great opportunity, while others were more reserved and wondered things like “why was I chosen?” and “what’s the catch?”.

Those that were immediately enthusiastic about participating in the trial indicated that their interest was primarily driven by feeling lucky to have been chosen and an eagerness to find out about what the trial would mean for them.

“I was excited, it felt like Christmas ... generally I ask for nothing, I find it difficult to ask for what I want.” (Male, Melbourne)

“A feeling that somebody cares, that you are not alone, that there are options like this that can help you.” (Male, Brisbane)

3.3.2. Joining the trial

For the purpose of this trial DES providers in the three chosen Employment Services Areas (North Brisbane QLD, Maroondah VIC and North Metro WA) were asked to select appropriate job seekers and invite them to participate (including giving them ample information about the trial to assist with the decision). It was then up to the job seeker to accept or decline the invitation to participate in the trial. The trial participants that were interviewed for this evaluation either immediately accepted or weighed up the pro’s and con’s before deciding to participate (as discussed below). It is important to note that job seekers who were invited but declined have not been included in this research. It is possible that barriers to participation exist, but have not been identified due to the declined cohort not being included in the evaluation. The figure below is a visual representation of the job seekers included in this research (as outlined above).

41

Figure 8: Trial participation decision making

For most of the trial participants interviewed for the evaluation, the decision to join was not difficult, however as demonstrated in Figure 8 above, there were a few participants who were reserved and cautious about the trial. This group needed to weigh up the pro’s and con’s before making their decision. They were anxious about trying something new and were apprehensive about the pressure and responsibility that would come with having control over such a large sum of money (i.e. $5,000). From speaking to others (such as their provider and their family) for advice and guidance it didn’t take long for them to decide that they wanted to participate.

A large number of trial participants interviewed mentioned that they were so eager to be a part of it that they agreed to participate as soon as they were told about it (i.e. during the same catch up/meeting).

“I agreed pretty much straight away when she told me about it.” (Male, Perth)

Encouragingly, all participants included in this research became more positive about the trial the more they were informed about it.

“I am grateful that there is a program out there supporting people like me.” (Male, Melbourne)

Again it should be noted that only those that agreed to participate were included in this evaluation and so there is no way of knowing whether further information would have assisted to overcome the barriers to participation among those who declined to participate.

3.3.3. Making decisions

As previously discussed, the experience of deciding how the Career Account money could best be spent varied greatly across the trial participants that were interviewed. Some mentioned that they immediately had ideas of how they could spend the money, while others felt that they needed more guidance and help from their DES provider.

“We started thinking about how I could spend the money. My case manager made a lot of suggestions. There was a lot of brainstorming to try and fit my needs to the money.”(Male, Brisbane)

“It's good that the use of the money is policed. It could have ended badly if I was given cash.” (Female, Melbourne)

42

Trial participants mentioned family and friends as having an influence over how they decided to spend the Career Account money, with DES providers being mentioned as the biggest influencer. When asked who ultimately decided how the money was spent all trial participants indicated that it was themselves, with their DES providers giving them guidance and approval.

“(My provider) gave me ideas, but didn't force me.” (Male, Melbourne)

The vast majority of trial participants indicated that they felt empowered by the fact that they could choose to spend the money on what they felt would help them get a job (within reason and with approval from their DES provider).

“I do know what's good for me so the freedom and flexibility is very good.” (Male, Melbourne)

DES providers interviewed for this evaluation also commented on the positive nature of allowing and encouraging trial participants to make decisions. This aspect of the trial was significant for all involved and was considered to be a key aspect to most participants reporting a positive trial experience.

“This trial doesn’t impose itself upon you, it enables you to strive to become who you are at your greatest potential, but it is up to you.” (Male, Brisbane)

3.3.4. Changes and Impacts

Increases in confidence, self-esteem and motivation

Increases in confidence, self-esteem and motivation were noted by not only DES providers, but also by trial participants themselves. Trial participants included in this research mentioned feeling more confident and having greater self-esteem as a result of the trial and being given the opportunity to make decisions for themselves about what will make them more job ready.

“If somebody else is saying what to do you probably wouldn't do it.” (Male, Melbourne)

Many of the trial participants also noted that they now feel more motivated to find a job or do things that will advance them in a career as a result of the trial.

“It gives you the tools necessary to get back on your feet, especially if it is someone who has been unemployed for some time.” (Male, Brisbane)

This increased confidence, self-esteem and motivation led some trial participants to be more active and effective in their job seeking activities. At the time of the first interview some trial participants were still in the first stage of the employment journey (Figure 9 below) and as such were focusing on first addressing their mental health issues. For others, at the time of the first interview, the trial had already led them to more active and effective job seeking.

By the time follow up interviews were completed, many of the trial participants were in Stage two of the Employment journey map, while only a small proportion (as according to DES providers and the trial participants interviewed) had progressed to Stage three.

When looking at how many had progressed to Stage three, the administrative data supplied by the Department demonstrated that 11 trial participants had gained employment and received a Full Outcome (13 weeks) at the conclusion of the trial. A further seven trial participants had received a 13 week Pathway Outcome by the conclusion of the trial – meaning that they were progressing towards

43

the achievement of sustainable employment. The administrative data also showed that no trial participants had achieved a 26 week Full Outcome and none were on a 26 week Pathway.

Figure 9: Employment journey map

Improved relationships between trial participants and providers

A number of trial participants indicated that the relationship they have with their DES provider had improved as a result of the trial. They felt that their provider respected them and really did want to help them succeed in finding the job/career they wanted - not just any job. Some also mentioned feeling happier with the government, knowing that they were doing something to help people in situations like theirs.

“It's not set-up so they are telling me what to do; I'm not forced to be in a job that I don't want to be in; it's not just about what they want me to do.” (Male, Melbourne)

“I felt really honoured because it meant that the people at the DES provider knew that I was really trying and believed in me enough that they would put money behind me.” (Female, Melbourne)

Flow on effects to family

A number of trial participants mentioned that their home lives and their relationships with their families had also improved as an outcome of the trial.

• Some mentioned being happier and more optimistic about their future, which in turn made

their families happier and more optimistic as a result.

• For others it was about having a financial burden lifted. This was particularly true for trial

participants from lower income families.

“It is helpful for just the fact that it exists, for both my mental health and for the living situation I am in, because the people around me aren’t as stressed… which means that I am less stressed.” (Male,

Brisbane)

44

“By me participating in this the whole feel of my home is uplifted, it is more joyful and it’s not as depressing. It is not in a downward spiral, there is hope there.” (Male, Brisbane)

Anxiety challenges

Some trial participants were concerned that using the Career Account would take them out of their comfort zone and put them into new situations that may be uncomfortable. Despite these feelings of anxiety and nervousness, most felt that it would likely be/was (if already completed) worth it as it would help them to grow.

One trial participant mentioned feeling ill from anxiety prior to completing a first aid course, because he was going to be turning up at a new place with new people. However, he felt fantastic after he had done it – “I felt like I was doing something with my life.” (Male, Melbourne)

3.3.5. Employment/Education

Employment

Only a few of the trial participants interviewed as part of the evaluation noted that they were in employment at the time of the follow up interviews. DES providers also made mention of some of their trial participants finding employment during the trial, with some unfortunately not maintaining the employment. This was consistent with the findings from the administrative data that showed very few trial participants had achieved a 13 week Full Outcome, a 13 week Pathway Outcome or a that a DES provider had received a Job Placement Fee.

Those that were employed at the time of the follow up interviews were employed in a range of job types and industries, including child care, hospitality, landscaping and labouring, administration and one was even in the process of setting up their own business. There were mixed feelings amongst the few trial participants that were employed with regards to how much the trial assisted with them gaining employment. Some made mention of the trial affording them opportunities that led them to their employment, while others felt that they found the employment without the assistance of the trial. One trial participant noted that the while finding employment was not related to the trial, being able to accept the position was. In this instance, he needed specific clothing for the position and needed to move to a new state. He felt that without the opportunity to purchase goods through the Career Account he would not have been able to accept the job.

“I was at the stage where I was thinking that I would never get a job, so actually hearing back “yes you have got the job” were words I never thought I would hear… I am able to start work really confidently and not feel self-conscious about lacking equipment… (The most significant change was) Being able to happily take the job and probably be nowhere near spending all of the money, which is fine, and

still have this job and not have to stretch myself to do it.” (Male, Brisbane)

Education

The trial was seen by some participants and DES providers as a key factor in supporting long-term employment opportunities. Many of the trial participants were either already studying or gaining formal qualifications and using some of the money to assist with textbooks and fees, while other participants were using some of the Career Account money to sign up for a formal education pathway. This was supported by the quantitative analysis of the administrative data, which showed that the largest spend category for all trial participants was on education and education related expenses (including TAFE courses, First Aid courses, textbooks and laptop computers etc.).

45

“Generally speaking I wouldn't have had either (driving lessons and TAFE course) of these chances; basically I would have just been searching for jobs that I keep getting turned down for.” (Male, Perth)

DES providers and some trial participants also mentioned that studying was not just about gaining a qualification, certificate or diploma. They also saw study as a way of helping to get used to routines (e.g. needing to attend classes at specific times throughout the week) and being around other people, both of which were observed as a struggle for many of the trial participants. If job seekers are able to stick to these routines then this offers a good opportunity for growth, however it could also be an area where job seekers fail and take a step backwards if they aren’t able to cope with the situation.

3.3.6. Next steps

As previously noted, not all trial participants had reached a Full Outcome or a Pathway Outcome at the conclusion of the trial. However, the majority of participants interviewed for this evaluation saw the trial as enabling them to do something that they otherwise wouldn’t have been able to do, in order to help them with long-term employment.

The chance of being able to get ‘the’ job they want greatly motivated some trial participants to put increased energy into their job seeking. One trial participant in Melbourne described it as a “stepping process”, with the Career Account significantly assisting him to progress his career plan. Another trial participant in Perth felt that the trial had helped him to discover what he is truly “passionate about” and he feels as though he will now be doing what he is “supposed to be doing”.

“It was probably better than I thought, because it means I can actually get on with my life and actually do stuff.” (Male, Perth)

For the trial participants interviewed, it really was about the opportunities they were afforded as a result of the trial and the benefits that they felt would set them up for their future.

“It's an exciting opportunity, it's something that you need to have a bit of commitment for, you need to be committed to working out what you are actually going to do… It hasn't been too stressful; it has

been quite a smooth procedure. It has definitely been exciting and it has opened up a lot of opportunities.” (Male, Brisbane)

These self-identified benefits of the trial included:

• Having power and responsibility and learning to make decisions – The vast majority of

trial participants spoke encouragingly of the fact that they had the power to make decisions

about how to spend the Career Account money in ways that would assist them.

“It’s helping out a lot, it’s good to spend the money in any way and having the power to make my own decisions.” (Male, Perth)

• Improved confidence, self-esteem and motivation – As previously mentioned many trial

participants interviewed expressed that they felt more confident, had greater self-esteem and

increased motivation as a result of the trial. With most indicating that these changes stemmed

from the participant-driven focus of the trial where they were given a high degree of choice

46

and control over how the Career Account funds would be used. Several mentioned that they

felt as though this improved confidence, self-esteem and motivation would assist them with

their job searching activities during the trial and beyond.

• Improved mental health – Quite a few trial participants that completed the follow up

interviews also noted that they felt more mentally healthy at the conclusion of the trial. Some

conveyed that this was as a result of having access to specialist mental health services, while

others noted that it was a result of the other benefits they were seeing.

“Yeah, it's had a good impact on my mental health, how I feel.” (Male, Melbourne)

• A sense of progression and moving forward – Many trial participants felt that they were

moving forward at a greater pace then they would have had they not participated in the trial.

This was especially true for those that had engaged in some form of formal education.

“It was positive and overall it has helped me to get out of a rut and think about the future.” (Male, Brisbane)

“Helping me to move forward, I feel like a busier woman. I'm actually doing something with my life.” (Female, Perth)

• Improved relationships – Some trial participants saw the improvements in their relationships

with their provider as well as their relationships with their families as a direct benefit of the

trial. In terms of the improved relationships with their providers, this emerged from the change

in the way participants and DES providers interacted and the increased responsibility that was

afforded to participants. The improved relationships with family tended to stem from a

reduction in stress, either with regards to finances or just their situation in general.

These trial participants not in employment at the conclusion of the trial were hopeful that these changes that they had seen in themselves would lead to employment in the near future.

3.4. Providers experience with the trial

The DES providers interviewed as part of the evaluation were all very positive about the trial. They saw the trial as an opportunity to help their job seekers in ways that aren’t always feasible under normal circumstances (due to time and money) and were also encouraged by the benefits the trial afforded not just their trial participants but also for their staff and organisation as a whole.

“Nothing but positive, gives clients an opportunity to break through all of the barriers and excuses.” (Provider, Melbourne)

47

3.4.1. Benefits for trial participants – as noted by DES providers

As previously discussed, trial participants noted several benefits of the trial. DES providers also noted the changes that they had seen in trial participants as a result of the opportunities they had been afforded through the trial and the Career Account.

All providers were excited for their trial participants and noted observing several positive changes (listed below) over the course of the trial, which were largely consistent with the benefits that trial participants identified.

• Greater confidence and composure – Providers spoke of seeing improvements in their trial

participants with regards to their confidence and composure. They noted that the nature of the

trial being participant-driven was largely impacting this increased confidence and composure

that they were observing in their trial participants. Providers did note that this was not the

case for all of their participants, however it was true for the vast majority.

“The job seeker we have participating in the trial is more empowered, is likely to be more

employable and will be more job ready(at the conclusion of the trial)… their confidence has

definitely grown.” (Provider, Perth)

“It's been fantastic to see how it's enriched his life, and given him confidence. But there is a

long way to go.” (Provider, Perth)

• Improved communication skills and abilities – Many providers also made mention of their

trial participants improving their communication skills and abilities over the course of the trial.

For many of the job seekers participating in the trial this was the first time that they felt they

had truly been afforded an opportunity to talk about what they thought would be best for them

in terms of assisting them in overcoming any vocational or non-vocational barriers, and to

further their employment goals. Providers mentioned being able to have conversations with

their trial participants that they would likely not have been able to have without the trial, as

they simply wouldn’t have had the resources or funds to follow through with requests. As

providers were able to assist more through the trial, they noted that participants were more

willing to engage and were communicating better.

“They're more open to doing something different. If they need something they'll call, before

they wouldn't ask questions. They're more engaged with us now.” (Provider, Melbourne)

“For a couple of the job seekers there certainly has been increases in their levels of

engagement with their consultant, because stuff happened... it is much easier for a job seeker

to engage when they are actually seeing outcomes.” (Provider, Brisbane)

48

• A greater willingness to attend their scheduled meetings and stay in touch – The

providers interviewed for the evaluation noted that a large number of trial participants were

more willing to attend their scheduled meetings as they felt that more motivated and could

see that things were happening. One provider also mentioned that their trial participants that

had found employment were more inclined to stay in touch.

• Improvements to mental health conditions and a more positive outlook about their

future – Many of the providers interviewed mentioned that at least some of their trial

participants showed signs of improved mental health over the course of the trial and most

were more optimistic about their future. It was noted that for some this was due to the

specialist services they were engaging with (i.e. counselling, psychological consultation and

dialectical behavioural therapy etc.) while for others it came from the change in attitude and

was linked to their increased confidence.

“They have more goals, are talking more about hope, changing living styles, one has made a

move out of home.” (Provider, Melbourne)

3.4.2. Benefits for DES providers

As part of the evaluation providers were asked about their organisation’s experience with the trial and the benefits and challenges they faced. The primary benefits noted by the providers were:

• Greater ability to assist job seekers with complex needs – While standard support

processes delivered by DES providers do allow for a small degree of tailoring, the Career

Account funds enabled timely access to a range of more customised supports that are not

possible under the current DES model. These included admission to education courses and

qualifications not covered under the DES Deed and Guidelines, access to specialist mental

health services (i.e. Dialectical Behavioural Therapy and psychiatrists) that are not

necessarily covered under normal circumstance, and assistance with travel expenses (i.e.

driving lessons) that providers would normally be reluctant to assist with due to limited

funding;

“We have been able to do those little bits and pieces (with regards to improving mental

health) that we wouldn't normally have been able to do, because we have been so focused on

employment.” (Provider, Brisbane)

• Staff being invigorated and improved relationships – The changes detected in the

motivation and engagement of trial participants was observed to have a similar effect on the

staff interacting with them. Many DES providers expressed great satisfaction in terms of the

49

trial helping participants reach their full potential and the positive transformational impact this

had on staff at the DES provider as whole. In addition to staff being invigorated, many

providers noted that relationships between trial participants and case managers had improved

over the course of the trial. Trial participants felt more valued and were more involved, which

in turn meant that they were more willing to attend meetings and engage with their case

manager;

“Engagement of the jobseeker... They're more willing to participate with their appointments,

there's more working together… It has improved quickly - the relationship with the job seeker,

I didn't expect that deeper relationship to develop... that was unexpected.” (Provider,

Melbourne)

• Having more leverage, incentive and funds to help job seekers – Providers were

enthusiastic about assisting job seekers to up skill and obtain goods and services that they

don’t usually have the funds for. They liked that there was a large degree of flexibility with

regards to how the funds could be spent and, for the most part, they did like that the funds

were participant-driven. However, providers did note having difficulties with some trial

participants when it came to deciding how to spend the money. A few providers mentioned

feeling frustrated by some trial participants not using the Career Account to its full capacity,

despite providers offering many suggestions and support. In a few rare instances, providers

felt that it would have been good if they could have directed how some of the funds were

spent;

“A lot of power has gone to the client to identify resources and activities but their judgement

may not be the best. The hardest thing has been engaging clients with suitable mental health

professionals - clients have gone for more social things like personal training and dance

lessons.” (Provider, Melbourne)

• Rewarding feeling – All of the providers interviewed for the evaluation made note of the

rewarding feeling associated with seeing participants provided with the opportunity to move

forward faster than through the usual system.

“You get a real buzz actually having money to spend on them.” (Provider, Perth)

50

3.4.3. Challenges experienced by providers

As discussed, the DES providers interviewed for the evaluation were very supportive of and happy with the trial. However, many also mentioned that there were a few challenges faced during the course of the trial.

Many of the providers suggested that one of the key challenges they faced was the engagement of trial participants. Some trial participants struggled to engage just in the beginning of the trial, while others struggled to fully engage throughout the entirety of the trial. This was also supported by the underspend of the Career Account that was observed in the administrative data, where there were 52 trial participants that had spent less than a fifth of the funds (i.e. spent between $0 and $1,000).

“The trial has only been a success for one person. With the others, it didn’t matter what we offered or suggested, they would say “I've got everything”… they just weren't willing to engage… For one I think she had different priorities, it wasn’t high up her priority list because she was transitioning genders.

Maybe the timing wasn’t right for her.” (Provider, Perth)

“I did have to be more proactive, pushing them a little bit. I didn't think it would be such a struggle for some of them to know what to use the money for.” (Provider, Melbourne)

The providers mentioned that the lack of engagement was surprising for them and was caused by a number of factors, including:

• The amount of money ($5,000) being a large sum that was daunting to some trial participants;

• Some trial participants experiencing mental health episodes outside of the trial that then

impacted their ability and willingness to engage with the trial process; and

• Some trial participants simply not engaging with no foreseeable reason from the providers

perspective.

Another challenge faced by a few DES providers was trial participants trying to take advantage of the funds being made available to them (e.g. one participant tried to make a claim against their own ABN) or not being happy with providers when they didn’t approve a good or service. A couple of the providers mentioned that the lack of framework and clear guidelines was challenging, as some trial participants became upset when their provider opposed or rejected goods and services and there wasn’t a clear guideline to refer to. Some providers noted that while they were aware that the Guidelines of the trial were kept broad to assist with the participant led approach, they found it challenging to interpret the limited information they were given. Providers also noted being very mindful of the need to ensure that spend was “defensible and capable of withstanding a reasonable level of public scrutiny so as to bring DES into disrepute”, as per the Guidelines.

Some of the providers interviewed also noted that not being able to assist all of their job seekers was challenging. They appreciated that the trial needed to be limited, however they did note that it was a struggle for them, especially when other job seekers were made aware of the trial and could not understand why they weren’t being given the same opportunity.

“It would be good to offer it to everyone. It's difficult when other clients overhear your conversations with these trial participants.” (Provider, Perth)

51

3.4.4. Different approaches

DES providers were happy to try this new way of thinking and new way of supporting job seekers; however some providers did find it more challenging than others to change their ways of thinking and doing things. It was evident from the interviews with providers and trial participants, as well as from the comments made in the administrative data, that there were a broad range of approaches adopted with regards to providing trial participants with assistance. These approaches varied depending on the provider, the trial participant and the stage of the trial and included.

• A very laid-back and hands off approach compared to a quite involved and directive

approach;

• Really leaving it up to the trial participants to think about how to spend the Career Account

funds compared to providing strong guidance on how the money could be spent and/or

creating situations, such as workshops, where trial participants could help each other to think

of ideas; and

• A willingness to find a way to approve goods and services compared with a more risk adverse

approach and a reluctance to approve.

Some providers noted that they would not be making changes to their standard service delivery approaches; however they were happy that the trial had enabled them to support their participants in ways they hadn’t previously. Others noted that they had learnt from the trial, stating that these learnings would influence how they support their job seekers in the future.

• Some providers noted that they were surprised by some of the goods and services that trial

participants wanted to spend the Career Account on (such as driving lessons) and will look at

how they can assist other job seekers with accessing these types of goods and services.

• One provider noted that they would be looking at changing their approach to interacting with

job seekers from this cohort to try to have more “we” conversations.

• One provider mentioned that their organisation was moving towards parallel servicing and a

more intensive support model, however they did note that this will be challenging without the

extra funding (i.e. the Career Account).

3.5. Areas of potential improvement

As previously noted, DES providers were largely positive about the trial and the opportunities the Career Account provided participants. The providers saw it as a new way of doing things that they could very much support and get on board with, however they also noted areas of potential improvement if such a program were to be implemented more widely.

Even though providers were very positive about the trial and the benefits it afforded the job seekers, there were still some areas that they felt could be improved if a similar program were to be rolled out more widely in the future. These areas of potential improvements identified were:

52

• The funds themselves (i.e. improving the spend process, improving the distribution process of

the funds and slightly changing what the funds are spent on);

• The amount of time allocated to spend the allocated money;

• The information given to DES providers;

• The recording of details in the system; and

• Communication to participating job seekers.

Improving the spend process

The trial Guidelines did not go into great detail about the process for how Career Account funds could or should have been processed. The Guidelines simply outlined that funds were not be paid directly to the trial Participants, with paid work experience being the only exception.

Some providers mentioned having difficulties with the rules around how the money could be distributed and processed. Some felt frustrated that they were not able to reimburse trial participants or give them cash and then receive receipts at a later date. The primary reason being that the trial empowers the participants and gives them a sense of ownership, which can be partly taken away when they aren’t able to go and purchase things for themselves, such as clothing. This also highlights the degree to which the guidelines are clear (or not) and that they were open to different interpretations by the providers, as some providers mentioned that they had reimbursed trial participants.

“There have been a couple of times when I have really struggled with the guidelines. One of the guidelines talks about us not being able to reimburse the participant… For example, I had to go

shopping with the participant who wanted the clothing purchased and she was really self-conscious of doing that side of it. It was good because she got good feedback and she bought some very

appropriate clothes but it would have been good to be able to say “you’ve got the power, you are a mature, sensible person, you know what is appropriate, we have had a look at what is suitable for the job so go for your life, you’ve got your own job, good on you, keep with the self-empowerment”. And that is about trusting that she will buy the appropriate clothing, so it is about the individual” (Provider,

Brisbane)

If a similar program is to be delivered in the future DES providers would like to be provided with greater clarity around this area of the model. Also, as previously noted, one possible solution that was noted by a provider would be to give DES providers a credit card with the $5,000 on it for each participant. This card would still be kept in the possession of the provider; however it would afford each participant a feeling of using their own money and would make it easier for providers to make purchases for participants. It would also be a good mechanism for record keeping. Another idea that was suggested by a provider would be to include a “plan creation phase”. This would be a step at the beginning of the process whereby participants and providers would create a plan and outline potential ways that the funds may be spent (in general terms). This plan would then be sent to the Department for endorsement and approval, meaning that participants and providers would be clearer on what could be approved, right from the beginning of the process.

53

Improving the distribution process of the funds

As previously mentioned in this report, many trial participants and providers felt that $5,000 may be too much money for a job seeker in this cohort. Overall, providers felt that between $2,500 and $5,000 would be appropriate for a program such as this trial. Ideally there would be some level of flexibility as it was mentioned that some trial participants could use more money than others.

One provider suggested that it would have been good if any “left over” funds from one participant could be given to another participant (with the participant’s approval). It is envisioned that strict guidelines would need to be in place if such a change were to be made, to ensure participants didn’t feel “bullied” into transferring some of their funds to another participant.

Another suggestion was that there be tiers of funds and that these could be determined based on Job Seeker Classification Instrument (JSCI) scores. While another provider felt that the amount of funds could be based on the participant’s family income and education levels (i.e. those from lower socioeconomic families would have access to more money, while those from wealthier families would receive less funding).

Across the different suggestions from providers and from the feedback from trial participants themselves it is clear that a “one size fits all” model with regards to the amount of funds is not necessarily the best approach. It was noted that the key driver of success was the fact that the trial was participant-driven. As such, any future programmes should look at keeping the key element of being participant-driven and then explore different options to determine the best way of distributing the funds to ensure that the amount is suitable for the job seeker.

Slightly changing what the funds are spent on

It was also suggested that a program for those with mental health issues should mean that at least some of the money goes towards improving the mental health of participants. A provider in Melbourne suggested that a possible change to the guidelines could be that participants “must engage in some services to address mental health”.

“Particularly with our job seekers that have had counselling, we have seen increased communication skills. We have seen a reduction in maladaptive communication skills, which will really help. We are removing a negative influence for the long term that is really going to help these job seekers to thrive

in a work environment.” (Provider, Brisbane)

More time to spend the money

Some providers did acknowledge that a number of their trial participants didn’t start at the beginning of the trial, however they also mentioned that the timeframe to spend the money could be extended, especially considering the complex situations for the cohort.

“Mental health is a lifetime journey not a six month fix.” (Provider, Melbourne)

“My overall feeling is that individually, people are complex and there is no quick fix. Some of their issues will take years to sort out, not months.” (Provider, Perth)

It was suggested that if this trial were to lead to a wider program the timeframe to spend the money should be longer than 12 months. Ideally the providers would like to see the money available for the “DES lifetime” of participants or until it runs out (whichever comes first).

54

Trial participants also felt that the length of time to spend the fund should be extended if a program such as this were to be more widely available.

“It's been easy. What made it slightly harder was the time…it needs to be a couple of months longer as not everything goes to plan.” (Female, Perth)

“There could have been more planning and brainstorming. More time to work out how to spend it.” (Male, Perth)

Improving information

Many of the providers mentioned that while it was good that there was flexibility around how the money could be spent, it also presented difficulties with regards to them determining what they could suggest to trial participants and what they could ultimately approve.

Many mentioned trying to use the email address on a number of occasions to enquire about whether something could be approved or not and receiving vague and ultimately unhelpful information.

Greater ability to record details

As per the trial Guidelines DES providers were required to “complete quarterly updates of the trial participant’s Job Plan as outlined in the Job Plan Guidelines”. The Job Plan Guidelines (as found on the Department of Social Services website) describes a Job Plan as “an individually tailored plan that considers the Participant’s employment goals and records activities the Participant will undertake to gain sustainable employment. The Job Plan is a key document which underpins provision of services to Participants. This document is an important way for participants, and if appropriate their families, to know what support and assistance they will receive.”

Some of the providers interviewed mentioned that they experienced difficulties with regards to including all of the necessary details related to the trial in the participants’ Job Plans. They noted that if such a program were to be rolled out beyond a trial that this may be an area the Department needs to look into.

“The Job Plan is one area that we have struggled with a little bit. The Department told us that the Job Plan had to include the Youth Mental Health Pilot, but we couldn’t find it in the drop down, so we have

been free texting it. We also find that the Job Plans are really minimalist, they used to be called an Employment Pathway Plan and you used to be able to put a goal in there, but we can’t do that anymore. So it is just very much compliance focused because of the Job Active framework.”

(Provider, Brisbane)

One provider suggested the idea of a Job Plan application that both DES providers and job seekers could download onto their phones or computers, thereby allowing job seekers to be more involved in the whole process and enabling both parties to keep track of spend in an easy format.

Better communication with participating job seekers

Some trial participants initially found the idea of the trial daunting and it caused them anxiety to think about how they would spend the money (and what the “catch” was). It was suggested by some providers that more direct communication to trial participants from the Department could have helped trial participants to more quickly overcome the feeling that the trial was not real and also help them to engage with the trial more quickly.

55

It a program similar to the trial is to be developed, providers suggested that examples could be given of how the money could potentially be spent or examples given about how spending the money in a certain way has helped a job seeker find employment. The providers mentioned that any such communication would need to be clear and easy to understand, yet still provide sufficient information to participants and their families to ease any concerns they may have.

56

4. Appendix A: Methodology in detail

This section details the project scope, methodology process, deliverables and timings.

4.1.1. Project scope

The Department of Social Services (the Department) commissioned Colmar Brunton to conduct an evaluation of the Disability Employment Services (DES) Youth Mental Health trial.

The purpose of the study was to explore the experiences of job seekers and DES providers participating in the Youth Mental Health trial. The evaluation was conducted through a multi-staged qualitative and quantitative study that ran from October 2015 to September 2016.

The information obtained through this research was designed to test the effectiveness (including cost effectiveness) of a more participant-driven service model within DES and will assist with informing the design of employment services for people with a disability from 2018.

4.1.2. Deliverables and timings

The agreed and the actual schedule and the list of deliverables of the research are outlined in Figure 10 below. The timeline for the research was agreed upon at the beginning of the project; however, in some cases changes to the dates were required; therefore, the actual dates have been specified as well. Comments by Colmar Brunton are indicated with ‘*’.

57

Figure 10: Deliverables and timings

Project task Original Timing Date delivered

Ethics clearance application submitted By 18 November 2015 18 November 2015

Ethics committee approval 11 – 14 December 2015 4 January 2016

Fact sheet issued to DES providers w/c 14 December 2015

14 September 2015 to 5 February 2016

Recruitment of participants by DES providers From 27 January 2016 w/c 25 January 2016

Provision of trial and control data samples from administrative data 11 January 2016 14 January 2016

Discussion guides for Qualitative Fieldwork Phase 1 finalised 27 January 2016 5 February 2016

Qualitative fieldwork period – Phase 1 (n=45 1-hour depth interviews, face to face) 1 – 28 February 2016 9 - 18 February 2016

Mid-research project debrief provided 15 March 2016 16 March 2016

Discussion Guides for Qualitative Fieldwork Phase 2 finalised 30 April 2016 5 May 2016

Qualitative fieldwork period – Phase 2 (n=45, 1-hour depth interviews, phone) 1 – 29 May 2016 11 May - 15 June

2016

Provision of trial and control data samples from administrative data 3 July 2016 6 July 2016 (updated

on 26 July 2016)

Draft written report 3 August 2016 5 September 2016*

Draft methodology report 3 August 2016 5 September 2016*

Feedback received from the Department on the draft reports 24 August 2016 8 September 2016

Final written report ^ 12 September 2016 12 September 2016

Final methodology report ^ 12 September 2016 12 September 2016

Presentation / Discussion (if required) TBC TBC

Contract completion 12 September 2016

58

* The timing of the draft report was delayed due to the delay in administrative data being delivered to Colmar Brunton and a subsequent delay with regards to answering questions on the data.

4.1.3. Methodology outline

The evaluation of the DES Youth Mental Health trial was split into two key research areas that each contained multiple phases (which are explained in detail below):

• Qualitative research with trial participants and DES providers involved in the trial; and

• Quantitative analysis of Departmental administrative data.

The final methodology used for this research project was chosen based on the Departments original desired approach and suggested inclusions (as outlined in the RFQ provided to market), as well as the additional detail given to Colmar Brunton during a subsequent briefing on the background and context to the project at a meeting on 20 October 2015. The final approach agreed to and used for the evaluation is outlined in Figure 11 and is explained in further detail below.

Figure 11: Research approach

4.1.4. Project planning and scoping

The initial project planning and scoping meeting was conducted on 3 November 2015. The meeting allowed for finalisation of the proposed methodology, creation of a program logic framework (discussed further below) and agreement on the key research questions that needed to be answered through the project.

All subsequent phases of the study were then undertaken within the context of this agreed project plan framework, including the ethics application and the development of all research instruments (i.e. interview guides).

59

4.1.5. Program logic framework

In order to properly evaluate the program, Colmar Brunton recommended conducting a program logic workshop, which was completed as part of the initial planning and scoping meeting held on 3 November 2015.

Program logic is a process of identifying a hierarchy of outcomes where specific actions and mechanisms are linked to changes in behaviour which ultimately lead to the achievement of defined success and end-states. It is a logical stepwise process that enables clear thinking and clear identification of goals, gaps and required actions and programmes.

Colmar Brunton uses six focussing questions to facilitate development of the project logic framework, which the Department and Colmar Brunton project teams worked through together. The purpose of developing the clearly defined program logic was to enable the appropriate focussing of evaluation questions at all points of the evaluation on the basis of how the program should operate in theory, including the key metrics that would define success and all of the pre-cursors required to achieve this.


4.1.6. Ethics approval

Colmar Brunton worked with the Department to secure ethics clearance for the project through Bellberry Limited. Bellberry Limited is a national, private not-for-profit organisation providing streamlined scientific and ethical review of human research projects across Australia.

The final agreed specifications that received formal ethics clearance are included at Appendix B. Final ethics approval was granted on 4 January 2016.

60

4.1.7. Baseline data analysis (test vs. control)

The Department supplied Colmar Brunton with administrative data to be used to examine employment outcomes for both trial participants and a control sample of job seekers (i.e. a matched sample of non-participants).

The initial data sets for both trial participants and the control group contained a variety of details per jobseeker, including but not limited to the following data items:

• age

• gender

• disability/vulnerability

• primary psychiatric condition (and if relevant, secondary condition)

• commencement date

• length of unemployment

• future work capacity with intervention

• JSCI score

This data was used by Colmar Brunton in terms of both familiarising ourselves with the data set and with regards to defining the control group to be used for the final analysis.

Creating the control group

Being an observational study, a key part of this research involved the construction of a comparable control group. In recent times, matching has been seen as an increasingly popular methodology for assigning robust causal inference in a range of fields including statistics, medicine, econometrics and political science.

The broad concept of matching is to match as closely as possible each individual in the trial cell (i.e. the trial participants) with an equivalent individual from a pool of control subjects. This process ensures that a control group can be considered more closely comparable across certain characteristics that may confound causal inference (for example, for each Male aged 18 in the trial we would like to assign another Male aged 18 to our control group).

Each trial participant was matched to two possible subjects to act as a control group using a methodology based on Mahalanobis distance optimised to achieve a balance between key variables (GenMatch)78. Each participant was matched on the basis of:

• Age;

• Gender;

• Funding level;

7 Diamond, Alexis and Jasjeet S. Sekhon. 2013. “Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies.” Review of Economics and Statistics. 95 (3): 932–9458 Jasjeet S. Sekhon. 2011. “Matching: Multivariate Matching with Automated Balance Optimization in R” Journal of Statistical Software, May 2011, Volume 42, Issue 7

61

• Months unemployed;

• Education level;

• Aboriginal, Torres Strait Islander (ATSI) status;

• Culturally and linguistically diverse (CALD) status;

• Homeless status;

• Refugee status; and

• Ex offender status.

The pool of control subjects were based on geographic areas that had similar employment outcomes to the trial locations (with these being provided by the Department). Further exclusions were made on the basis of an employment outcome occurring before the beginning of the trial and trial participant who had drop out or been excluded.

Results and details of the trial to control groups can be found in Appendix G.


It was agreed during the planning and scoping stage of the project that the above mentioned administrative data would be useful with understanding outcomes at the macro level only (e.g. do trial participants show better employment outcomes than those in the control sample). As such, Colmar Brunton suggested that there was a strong need to understand how the program was being implemented and the degree to which participants – working with their DES providers – were feeling empowered to make their own decisions regarding addressing barriers to employment (via the funds made available to them in their Careers Account).

To better understand the administrative data, Colmar Brunton conducted a series of 1-hour, face to face depth interviews with both trial participants and DES providers participating in the trial. Colmar Brunton conducted a total of n=33 depth interviews with trial participants (n=12 Melbourne, n=11 Brisbane and n=10 Perth) and n=9 depth interviews with DES providers who have job seekers participating in the trial (n=3 Melbourne, n=3 Brisbane and n=3 Perth), between 9 and 18 February 2016.

Recruitment of depth interviews

To assist with recruitment of participants for the evaluation, the Department wrote to DES providers and trial participants advising that Colmar Brunton had been engaged to conduct an evaluation of the trial and encouraging their voluntary participation in the evaluation itself. A random stratified sample of trial participants were selected from Departmental data based on agreed quotas per trial location. The Department then contacted the relevant DES providers and asked for their assistance to recruit trial participants (in their area) to attend a 1-hour face to face interview at their offices.

Moderation

All interviews were conducted by senior, experienced moderators. All interviews were approximately 1 hour in duration to allow sufficient time to fully explore the key issues of interest. Participants were provided with a $25 EFTPOS card to cover time and travel costs incurred in attending these interviews.

62

Prior to the interviews being conducted, a verbal briefing session for all moderators was held, led by the core Colmar Brunton project team and attended by the Department project team. This was supported by the discussion guide, which included training notes for the moderators.

All interviews were audio recorded (with the permission of participants) to facilitate detailed qualitative analysis. In any instances where permission was refused for an audio recording of the session, the moderator used a combination of in-session detailed notes as well as a post-interview debrief template to adequately capture the content of the sessions for analysis purposes.

Interview and discussion guide design

Detailed interview guides were developed in close consultation with the Department, who approved the final guides before fieldwork commenced. Different discussion guides were developed for each group of interest (e.g. trial participants and DES providers). These guides were similar, but with different areas of focus. These discussion guides can be found in Appendix C and Appendix E.

At the end of the interview, both trial participants and DES providers were asked if they would be willing to be interviewed again (this time by phone) towards the end of the program (i.e. May 2016), with moderators noting the best contact details to reach them on for this follow up qualitative research.

4.1.9. Interim analysis and reporting

Upon completion of the initial qualitative research all of the researchers involved in this project were involved in a debrief based on their notes for each interview. The aim of the debrief was to understand both the overarching themes and any specific insights emerging from each individual interview.

An interim summary report/presentation was provided to the Department after these initial interviews with trial participants and DES providers were complete. This report was provided in MS PowerPoint format and summarised the core findings from the research conducted up to that point in time and was not a full analysis of the situation. The report was not intended to be circulated as an output document to a wider audience but was purely for the informational use of the Department project team.


Having interviewed trial participants and DES providers during the early stages of their participation in the trial, Colmar Brunton re-engaged these people for a second round of depth interviews towards the end of the trial (May 2016). These interviewers were conducted by telephone between 11 May and 15 June 2016. Every effort was made to try and arrange a second interview for each of the trial participants interviewed in Phase 1, and in total n=23 follow up depth interviews were completed with trial participants (n=9 Melbourne, n=9 Brisbane and n=5 Perth). In order to maximise response rates, trial participants were offered an additional $25 EFTPOS gift card (which was mailed to them) for completing the final telephone interview and all interviews were conducted by the same senior moderators from the initial interview.

In addition to interviewing trial participants, Colmar Brunton was able to arrange follow up depth interviews with all 9 DES providers interviewed in Phase 1 (n=3 Melbourne, n=3 Brisbane and n=3 Perth), which also occurred via telephone. As per the initial round of interviews, the follow up depth interviews were audio-recorded, with participant permission, to facilitate detailed qualitative analysis.

63

These follow up interviews allowed for further exploration of the trial participant’s experiences over the life of the trial (e.g. the degree to which the career account had assisted in removing barriers to employment, overall perspectives on the trial, whether participating had been a worthwhile experience, etc.). As with the initial qualitative phase, detailed interview guides were developed in close consultation with the Department for each group of interest (e.g. trial participants and DES providers). These discussion guides can be found in Appendix D and Appendix F.

4.1.11. Quantitative analysis of administrative data (trial participants vs. control group)

In July 2016 the Department provided Colmar Brunton with updated administrative data sets for both the trial participants and control group. This data was used to run a series of statistical tests to look at differences in the employment outcomes for both trial participants and the control group.

The Department also provided administrative data for each trial participant with regards to how the Career Account funds were spent during the course of the trial period. This information was provided to the Department by the DES providers and was listed at an individual participant level. This data was examined to determine average spend across trial participants and to take a deeper look at what the money was being spent on.

4.1.12. Final analysis and reporting

This final report presents the findings of all stages of the study, with the key findings and themes identified in this report reflecting a detailed analysis of all qualitative findings observed across all interviews conducted for the study, as well as the quantitative findings.

64

5. Appendix B: Ethics Application

Bellberry Ethics Application

Protocol TitleDisability Employment Services (DES) Youth Mental Health trial Evaluation

Investigators and QualificationsThis study is being undertaken by Colmar Brunton, a professional market and social research agency.

The project is being led by:

James WunschResearch DirectorColmar BruntonLevel 2, 9 Sydney AvenueCanberra, ACT, 2600.

Phone:02 62498566Mobile: 0422433231Email: [email protected]

Other Colmar Brunton staff working on this project (all of whom are qualified professional market and social research consultants) are:

Joan Young, CEO, Canberra Erin Cooper, Account Director, Canberra Joanne Davidson, Research Director, Adelaide Sarah Zanker, Account Manager, Adelaide Amy Bartlett, Account Manager, Canberra Hayley Pennock, Research Executive, Canberra

Site LocationThe study will be managed from Colmar Brunton’s Canberra office. Proposed fieldwork is a series of one-hour in-depth interviews with both program participants and DES providers, which will occur across two phases:

The first phase will be face to face interviews at the DES provider’s office in each of the three trial locations (Perth, Melbourne and Brisbane - February 2016).

A follow-up interview will be sought with each participant towards the end of the program (May/June 2016).

65

Purpose of StudyThe purpose of the study is to evaluate the DES Youth Mental Health trial, a new approach to funding provision for those with pre-existing mental health conditions to address barriers to employment. The study aims to explore whether this model delivers better employment outcomes for the trial cohort than a matched control sample. This information will help to evaluate the effectiveness of the trial program and provide guidance on how such approaches (compared to standard approaches) can potentially be refined or improved to assist young people with mental illness overcome barriers to their vocational and non-vocational goals.

Background InformationThe Department of Social Services (DSS) is responsible for the DES program, helping job seekers with a disability to gain and keep employment9. The DES Youth Mental Health Initiative is a part of the Government’s Youth Engagement Strategy to assist disengaged young people with a mental illness transition to work, as announced in the 2015-16 Budget10.

According to the Australian Bureau of Statistics (ABS), 26 per cent of people aged 16-24 reported experiencing a mental illness in 200711. In addition, people with a mental illness are more likely to be unemployed than those who do not report a mental illness12.

Earlier this year, then Assistant Minister for Social Services, Senator the Hon Mitch Fifield, emphasised the need to increase workforce participation by people with a disability in an address to the Disability Employment Australia Leaders’ Forum. In the same address, the Minister also flagged the need for an improved and innovative employment system ready for 2018 when current contracts will be reviewed.

The DES Youth Mental Health trial is a one-year trial designed to test an individually tailored participant-driven employment support model, the results of which will inform the design of the policy framework for employment services beyond 2018. In the trial, participants will be provided access to $5000 of funding through a ‘Career Account’ administered by DES providers. Trial participants will be allowed a high degree of choice in how the funds are spent in overcoming vocational and non-vocational barriers and meeting employment goals (one of the key goals of the trial is to increase perceptions of agency among these job seekers, allowing them to identify the types of services or treatments they feel will best help them gain employment).

The trial will operate in Employment Services Areas in Queensland, Victoria and Western Australia.

Once a participant has agreed to partake in the trial, DSS provides participants with a commencement letter which outlines the requirement for them to engage and complete trial-related evaluation activities. This may include involvement in surveys and interviews around their Career Account funds.

9 See https://www.dss.gov.au/our-responsibilities/disability-and-carers/programmes-services/disability-employment-services, accessed 25 September 201510 See http://budget.gov.au/2015-16/content/glossy/sml_bus/html/sml_bus-14.htm, accessed 25 September 2015 11 See http://www.abs.gov.au/ausstats/[email protected]/Latestproducts/4326.0Media%20Release12007?opendocument&tabname=Summary&prodno=4326.0&issue=2007&num=&view=, accessed 29 September 201512 See http://www.abs.gov.au/ausstats/[email protected]/Latestproducts/4159.0Main%20Features12014?opendocument&tabname=Summary&prodno=4159.0&issue=2014&num=&view=, accessed 29 September 2015.

66





http://budget.gov.au/2015-16/content/glossy/sml_bus/html/sml_bus-14.htm

https://www.dss.gov.au/our-responsibilities/disability-and-carers/programmes-services/disability-employment-services

Aims and ObjectivesThe proposed research aims to measure the effectiveness of the trial, including how participants use the funds and how well the model assists young people with mental illness overcome barriers to their vocational and non-vocational goals compared to standard approaches (that is, services or supports for the job seeker directed by the DES provider).

The objectives of the evaluation are to:

Identify whether trial participants achieve better employment outcomes than non-participants; Assess the extent to which the trial participants feel they have, and exercise, greater control

and choice in services or supports to assist them to find sustained employment; Identify whether, and how, the flexibility of a Career Account benefits trial participants through

improved employment outcomes; Identify and take into consideration concerns raised by trial participants and DES providers; If successful, identifying key risks in further expanding the model and mitigation measures to

address any risks that have been identified in the trial; Draw out possible implications for design of support for employment outcomes for young

people with mental health conditions in the future, which do not significantly increase the risk of inappropriate use of Government funding.

MethodologyThe research project will consist of both quantitative and qualitative phases. The key phases of this project have been outlined below:

Phase 1: Baseline review of Departmental administrative data

DSS has extensive administrative data that will be used to examine employment outcomes for both participants of the DES Youth Mental Health trial and a matched sample of non-participants (by age, location, gender and eligibility status).

At this stage of the project, we would receive the data sets for both trial participants (approximately n=200 across the three trial locations) and non-participants (these would be drawn randomly within the same parameters as the control group by factors such as age, gender, location, length of unemployment and condition type). These data sets – which will be de-identified before being provided to us for analysis - contain the following data items per job seeker:

age gender vulnerability/ demographic indicators commencement date in DES which DES program are they in – DMS or ESS their primary and secondary medical conditions length of unemployment current work capacity/future work capacity with intervention basic information in their Job Plan about the Career Account allowance type employment outcomes (placement, sustainability of placement)

This data will be reviewed in terms of both familiarising ourselves with the data set and in setting our defined hypothesis (e.g. mapping the indicators we would expect to vary as a result of trial

67

participation), which matches to our aims and objectives as previously defined in this document. The non-participants (control group) will only be a part of the study in a quantitative nature and will not be contacted for Phase 2 or Phase 3. Only trial participants will be invited to participate in the Phase 2 and Phase 3.

Please note that the data will be transferred from DSS to Colmar Brunton via a secure file transfer system or via email with all documents password protected. These documents will be stored on our secure network and will only be accessed by members of the project team who all work from password protected computers.

Phase 1 will commence upon HREC approval.

Phase 2: Initial Qualitative Fieldwork (n=45 1-hour face-to-face depth interviews)

The target audience for the study is job seekers who are involved in the DES Youth Mental Health trial as well as their DES providers. The project will entail recruitment of these participants for a series of qualitative in-depth interviews with members of the project team:

We will conduct a total of 36, one-hour face to face in-depth interviews with job seekers who have recently begun participating in the DES Youth Mental Health trial and are aged 18 years or over. It is planned that these interviews will be evenly split across three locations (Brisbane, Melbourne and Perth) by completing 12 interviews in each city. All participating job seekers will receive an incentive payment of $50 to cover their time and any travel costs, at the conclusion of the final interview. These in-depth interviews will be conducted in a private room at the DES provider facility – a location these job seekers will know and be familiar with. All trial participants were informed that they may be contacted throughout the trial to participate in research (please see the attachment ‘DES Youth Mental Health trial - Participant Letter’). It should also be noted that participation is voluntary and all participants will be given the option to decline the invitation.

We will also complete nine interviews with DES providers across the same locations (three interviews in each city). These interviews would also be one hour in duration and would take place at the DES provider’s office. As such, no incentive will be provided.

The DES providers will be contacted by DSS to ask for their participation and the job seekers will be contacted by their DES providers. Our experience from undertaking previous research with this cohort has provided us with a detailed understanding of the specific challenges associated with undertaking research with the unemployed. As such, we can be very flexible with interview times and can accommodate the last minute rescheduling of interviews if needs be.

Phase 3: Follow-up Qualitative Fieldwork (n=45 1-hour phone depth interviews)

Having interviewed program participants shortly after their entry to the program, we hope to re-engage these participants for a second round of depth interviews towards the end of the program (May 2016) through a series of a further 45 one-hour depth interviews (36 with the participants we interviewed in Phase 2, and nine with the DES providers we interviewed in Phase 2).

This follow-up interview will allow us to explore the program participant’s experiences over the life of the program (e.g. whether monies expended under the Career Account have assisted in removing barriers to employment, their perspectives on the program overall, whether participating has been a worthwhile experience, etc.).

68

For both Phase 2 and Phase 3, the following will apply: Maximum one hour interview sessions; Senior experienced moderators to conduct the interviews; A verbal briefing session for all moderators will be held prior to each phase; Interviews will be audio recorded (or notes / debrief templates utilised if permission to record

not granted) solely for note taking and analysis purposes (this will not be shared with DSS); Participants are not compelled to participate in either of the research phases, and are free to

opt out of the study at any time; and In the case of a participant becoming upset during an interview, our interviewers will be

briefed in appropriate protocols to ensure they can either take a break or discontinue the interview and offer the participant a range of support channels to access if required (e.g. their DES contact, or phone counselling services such as Lifeline and Beyond Blue).

Interview and discussion guide design

A detailed interview guide will be developed in close consultation with DSS, who will approve the final guide before fieldwork commences. There will be a different discussion guide for each group of interest (e.g. one for participants and one for DES providers) as each will have different areas of focus. While the guides may use a similar framework for the interviews, the language used, phrasing of questions and the perspective will be appropriate to each participant type.

The table below summarises the nature of the content we propose to include for each type of participant:

Areas of discussionTrial

participants

DES providers

Establishment of the interview

Background information on the participant (including standardised demographic information)

-

Background information on the DES provider office (including case load etc.) -

Barriers to employmentExample questions include: What do you feel are the barriers you face in terms of finding

employment in this area? How has your DES provider/s helped you to address these barriers, if at

all?

Understanding of and attitude towards the trial programExample questions include:

How did you find out about the trial? Can you tell me what you know about the trial? How did you find out this information? When did you agree to be a part of the trial? What appealed to you in terms of participating in the trial? How was this

different to the type of support you already received?

Appeal / benefits of the trial programExample questions include:

What are the good things about the trial? And what makes them good things?

Detailed examination of the decision making process in relation to

69

Areas of discussionTrial

participants

DES providers

how the Career Account funds are being spentExample questions include:

Can you take me though what you have done since joining the trial? How is this different to what you were doing before? Tell me about how you plan to spend the $5000 in the Career Account?

o How did you decide this was the best way to use these funds?o Did you make this decision?o Did anyone influence you in deciding how the funds were to be

spent? How so?

Costs / barriers / concerns of the trial programExample questions include: What are the not so good things about the trial?

How is money being used differently for trial participants to non-trial job seekers -

Recommendations for improving the program -

Thanks & close(and reminder about the duration of the process and intention to recontact trial participants by phone towards the end of the trial)

Phase 4: Post-program analysis of Departmental administrative data

At Phase 4, we will obtain the updated sample and control administrative data sets from DSS (early July 2016). We will run a series of statistical tests on the data sets to determine:

Whether there have been any statistically significant changes in employment outcome measures between baseline and this end of program data point (for both groups, both on a placement basis and duration of placement basis);

Whether any changes in employment outcomes are consistent across both trial and control groups, or are different (and indeed, the scope of the difference if one exists);

Whether there are any other significant changes in employment outcomes across the two groups by other factors such as age, gender, location or time since initial registration with DSS, etc.

AdvertisingNo public advertising is planned.

Research Involving Ionising RadiationNot relevant.

Inclusion Criteria

All of those who are participating in the trial and who are not excluded for reasons outlined below will be offered participation in the research.

Exclusion Criteria

70

People who are participating in the trial who have indicated they do not wish to participate in research will be excluded, along with those who have recently been approached for other research and evaluation by DSS.

Withdrawal Criteria

Participants may choose not to participate at the point of initial recruitment, and can also withdraw from the study at any time.

Other ParticipantsWhile not being targeted specifically for the project, it is likely that at least some of the research participants may identify as being of Aboriginal or Torres Strait Islander decent. Colmar Brunton is experienced in research with Aboriginal and Torres Strait Islander people and always does so in accordance with the requirements stated in Chapter 4 of the NHMRC National Statement on Ethical Conduct in Human Research (2007).

EfficacyNot applicable.

AmendmentsThe Investigator(s) will inform the HREC, and seek its approval, of amendments to the protocol.

Data Analysis and Reporting of ResultsUpon completion of the research, the researchers involved in this project will debrief based on their notes for each interview. This will occur twice after each stage of qualitative research. The aim of this debrief will be to understand both the overarching themes and any specific insights emerging from each individual interview. Colmar Brunton will then commence with full reporting of the interviews.

No information provided by Colmar Brunton to DSS will enable the identification of any participant in the study.

Mid-research reporting

This research study spans the better part of a year with a final report due in September 2016. Colmar Brunton recommends that a mid-research debriefing process is implemented so that DSS can maximise the utility of the research by obtaining some initial early in 2016 after the completion of the initial phases of both quantitative and qualitative research.

We propose to provide a summary report to DSS and conduct a debrief session. This report will be in MS PowerPoint format rather than a published report in MS Word and will summarise the core findings from the research conducted to date. This is for two reasons: this report is not intended to be circulated as an output document to a wider audience but is purely for the informational use by DSS’ project team; and it will only contain interim information as to the success of the program and is not a full analysis of the situation.

Methodology report

71

At the completion of the project, a methodology report will be provided as a separate document to the written report. It will be in MS Word format and issued in both draft and final versions allowing for Departmental feedback.

The methodology report will contain all relevant information regarding the design of the project and the nature of the fieldwork completed including:

Project developmento Project and methodology overviewo A description of changes made to discussion guides and other materials

Main fieldwork processo Data collection and control measureso Contact incidence and response rateso Actions taken to maximise response and retention rates

Data processingo Data managemento Coding

Any difficulties encountered during the project Methodological recommendations A copy of the discussion guides and any other materials used.

Please note that the methodology report will form a part of the final deliverables and will supplement the final report. It is designed to provide details of the project so as to be transparent about the process that occurred.

Collection Control Register

A Collection Control Register will be developed and maintained throughout the course of the project. At the completion of the project, this register will be provided to DSS as a full record of all activity and communications with the individuals provided in the original lists supplied by DSS.

This will be in Excel format and will include unique identifiers (as supplied by DSS), results of all contact attempts, actions taken and outcomes of the contact.

Draft and final written report

At the completion of all stages of the research process, a full written report in MS Word will be provided in draft format. DSS will have the opportunity to provide feedback on this draft report and ask for any additional areas to be addressed. Colmar Brunton will then work to finalise the report for DSS.

This report will contain a full analysis of the entire research project including the specific addressing of each of the five core research objectives and conclusions as drawn by Colmar Brunton. Supporting evidence will be provided for each conclusion drawn and additional findings that emerge from the research would also be identified.

The final research report will be used by DSS for policy development and program improvement. Dissemination and publication will be at the discretion of the Minister for Social Services.

Summary of deliverables

72

The electronic outputs that will be provided to DSS include:

1. A project plan2. Project status reports3. Draft and final versions of all data collection instruments4. Copies of written briefing notes provided to interviewers / moderators5. A collection control register, including unique identifiers6. Draft and final versions of the Methodology Report7. A mid-research debrief report and verbal debrief8. Draft and final versions of the Research Report9. Face-to-face presentation of findings to DSS (optional)

Timeframe

Project task Timing

Ethics clearance application submitted By 18 November 2015

Ethics committee approval 11 – 14 December 2015

Recruitment of participants by DES providers From 27 January 2016

Provision of trial and control data samples from administrative data 11 January 2016

Discussion guides for Qualitative Fieldwork Phase 1 finalised 27 January 2016

Qualitative fieldwork period – Phase 1 (n=45 1-hour depth interviews, face to face) 1 – 28 February 2016

Mid-research project debrief provided 15 March 2016

Discussion Guides for Qualitative Fieldwork Phase 2 finalised 30 April 2016

Qualitative fieldwork period – Phase 2 (n=45, 1-hour depth interviews, phone) 1 – 29 May 2016

Provision of trial and control data samples from administrative data 3 July

Draft written report 3 August 2016

Draft methodology report 3 August 2016

Feedback received from DSS on the draft reports 24 August 2016

Final written report ^ 12 September 2016

Final methodology report ^ 12 September 2016

Presentation / Discussion (if required) TBC

Contract completion 12 September 2016

73

Other Ethics Committees to Which This Study Has Been Submitted

None

Other Relevant Information

A draft participant consent form for job seekers has been prepared and is included at Attachment A and a draft participation consent form for providers is included at Attachment B.

Student Research

Not applicable.

74

Attachment A

DSS0004 IN-DEPTH INTERVIEW PARTICIPANT CONSENT AND ACKNOWLEDGEMENT OF REIMBURSEMENT FORM

To be read out to participant prior to interview, with verbal consent to be provided for each point

I am aged 18 years or older

I understand that participation in the study is voluntary

I consent to take part in this research study, which involves taking part in a discussion with a researcher from Colmar Brunton Social Research about my experience of the Disability Employment Services (DES) Youth Mental Health trial

I consent to both discussions with the researchers being audio-recorded for analysis purposes

I understand that the discussion is confidential and that my details will not be shared outside the research team. In particular, no information will be passed on to my DES provider

I am aware that no information I provide will be attributed to me in any way in the reporting; all information will be made anonymous

I understand that I am free to withdraw at any time from the research (and do not need to give a reason for this). If I decide to withdraw from the study, I agree that the information collected about me will not be used in the research

I acknowledge that any monies or gratuities received by me today ($50 cash) regarding this project are accepted as compensation for expenses incurred

I agree that I am the person whose signature appears below

Confirmation Date

Participant Name:Participant signature:Researcher’s Name:I [researcher] confirm that I have read out each statement above to the participant and have obtained verbal agreement for each statement, and that the participant has understood the information and any explanations given.Signature of Researcher:Please note: A signed copy of this letter will be provided to all participants.

Attachment B

75

DSS0004 IN-DEPTH INTERVIEW PROVIDER CONSENT FORM

To be read out to the DES provider prior to interview, with verbal consent to be provided for each point

I am an employee of a Disability Employment Services (DES) provider that is participating in the DES Youth Mental Health trial

I understand that participation in the study is voluntary

I consent to take part in this research study, which involves taking part in a discussion with a researcher from Colmar Brunton Social Research about my experience of the DES Youth Mental Health trial

I consent to both discussions with the researchers being audio-recorded for analysis purposes

I understand that the discussion is confidential and that my details will not be shared outside the research team

I am aware that no information I provide will be attributed to me in any way in the reporting; all information will be made anonymous

I understand that I am free to withdraw at any time from the research (and do not need to give a reason for this). If I decide to withdraw from the study, I agree that the information collected about me will not be used in the research

I agree that I am the person whose signature appears below

Confirmation Date

DES Provider Name:DES Provider signature:Researcher’s Name:I [researcher] confirm that I have read out each statement above to the DES provider and have obtained verbal agreement for each statement, and that the DES provider has understood the information and any explanations given.Signature of Researcher:Please note: A signed copy of this letter will be provided to all DES providers.

76

6. Appendix C: Initial Participant Discussion Guide

QMS QUAL MODERATOR DISCUSSION GUIDE

Project No.: DSS0004 Project Name: Youth Mental Health trial EvaluationMain Client Service Contact: Amy BartlettClient Service Project Leader: James WunschOther Client Service Team Members: Joan Young and Hayley PennockIssue Date: 02 February 2016 Interview Type: Depth interview

GENERAL INTRODUCTION 3 MINS

• Hi, my name is X from Colmar Brunton Social Research, as you may already know we are a

social research company who have been asked to conduct this research on behalf of the

Department of Social Services.

• Research is all about understanding how people feel, what they experience and their

perceptions, awareness and knowledge.

• Today we are going be discussing your perceptions and experiences in relation to the Youth

Mental Health trial that you are currently a part of.

• There are no right or wrong answers. We just really want to hear about how you really feel.

• As we just discussed, all feedback will be treated in the strictest of confidence and no

individuals will be identified in the final report we produce.

• Our conversation will last for about 60 minutes.

• I have a short form I would like to quickly go over with you before we begin. COMPLETE

CONSENT FORM.

• Any questions before we begin?

BACKGROUND INFORMATION 5 MINS

Let’s begin by talking about living here in [LOCATION].

• How long have you lived here for?

• What are all the good things about living here?

• And what about things that are less positive about living here?

77

BARRIERS TO EMPLOYMENT 5 MINS

Now let’s chat about what it is like trying to find a job…

• How easy or difficult do you feel it is to find a job in this area?

• [IF DIFFICULT] And what makes it difficult?

[PROBE IF NECESSARY]o Employers not being willing to hireo Not having the right skills or qualificationso Lack of job opportunities in generalo Lack of jobs you find appealingo Managing my mental health appropriately

• How has your DES provider/s helped you to address these difficulties, if at all?

TIMELINE 25 MINS

Now we are going to draw a line on this paper and the first point of the line is the time when we first heard about the trial. The last point of the line is where we are today – so let’s write on the line each of the different things that have happened at each point of the way.

• To begin with, could you write down the date that you heard about the trial?

• And at the end of the line we will write down today’s date.

• Now let’s write down all of the points in the middle… after the first point on the line what

happened next, and after that, and after that… ONLY PROBE IF NOT DRAWN ON

TIMELINE

o At what point did you decide to participate in the trial?o And when was your first meeting with your service provider about the trial?o When was the first time you used some of the money? And when was the next time?

ONCE THEY HAVE FINISHED THEIR TIMELINE CHECK THAT EVERYTHING IS COVERED, INCLUDING ‘AWARENESS’, ‘TRIAL PARTICIPATION ACCEPTANCE’, ‘INITIAL INTERVIEW’ AND ‘SUBSEQUENT ACTIONS’

THEN ASK THE FOLLOWING QUESTIONS OF EACH POINT

• What happened?

78

• What did you think, know, feel and do? What is the best word to describe how you felt at that

particular point in time?

• What are all the things that influenced you at that specific point in time?

• Who were the people and what did they say and do? [IMPORTANT TO PROBE THE

DECSION MAKING PROCESS ON WHAT THE CAREER ACCOUNT FUNDS WERE USED

IN DETAIL – WE ARE VERY KEEN TO UNDERSTADN THE DEGREE TO WHICH

PARTICIPANTS MADE THEIR OWN DECSIONS OR WERE INFLUENCED BY OTHERS]

IF NECESSARY, SPECIFIC PROBES INCLUDE:

• Could you tell me how you first heard about the trial?

• And what were your initial thoughts when you first heard about it? Where you positive or

negative? How come?

o In what ways did the trial sound different to the way your DES provider had worked with you in the past?

• Why did you decide to participate in the trial?

o What appealed to you about the trial?

o Did you have any key questions or concerns about participating in the trial?

• What sort of things have you used the money for so far?

[CONFIRM IF THESE ARE INTERNAL TO THE PROVIDER OR EXTERNAL]

o How did you decide to use the money on that/those things specifically?

o [IF HAVEN’T USED ANY MONEY] What sort of things do you plan on using the money for?

• And how are these things different to what you would usually do (or have done previously)

with your job service provider?

APPEAL / BENEFITS AND COSTS / BARRIERS / CONCERNS 10 MINS

Thinking now about the whole experience and everything we have just discussed…

• How would you describe the experience of being involved in the trial?

79

• What have been the good things or the benefits about participating in the trial so far?

[PROBE IF NECESSARY]

o Being able to spend the money in any wayo Having the power to make own decisions

• Did you expect these positive things from the start?

o Have any of these positive things come as a surprise to you?

• What have been the not so good things about the trial so far?


o Not knowing what to spend the money ono Not feeling comfortable with being able to make decisionso Provider controlling how the money is spento It requires more time and effort than usual

• Did you expect these negative things at the start? Have any of these negative things come as

a surprise to you?

• How have you or your circumstances changed as a result of participating in the trial so far?

o [IF MULTIPLE CHANGES IDENTIFIED] Of these, which would you say was the most significant change for you or your circumstances?

o Do you feel more likely to obtain work from having participated in the trial? How come? [PROBE – NEW SKILLS, INCREASED CONFIDENCE, IMPROVED HEALTH, OTHER]

o Do you feel having the career account is a better way of getting the help you need to obtain work than how you have been helped by job service providers in the past? How come?

RECOMMENDATIONS FOR IMPROVING THE PROGRAM 10 MINS

ASK THE FOLLOWING QUESTIONS OF THOSE WHO HAVEN’T FOUND A JOB YET

• Do you feel more ready to find employment as a result of participating in the trial? How come?

• And do you think the trial will be helpful in assisting you to find employment? How come?

• I know that the trial is still going - to date, what was the most significant change you have

experienced from participating in the trial?

• At this point of the trial do you feel that it has provided what you originally thought it would?

[IF YES]

80

o And what was that?[IF NO]

o What has made the experience different from what you were expecting?

• How would you rate the overall effectiveness of the trial in helping you to develop skills and/or

find employment?

• With what you know now, would you have originally participated in this trial?

o Why / why not?

• What do you feel has been the best thing or the biggest benefit of participating in the trial?

• Based on your experience, would you recommend other job seekers participate in the

program (or a similar program)? How come?

• Let’s imagine you were going to do this all again from the start, what would you do differently?

What have you learned that would have been good to know right back at the beginning?

81

ASK THE FOLLOWING QUESTIONS OF THOSE WHO HAVE ALREADY FOUND A JOB

• Did you feel that the trial made you more ready to find employment? How come?

• And do you think the trial was helpful in assisting you to find employment? How come?

• What was the most significant change you have experienced from participating in the trial?

• Has the trial provided what you originally thought it would?

o [IF YES] And what was that?o [IF NO] What has made the experience different from what you were expecting?

• How would you rate the overall effectiveness of the trial in helping you to develop skills and

find your current job?

• With what you know now, would you have originally participated in this trial?

o Why / why not?

• What do you feel has been the best thing or the biggest benefit of participating in the trial?



• Do you have any suggestions on how the program could be improved?

WRAP UP 2 MINS

• We’ve covered a lot today and we really value your input – thank you. Do you have any final

comments to add?

• That’s the end of this first interview. As a thank you, we’d like to give you this pre-paid visa

card with a balance of $25.

• I believe that we mentioned when you agreed to take part in this research that we would like

to chat with you again in about 3 months’ time. We would phone you to arrange a second

interview by phone at a time convenient to you. It will take around 45 minutes to an hour and

you will receive another $25 pre-paid visa card; can you please confirm that you are happy for

us to do so?

RECORD CONTACT PHONE NUMBER: ___________________________________________

• I just need to state that as this is market research, it has been carried out in compliance with

the Privacy Act 1988 / information provided will only be used for research purposes.

• Also just to remind you, I am from Colmar Brunton and if you have queries about this

research or how it is being conducted you can call the Market Research Society’s toll-free

Survey Line on 1300 364 830 or CBR on (1800 555 145).

82

• Issue incentives - ensure respondent has signed the consent and reimbursement form.

83

7. Appendix D: Follow up Participant Discussion Guide


Project No.: DSS0004 Project Name: Youth Mental Health trial EvaluationMain Client Service Contact: Amy BartlettClient Service Project Leader: James WunschOther Client Service Team Members: Joan Young and Hayley PennockIssue Date: 05 May 2016 Interview Type: Follow up depth interview


• Hello again, it is X from Colmar Brunton Social Research, as we mentioned last time spoke

we are a social research company who have been asked to conduct this research on behalf

of the Department of Social Services.

• Just the same as last time we spoke we are going to be discussing your perceptions and

experiences in relation to the Youth Mental Health trial that you are currently a part of.


perceptions, awareness and knowledge.

• There are no right or wrong answers. We just really want to chat about what has happened

since we spoke back in February.

• All of the points that we went through on the form last time still apply and all feedback will be

treated in the strictest of confidence and no individuals will be identified in the final report we

produce.


• Are there any questions before we begin?

TIMELINE 25 MINS

When we spoke back in February we created a timeline of your experience with the program from when you first heard about it through to the day of our chat. What we are going to do today is continue this timeline from the last time we spoke up until today. So while we talk about everything that has happened in relation to the trial during that time I will write it down on my piece of paper.

• Let’s talk about where things were at the last time we spoke….what was the situation then?

84

• What was the first thing that happened/changed after we spoke?

• And then what happened?

• And after that, and after that… CONTINUE PROBING UNTIL ALL POINTS ARE ON THE

TIMELINE.

PROBE THESE QUESTIONS IF NOT MENTIONED OR NOT CLEAR:o How many meetings have you had with your provider during this time? And have

these been your usual appointments or special ones because of the trial?o When was the first time you used some of the money? And when was the next time?

ONCE THE TIMELINE IS FINISHED READ BACK THE POINTS WRITTEN DOWN AND CHECK THAT EVERYTHING IS COVERED.

IF NECESSARY PROBE

• What happened?

• What did you think, know, feel and do?

• What is the best word to describe how you felt at that particular point in time?

• What are all the things that influenced you at that specific point in time?

o Who were the people and what did they say and do? [IMPORTANT TO PROBE THE DECSION MAKING PROCESS ON WHAT THE CAREER ACCOUNT FUNDS WERE USED FOR IN DETAIL – WE ARE VERY KEEN TO UNDERSTAND THE DEGREE TO WHICH PARTICIPANTS MADE THEIR OWN DECSIONS OR WERE INFLUENCED BY OTHERS]

IF NOT COVERED

• What sort of things have you used the money for so far?


o How did you decide to use the money on that/those things specifically?

o [IF STILL HAVEN’T USED ANY OF THE MONEY] What sort of things do you plan on using the money for? When do you plan on doing these?

• And how are these things different to what you would usually do (or have done previously)

with your job service provider?

• How have you felt when you have used some of the Career Account money?

• What have been the good things about using the Career Account money?

• What, if any, have been the not so good things about using the Career Account money?

• How easy or hard has it been to use the money?

85

• What made it harder? What made it easier? What needs to change to make it easier?


Thinking now about the whole experience from when you first heard about the trial and everything that happened up until our first chat and everything that has happened since…

• How would you describe the experience of being involved in the trial?

• Is it the same or different to how you would have described it last time we spoke?

o [IF DIFFERENT] How is it different to last time?

• What were your expectations about what would happen?

o How is what has actually happened different?

• What have been all the good things or the benefits about participating in the trial?

[PROBE IF NECESSARY]o Being able to spend the money in any wayo Having the power to make own decisionso Starting some sort of studyo Getting a job o Improvements with family relationships/family lifeo Being more confident and having greater self-esteem o Feeling mentally healthier

• What have been all the not so good things about the trial so far?

[PROBE IF NECESSARY]o Not knowing what to spend the money ono Not feeling comfortable with being able to make decisionso Provider controlling how the money is spento It requires more time and effort than usualo Anxiety about doing things that are outside of your comfort zone

• Prior to participating in the trial what concerns did you have if any?

o Now that you have been participating in the trial for a while do you still have any of these concerns? [IF YES] Do you have any suggestions on how your provider or the department could help you or others with similar concerns to overcome these?

• How have you or your circumstances changed as a result of participating in the trial?

o [IF MULTIPLE CHANGES IDENTIFIED] Of these, which would you say was the most significant change for you or your circumstances?

86

o [IF HAVEN’T YET FOUND A JOB] Do you feel that you are closer to finding employment now? Has the trial made you more or less likely to get a job? In what ways? [PROBE – NEW SKILLS, INCREASED CONFIDENCE, IMPROVED HEALTH, OTHER]

• How useful or not useful is the career account in helping you to get work? In what ways is it

useful? In what ways is it not useful? What would make it more useful?

THE CAREER ACCOUNT AMOUNT 10 MINS

Let’s talk briefly about the career account and the money that you have had access to…

• How much money was available to use? And how much have you used so far?

• Why have you spent that amount of money?

o [PROBE IF NECESSARY]o What encouraged you to spend this amount of money?o What discouraged you to spend this amount of money?

[IF HAVEN’T SPENT ALL OF THE MONEY]

o Do you plan on spending the rest of the money?

• In what ways might the experience have been different if there was less money to spend?

[IF NO]o What do you think would have been different?

• Who or what has influenced how you have spent the money?

• And who has ultimately made the decisions on how to spend the money?

• Do you feel that you have been able to spend the money on what you have wanted to?

[IF NO]o What would you have ideally spent the money on?o How come you didn’t spend the money on these things?

• Have you had any difficulties in using all of the money available?

[IF YES]o How come?

[IF NO]o Was it the right amount of money, or would you have preferred more?

• In what ways might the experience have been different if you had more time to spend the

money?

87

• What, if anything, would you change about the career account money (e.g. amount, what you

could spend it on, how you could access/spend it etc.)? How come?

• In what ways might the program be been abused?

[PROBE IF NECESSARY]o What makes it easier for the program to be abused?o What makes it more difficult for the program to be abused?

RECOMMENDATIONS FOR IMPROVING THE PROGRAM 10 MINS

ASK THE FOLLOWING QUESTIONS OF THOSE WHO HAVEN’T FOUND A JOB YET

• Do you feel more ready to find employment as a result of participating in the trial? How come?

• And do you think the trial will be helpful in assisting you to find employment? How come?

• I know that the trial still has about a month before it finishes but so far, what has been the

most significant change you have experienced from participating in the trial? How do we feel

about this change? What difference has it made?


[IF YES]o And what was that?

[IF NO]o What has made the experience different from what you were expecting?


find employment?

• What if anything have your family, friends and/or service provider said about you participating

in the trial?

o How does this make you feel?

• Knowing what you know now, would you have originally participated in this trial?

o Why / why not?

• What has been the best thing or the biggest benefit of participating in the trial?





• What suggestions do you have for how the program could be improved?

ASK THE FOLLOWING QUESTIONS OF THOSE WHO HAVE ALREADY FOUND A JOB

88

• To what extent did the trial help us to get the job we have now? In what ways?

• What was the most significant change you have experienced from participating in the trial?

How do we feel about this change? What difference has it made?


[IF YES]o And what was that?

[IF NO]o What has made the experience different from what you were expecting?


find employment?

• What if anything have your family, friends and/or service provider said about you participating

in the trial?

o How does this make you feel?

• Knowing what you know now, would you have originally participated in this trial?

o Why / why not?

• What has been the best thing or the biggest benefit of participating in the trial?





• What suggestions do you have for how the program could be improved?

WRAP UP 2 MINS


comments to add?

• That’s the end of the interview. As a thank you, we’d like to send you a pre-paid visa card with

a balance of $25. What would be the best address to send it to?

RECORD ADDRESS:________________________________________________






89

90

8. Appendix E: Initial Provider Discussion Guide


Project No.: DSS0004 Project Name: Youth Mental Health trial EvaluationMain Client Service Contact: Amy BartlettClient Service Project Leader: James WunschOther Client Service Team Members: Joan Young and Hayley PennockIssue Date: 02 February 2016 Interview Type: Follow up depth interview


• Hi, my name is X from Colmar Brunton Social Research, as you may already know we are a

social research company who have been asked to conduct an evaluation of the Youth Mental

Health trial on behalf of the Department of Social Services.

• As you are probably already aware, we are also interviewing job seekers who are

participating in the Youth Mental Health trial. To build on this feedback we are also keen to

explore your perspective and thoughts on the trial, especially the aspects that have worked

well and possible improvements.


perceptions, awareness and knowledge. There are no right or wrong answers. We just really

want to hear about how you really feel.

• As we just discussed, all feedback will be treated in the strictest of confidence and no

individuals will be identified in the final report we produce.


• I have a short form I would like to quickly go over with you before we begin. COMPLETE

CONSENT FORM.

• Any questions before we begin?

BACKGROUND INFORMATION AND EMPLOYMENT 5 MINS

• To begin with can you please give me a brief description of your role? [CONFIRM: Does the

provider work directly with job seekers and assist with job placements or are they employed in

another capacity, e.g. site manager]

91

• What is the local job seeker market like here in [LOCATION]? Are there opportunities

available for those seeking work?

o [YES] How do these opportunities differ for job seekers aged 24 and under who have a mental health issue?

• How easy or difficult do you feel it is for job seekers aged 24 and under who have a mental

health issue to find a job in this area?

• [IF DIFFICULT] And what makes it difficult?


o Employers not being willing to hireo Job seekers not having the right skills or qualificationso Lack of job opportunities in generalo Lack of jobs that job seekers find appealingo Job seekers struggling to manage their mental health appropriately

• Prior to the trial how did you help these job seekers to address these difficulties?

• Since the trial started have you changed how you help these job seekers to address these

difficulties?

o [YES] Has this just been for participants of the trial or has this been for all job seekers in a similar situation?

AWARENESS AND UNDERSTANDING OF THE TRIAL 5 MINS

• How did you find out about/hear about the trial?

• And what were your initial thoughts when you first heard about it?

o In what ways did the trial sound different to the way you would usually work with a job seeker?

• How many job seekers did you ask to participate in the trial?

o What did you tell them about the trial?o What were the key selling points?o Did the job seekers have any questions or concerns about participating in the trial?

• And how many job seekers from this location are participating in the trial?

92


• Thinking about all the job seekers that you have participating in the trial, what have been the

good things or the benefits about the trial so far?


o Job seekers being empowered o Improved employment outcomes

• And what have been the not so good things or the challenges about the trial so far?


o Having to learnt the rules and requirements of the trialo Job seekers not knowing what to spend the money ono It requires more time and effort than usual

• Where you expecting these challenges?

[IF YES]

o What had you done to prepare for them?

• Have you noticed any positive changes in job seekers as a result of their participation in the

trial?

[IF NECESSARY PROBE]

o Enhanced confidence o Greater ability to communicate o Improved Health

• Has participation in the trial helped job seekers to improve their skills?

[IF YES]

o What skills have they gained from participating?o What skills have they learned that they would not otherwise have gained?o Do you think that these job seekers could have acquired these skills without being a

part of the trial?

[IF NO]

o Do you think job seekers are likely to gain skills through participation in the trial?

93

TRIAL DETAILS AND THE $5000 10 MINS

• Thinking again about all the job seekers that you have participating in the trial, what has been

your role in the process?


o Assisting to write a job plano Helping make decisions on how the $5,000 should be spent

• Do you feel that the job seekers have been capable of determining what things would help

them become more ready for employment?

o What makes you say that?

• What sort of things have they spent money on?


• Have they needed guidance as to how the monies would be best spent?

• Do you feel that these decisions on how to spend the money will likely lead to better

employment outcomes?

• How do you think the $5,000 account would have been best spent for these job seekers?

How come?

• Did the job seekers use this funding instead of, or in addition to other funding? How come?

9. JOB SEEKER AND TRIAL OUTCOMES 10 MINS

• Have any of your job seekers participating in the trial already found employment?

[IF YES ASK THE FOLLOWING QUESTIONS]

o Do you feel that the trial made the job seeker/s more “job ready”? How come?o And do you think the trial was helpful in assisting the job seeker/s to find

employment? How come?

[IF NO ASK THE FOLLOWING QUESTIONS]

o Do you feel that the trial is assisting job seekers to be more “job ready”? How come?o And do you think the trial will be helpful in assisting job seekers to find employment?

How come?

• What has been the most significant change you have seen in job seekers participating in the

trial?

94

• How would you rate the overall effectiveness of the trial in helping job seekers to develop

skills and find employment?

• For the job seekers what do you feel has been the biggest benefit of participating in the trial?

• And for yourself, what do you feel has been the biggest benefit of participating in the trial?

• What has been the biggest challenge of the trial for job seekers?

• And what has been the biggest challenge of the trial for yourself?

DETERMINING SUCCESS AND RECOMMENDATIONS 10 MINS

• Has the trial provided what you originally thought it would?

[IF YES]

o And what was that?

[IF NO]

o What has made the experience different from what you were expecting?

• How would you measure the success of the trial?

• In your opinion, how successful has the trial been to date?

o What have been the successes of the trial?o What made these a success?o What were the key factors in achieving these successes?

• At this point of time in the trial do you feel that it has delivered on its original intent?

[IF NO]

o What were they key reasons this original intent was not achieved?o Are the outcomes from the trial still valuable? How so?

• Have there been any unforseen or unintended outcomes of the trial (either positive or

negative)?

• How has the trial caused you to reflect on your standard service delivery approaches?

• As a result of the trial, are you likely to change the way you operate in the future or seek

changes to operating guidelines as provided by the department?

• Were there any other factors outside of the trial that either contributed to, or reduced the

impact of, the program?



95

• Do you have any suggestions on how the program could be improved?

WRAP UP 2 MINS


comments to add?

• That’s the end of this first interview, thank you so much for your time, it is greatly appreciated.

• I believe that we mentioned when you agreed to take part in this research that we would like

to chat with you again in about 3 months’ time. We would phone you to arrange a second

interview by phone at a time convenient to you. It will take around 45 minutes to an hour; can

you please confirm that you are happy for us to do so?

RECORD CONTACT PHONE NUMBER: ____________________________________________






• Ensure respondent has signed the consent form.

96

10.Appendix F: Follow up Provider Discussion Guide


Project No.: DSS0004 Project Name: Youth Mental Health trial EvaluationMain Client Service Contact: Amy BartlettClient Service Project Leader: James WunschOther Client Service Team Members: Joan Young and Hayley PennockIssue Date: 20 May 2016 Interview Type: Follow up depth interview


• Hello again, it is X from Colmar Brunton Social Research, as we mentioned last time spoke

we are a social research company who have been asked to conduct this research on behalf

of the Department of Social Services.

• Just the same as last time we spoke, we are also interviewing the same job seekers who are

participating in the Youth Mental Health trial as well as providers such as yourself.


perceptions, awareness and knowledge. There are no right or wrong answers. We just really

want to hear about how you really feel.

• All of the points that we went through on the form last time still apply and all feedback will be

treated in the strictest of confidence and no individuals will be identified in the final report we

produce.

• Just as last time our conversation will last for about 60 minutes.

• Are there any questions before we begin?

BACKGROUND INFORMATION AND EMPLOYMENT 5 MINS

• How would you describe the local employment market?

• Has the local employment market changed in any way over the last three months?

o [IF YES] How has it changed? Has this changed the opportunities for job seekers aged 24 and under who have a mental health issue?

97

• How easy or difficult is it for job seekers aged 24 and under who have a mental health issue

to find a job in this area? What makes it easy/difficult?


o Employers not being willing to hireo Job seekers not having the right skills or qualificationso Lack of job opportunities in generalo Lack of jobs that job seekers find appealingo Job seekers struggling to manage their mental health appropriately

EXPERIENCES WITH THE TRIAL 25 MINS

Last time we spoke about our overall experience with the trial - the positives and negatives and the things that could be improved. Today we are going to go over these areas again taking into account our experiences over the last three months.

• Overall what are your impressions of the trial?

o What are all the positive things or benefits? In what ways are these positive/benefits? o What are all the negative things? In what ways are these negative?

• How about for job seekers? What has their experience been?

• What have been the good things/benefits for job seeker? In what ways have these been

positive for job seekers?

• In what ways, if any, have jobseekers experienced… [PROBE IF NOT ALREADY COVERED]

o More Empowerment o Improved employment outcomeso Increased participation in training, work experience and skills development o Increased job readinesso Greater confidence and composure of job seekerso Improved communication skills and abilitieso Improved mental health o A more positive outlook about their future

• What about the bad things/negatives? In what ways have these been negative for job

seekers?

o Job seekers being anxious or stressed about the money (and how they spend it)o Job seekers not feeling comfortable with making decisions for themselveso Job seekers not knowing how to spend the money

• How about for this organisation, what has your experience of the trial been?

98

• What have been the good things/benefits for you? In what ways have these been positive for

you?

• In what ways, if any, have you experienced… [PROBE IF NOT ALREADY COVERED]

o Improved employment outcomeso Being able to build stronger relationships with job seekerso Staff being invigoratedo Having more leverage/funds to help participantso Job seekers being more willing to attend their scheduled meetings o Job seekers having improved communication skills and abilities o The rewarding feeling associated with seeing participants provided with the

opportunity to move forward faster than through the usual system

• What about the bad things/negatives? In what ways have these been negative for you?


o Having to learn the rules and requirements of the trialo Not fully understanding the rules of what the money can/cannot be spent ono Having difficulty understanding how the money can be accessed (e.g. reimbursement

system verse having to pay upfront)o Having difficulties recording all of the details in the current Job Plan systemo Job seekers not knowing what to spend the money ono Feeling that the timeframe is not long enough for the job seekerso It requires more time and effort than usual

• How are these same or different to what you expected?

[IF YES]

o What, if anything, had you done to prepare for them?o In hindsight, what might have helped you to be prepared for them?

• How have you addressed these challenges throughout the trial?

• For any of the job seekers that you have in the trial, what has been the most significant

change as a result of their participation in the trial?

NOTE: IF THEY HAVE MULTIPLE JOB SEEKERS THEY CAN MENTION A FEW POSITIVES


o Improved employment outcomeso Job seekers being more willing to attend their scheduled meetings o Improved communication skills and abilities o More Empowerment o Increased participation in training, work experience and skills development

99

o Increased job readinesso Greater confidence and composure of job seekerso Improved mental health o A more positive outlook about their future

• Has participation in the trial helped job seekers to improve their skills?

[IF YES]

o What skills have they gained from participating?o What skills have they learned that they would not otherwise have gained?o Do you think that these job seekers could have acquired these skills without being a

part of the trial?

• How would you rate the overall effectiveness of the trial in helping job seekers to develop

skills and find employment? How come?

• What element of the trial created any positive impact?

o The job seeker being able to decide how to spend the moneyo The amount of money that was available to each job seeker

• Which element played the greater role in creating any positive outcomes?

TRIAL DETAILS AND THE $5000 & JOB SEEKER TRIAL OUTCOMES 10 MINS

• How would you describe your role in the process for all of the job seekers that you have

participating in the trial?

• How has this role changed in the last three months?

• How easy or hard is it for job seekers to decide what things would help them become more

ready for employment? What makes it easier/harder?

• What role, if any, do we play in helping job seekers to consider what things would help them?

• What sort of things do job seekers in the trial spend the money on? How much did they

usually spend? Has this changed over the last three months?

• Did the job seekers use this funding instead of, or in addition to other funding? How come?


• What impact, if any, has spending the money in this way had on the job seeker? How come?

• Has spending the money in this way had any impact on employment outcomes? How come?

• How easy or hard is it for job seekers to decide what to spend the money on? In what ways is

it easy? In what ways is it hard? Have they needed guidance as to how the monies would be

best spent?

100

• What role have you played, if any, in helping job seekers decide what to spend the money

on?

• How would the $5,000 account been best spent for these job seekers? How come?

REFLECTION ON THE TRIAL 5 MINS

• To what extent has the trial provided what you originally thought it would?

• What has been delivered?

• What has not been delivered? How come?

o What were they key reasons this was not achieved?o Are the outcomes from the trial still valuable? How so?

• What other factors outside of the trial, if any, either contributed to, or reduced the impact of,

the program?

• What, if any, any unforeseen or unintended outcomes of the trial have occurred (either

positive or negative)?

• In what ways, if any, has the trial caused you to reflect on your standard service delivery

approaches?

• I know we discussed this last time we spoke, but now that the trial is coming to an end, how

likely or unlikely are you to change the way you operate in the future as a result of this trial?

• What is the one thing that you know now, that you wish you knew at the beginning of trial?

How would this have helped?

POTENTIAL CHANGES FOR FUTURE PROGRAMMES / RECOMMENDATIONS 5 MINS

• Based on your experience, what is the best way to assist young job seekers with a mental

health issue towards employment?

• What role does a program like the trial have to play in assisting young job seekers towards

employment?

• What impact, if any, might a program like the trial have if it was made widely available to

young job seekers with a mental health issue?

o What would be the good things about making it more widely available?

101

o What would be the bad things/risks in making it more widely available? What would be the risks to the Department? What would reduce these risks?

• What would make it more effective?

• What else is needed to move more young job seekers with a mental health issue towards

employment?

• How could the Department achieve similar outcomes for job seekers with less funds?

• If the trial gave job seekers the choice about how to spend money from the Department but

there was less money available for each job seeker – what difference would that make?

• Would less money for each job seeker have detracted from the impact of the trial?

WRAP UP 2 MINS


comments to add?

• That’s the end of the interview, thank you so much for your time, it is greatly appreciated.






102

11.Appendix G: Trial participants and control group matches

Figure 13: Trial participants and control group matches

Match ID

Group Esa NameFunding Level

Disability Disability Desc Eductaltion Level DescGender

CdAge At Extract

CALD Jskr

Ex Offender

Jskr

Homeless Jskr

Indigenous Jskr

Months in Unemploy-

ment

Jsc Jsci Score

1 Trial North Brisbane QLD ESS FL1 Autism Aspergers Syndrome Diploma or equivalent F 22 No No No No 24 241 Group 1 Central/West Metro ESS FL1 Psychiatric Depression Diploma or equivalent F 21 No No No No 19 281 Group 2 East Gippsland VIC ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 20 No No No No 22 262 Trial Maroondah VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 23 No Yes Yes No 48 432 Group 1 Eureka VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 23 No Yes Yes No 51 332 Group 2 Inner Sydney NSW ESS FL1 Psychiatric Depression Primary school or less than Year 10 F 24 No Yes Yes No 97 413 Trial North Brisbane QLD ESS FL1 Psychiatric Depression Non-trade vocational education M 22 No No No No 56 373 Group 1 Central/West Metro ESS FL1 Psychiatric Depression Non-trade vocational education M 21 No No No No 67 343 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Depression Non-trade vocational education M 21 No No No No 60 364 Trial North Metro WA DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 23 No No No No 29 274 Group 1 Toowoomba QLD DMS Psychiatric Depression Bachelor Degree or equivalent F 24 No No No No 30 384 Group 2 Eastern Adelaide SA DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 23 No No No No 30 265 Trial Maroondah VIC DMS Psychiatric Anxiety Non-trade vocational education F 20 No No No No 18 365 Group 1 Eureka VIC DMS Psychiatric Anxiety Non-trade vocational education F 20 No No No No 13 395 Group 2 Eureka VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 21 No No No No 16 356 Trial North Brisbane QLD ESS FL1 Psychiatric Anxiety Year 10/11 F 23 No No No No 64 416 Group 1 Nepean NSW ESS FL1 Psychiatric Post Traumatic Stress disorder Primary school or less than Year 10 F 24 No No No No 89 396 Group 2 Liverpool NSW ESS FL1 Psychiatric Depression Year 10/11 F 24 No No No No 61 347 Trial North Metro WA ESS FL1 Psychiatric Anxiety Non-trade vocational education M 19 No No No No 13 217 Group 1 Latrobe Valley VIC ESS FL1 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 19 No No No No 13 257 Group 2 Toowoomba QLD ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Primary school or less than Year 10 M 19 No No No No 13 258 Trial Maroondah VIC ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Year 10/11 M 24 No No No No 15 338 Group 1 Liverpool NSW ESS FL1 Psychiatric Depression Year 10/11 M 23 No No No No 13 258 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Anxiety Primary school or less than Year 10 M 23 No No No No 15 449 Trial North Brisbane QLD ESS FL2 Psychiatric Schizophrenia Not Stated M 18 No No No No 3 09 Group 1 Latrobe Valley VIC ESS FL2 Autism Aspergers Syndrome Not Stated M 19 No No No No 3 09 Group 2 Eureka VIC ESS FL2 Autism Aspergers Syndrome Year 10/11 M 18 No No No No 3 35

103

10 Trial Maroondah VIC ESS FL1 Psychiatric Depression Non-trade vocational education M 21 No No No No 5 3210 Group 1 Eureka VIC ESS FL1 Autism Aspergers Syndrome Non-trade vocational education M 22 No No No No 1 4210 Group 2 Toowoomba QLD ESS FL1 Intellectual Intellectual Disability Non-trade vocational education M 21 No No No No 1 2911 Trial Maroondah VIC DMS Psychiatric Depression Year 10/11 M 22 No No Yes No 57 3711 Group 1 Toowoomba QLD DMS Psychiatric Depression Year 10/11 M 22 No No Yes No 51 4211 Group 2 Central/West Metro DMS Psychiatric Depression Year 10/11 M 22 No No Yes No 70 3312 Trial North Metro WA ESS FL2 Autism Autism Special school or special support unit F 20 No No No No 10 4112 Group 1 Latrobe Valley VIC ESS FL2 Intellectual Intellectual Disability Non-trade vocational education F 20 No No No No 12 3512 Group 2 Latrobe Valley VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 22 No No No No 9 3013 Trial North Brisbane QLD ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 22 No No No No 36 3713 Group 1 Eastern Adelaide SA ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Secondary schooling completed (Year 12/13) M 21 No No No No 28 3313 Group 2 Central/West Metro ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 22 No No No No 26 3214 Trial North Brisbane QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 22 No Yes Yes No 42 4514 Group 1 Central/West Metro ESS FL2 Psychiatric Schizophrenia Year 10/11 F 19 No Yes Yes No 43 5014 Group 2 Central/West Metro ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 20 No Yes Yes No 42 4215 Trial Maroondah VIC DMS Psychiatric Psychotic Non-trade vocational education M 24 No No No No 15 2615 Group 1 Toowoomba QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 23 No No No No 16 1615 Group 2 Eastern Adelaide SA DMS Psychiatric Psychol/Psychiatric Disorder - Other Trades qualification M 23 No No No No 16 1516 Trial North Brisbane QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 23 No No No Yes 28 2716 Group 1 Liverpool NSW DMS Physical Shoulder and Upper Arm Disorder Secondary schooling completed (Year 12/13) M 22 No No No Yes 31 4316 Group 2 Central/West Metro ESS FL2 Autism Autism Secondary schooling completed (Year 12/13) M 20 No No No Yes 34 3917 Trial North Metro WA ESS FL1 Psychiatric Anxiety Non-trade vocational education M 22 No No No No 42 2617 Group 1 Liverpool NSW ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Non-trade vocational education M 21 No No No No 43 3817 Group 2 Liverpool NSW ESS FL1 Intellectual Intellectual Disability Non-trade vocational education M 21 No No No No 39 2218 Trial North Metro WA ESS FL1 Speech Speech Disorder - Other Year 10/11 M 23 No No No No 10 2618 Group 1 Latrobe Valley VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 23 No No No No 8 2318 Group 2 Inner Sydney NSW ESS FL1 Autism Aspergers Syndrome Not Stated M 24 No No No No 11 019 Trial North Brisbane QLD ESS FL1 Autism Autism Non-trade vocational education M 24 No No No No 9 3619 Group 1 Inner Sydney NSW ESS FL1 Psychiatric Depression Non-trade vocational education M 23 No No No No 11 819 Group 2 Nepean NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 23 No No No No 9 2720 Trial Maroondah VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 23 No No No No 16 4220 Group 1 Latrobe Valley VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 23 No No No No 10 4220 Group 2 Central/West Metro ESS FL2 Psychiatric Anxiety Non-trade vocational education F 23 No No No No 10 3821 Trial North Metro WA ESS FL2 Specific Learning Attention Deficit/Hyperactivity disorder Non-trade vocational education M 22 No No No No 18 3521 Group 1 Latrobe Valley VIC ESS FL2 Autism Autism Primary school or less than Year 10 M 21 No No No No 13 4021 Group 2 Nepean NSW ESS FL2 Autism Autism Primary school or less than Year 10 M 22 No No No No 14 4322 Trial North Brisbane QLD ESS FL1 Psychiatric Schizophrenia Not Stated M 18 No No No No 4 022 Group 1 Inner Sydney NSW ESS FL1 Autism Aspergers Syndrome Not Stated M 18 No No No No 1 022 Group 2 Nepean NSW ESS FL1 Psychiatric Obsessive Compulsive Disorder Not Stated M 19 No No No No 2 023 Trial North Brisbane QLD ESS FL1 Psychiatric Depression Not Stated M 21 No No No No 17 023 Group 1 Liverpool NSW ESS FL1 Psychiatric Psychotic Year 10/11 M 20 No No No No 13 2423 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Depression Not Stated M 20 No No No No 15 024 Trial North Brisbane QLD ESS FL1 Psychiatric Psychotic Year 10/11 M 19 No No No No 17 3624 Group 1 Latrobe Valley VIC ESS FL1 Psychiatric Anxiety Year 10/11 M 19 No No No No 20 2924 Group 2 Central/West Metro ESS FL1 Psychiatric Anxiety Year 10/11 M 19 No No No No 23 025 Trial North Metro WA ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 21 No No No No 18 4125 Group 1 Latrobe Valley VIC ESS FL2 Specific Learning Learning Disability Primary school or less than Year 10 F 21 No No No No 17 3125 Group 2 Nepean NSW ESS FL2 Specific Learning Learning Disability Not Stated F 21 No No No No 16 0

104

26 Trial North Brisbane QLD ESS FL1 Physical Psoriasis Year 10/11 M 19 No No No No 25 2726 Group 1 Liverpool NSW ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Year 10/11 M 18 No No No No 28 3326 Group 2 Toowoomba QLD ESS FL1 Psychiatric Depression Year 10/11 M 19 No No No No 27 2327 Trial North Brisbane QLD ESS FL1 Psychiatric Schizophrenia Non-trade vocational education M 21 No No No No 2 2327 Group 1 Central/West Metro ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 22 No No No No 2 3327 Group 2 Warwick QLD ESS FL1 Psychiatric Anxiety Primary school or less than Year 10 M 20 No No No No 6 2928 Trial North Metro WA ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 20 No Yes No No 20 3728 Group 1 Eureka VIC ESS FL1 Specific Learning Learning Disability Non-trade vocational education M 22 No Yes No No 19 3428 Group 2 Central/West Metro ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education M 22 No Yes No No 29 2829 Trial Maroondah VIC DMS Psychiatric Anxiety Non-trade vocational education M 23 No No No No 51 3829 Group 1 Central/West Metro DMS Psychiatric Depression Non-trade vocational education M 24 No No No No 76 3829 Group 2 Inner Sydney NSW DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education M 23 No No No No 67 4030 Trial North Metro WA ESS FL1 Autism Autism Non-trade vocational education M 23 No No No No 59 2430 Group 1 Eastern Adelaide SA ESS FL1 Psychiatric Schizophrenia Non-trade vocational education M 24 No No No No 58 3930 Group 2 Toowoomba QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 23 No No No No 52 3731 Trial North Brisbane QLD ESS FL2 Psychiatric Depression Primary school or less than Year 10 M 22 No No No No 47 4231 Group 1 Toowoomba QLD ESS FL2 Autism Autism Year 10/11 M 22 No No No No 48 3931 Group 2 East Gippsland VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Primary school or less than Year 10 M 20 No No No No 48 4132 Trial North Brisbane QLD ESS FL1 Psychiatric Psychotic Non-trade vocational education M 23 Yes No No No 10 2732 Group 1 Eastern Adelaide SA ESS FL1 Neurological Epilepsy Year 10/11 M 24 Yes No No No 9 4032 Group 2 Nepean NSW ESS FL1 Psychiatric Anxiety Year 10/11 M 23 Yes No No No 12 2133 Trial North Brisbane QLD ESS FL2 Psychiatric Depression Secondary schooling completed (Year 12/13) M 20 No No Yes No 32 4133 Group 1 Central/West Metro ESS FL2 Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education M 20 No No Yes No 23 3533 Group 2 East Gippsland VIC ESS FL2 Specific Learning Attention Deficit/Hyperactivity disorder Non-trade vocational education M 21 No No Yes No 37 4534 Trial North Metro WA ESS FL1 Psychiatric Obsessive Compulsive Disorder Year 10/11 M 20 No No No No 19 3434 Group 1 Nepean NSW ESS FL1 Psychiatric Anxiety Year 10/11 M 21 No No No No 21 4034 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Obsessive Compulsive Disorder Primary school or less than Year 10 M 22 No No No No 22 3135 Trial North Metro WA ESS FL2 Acquired brain injury Acquired Brain Injury Special school or special support unit M 20 No No No No 11 3635 Group 1 Central/West Metro ESS FL2 Autism Autism Year 10/11 M 20 No No No No 11 3435 Group 2 Liverpool NSW ESS FL2 Specific Learning Attention Deficit/Hyperactivity disorder Year 10/11 M 20 No No No No 9 1336 Trial Maroondah VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 19 No No No No 4 3536 Group 1 Eureka VIC ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 18 No No No No 5 3136 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 19 No No No No 3 2337 Trial North Metro WA DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 24 No No No No 18 2037 Group 1 Goldfields/Esperance DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 23 No No No No 13 1637 Group 2 Inner Sydney NSW DMS Psychiatric Post Traumatic Stress disorder Secondary schooling completed (Year 12/13) M 23 No No No No 16 3638 Trial North Metro WA ESS FL1 Psychiatric Post Traumatic Stress disorder Non-trade vocational education F 24 No No Yes No 5 4238 Group 1 Nepean NSW ESS FL1 Psychiatric Depression Diploma or equivalent F 22 No No Yes No 0 3638 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Personality Disorder Non-trade vocational education F 23 No No Yes No 7 5539 Trial North Brisbane QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 22 No No No No 3 4839 Group 1 Central/West Metro ESS FL1 Neurological Cerebral Palsy Non-trade vocational education F 22 No No No No 3 4139 Group 2 Liverpool NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 22 No No No No 5 2440 Trial North Brisbane QLD DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) M 23 No No No No 11 2340 Group 1 Toowoomba QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 23 No No No No 10 2540 Group 2 South Eastern NSW DMS Psychiatric Depression Trades qualification M 24 No No No No 9 25

105

41 Trial Maroondah VIC DMS Psychiatric Anxiety Non-trade vocational education M 24 No No No No 22 2241 Group 1 Central/West Metro DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 23 No No No No 20 3841 Group 2 Latrobe Valley VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Primary school or less than Year 10 M 23 No No No No 24 042 Trial North Metro WA ESS FL2 Psychiatric Anxiety Primary school or less than Year 10 M 19 No No No No 10 4142 Group 1 Nepean NSW ESS FL2 Specific Learning Learning Disability Year 10/11 M 19 No No No No 12 1942 Group 2 East Gippsland VIC ESS FL2 Intellectual Intellectual Disability Year 10/11 M 18 No No No No 9 3543 Trial North Brisbane QLD ESS FL2 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 21 No No No No 8 3243 Group 1 Central/West Metro ESS FL2 Autism Autism Secondary schooling completed (Year 12/13) M 20 No No No No 7 2643 Group 2 East Gippsland VIC ESS FL2 Specific Learning Learning Disability Secondary schooling completed (Year 12/13) M 22 No No No No 12 3944 Trial North Brisbane QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Diploma or equivalent M 22 No No No No 21 3344 Group 1 Latrobe Valley VIC DMS Psychiatric Anxiety Non-trade vocational education M 20 No No No No 24 3544 Group 2 Liverpool NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 21 No No No No 21 2845 Trial North Metro WA ESS FL1 Psychiatric Psychotic Year 10/11 M 24 No Yes No No 26 3445 Group 1 Liverpool NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 23 No Yes No No 35 3945 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No Yes No No 19 2946 Trial North Metro WA DMS Psychiatric Anxiety Graduate Certificate or equivalent F 21 No No Yes No 12 1946 Group 1 Inner Sydney NSW DMS Psychiatric Depression Non-trade vocational education F 23 No No Yes No 13 2646 Group 2 Toowoomba QLD DMS Psychiatric Personality Disorder Non-trade vocational education F 19 No No Yes No 14 3347 Trial North Brisbane QLD DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Bachelor Degree or equivalent F 23 No No No No 14 2847 Group 1 Eastern Adelaide SA DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 24 No No No No 14 2347 Group 2 Liverpool NSW DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 23 No No No No 15 3848 Trial North Brisbane QLD ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 22 No No No No 25 3048 Group 1 Eureka VIC ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) M 21 No No No No 34 3848 Group 2 Liverpool NSW ESS FL1 Acquired brain injury Acquired Brain Injury Secondary schooling completed (Year 12/13) M 22 No No No No 33 3649 Trial Maroondah VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 23 No No Yes No 25 3249 Group 1 Eureka VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 24 No No Yes No 19 3449 Group 2 Central/West Metro DMS Psychiatric Anxiety Non-trade vocational education F 23 No No Yes No 46 3750 Trial North Brisbane QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 22 No No No No 9 2750 Group 1 Central/West Metro ESS FL1 Psychiatric Depression Non-trade vocational education F 20 No No No No 8 3150 Group 2 Nepean NSW ESS FL1 Psychiatric Personality Disorder Non-trade vocational education F 22 No No No No 10 2851 Trial North Brisbane QLD DMS Psychiatric Personality Disorder Year 10/11 F 22 No Yes No No 9 3051 Group 1 Eureka VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 23 No Yes No No 8 4051 Group 2 Eureka VIC DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education F 24 No Yes No No 11 2952 Trial Maroondah VIC DMS Psychiatric Depression Non-trade vocational education M 24 No No No No 18 2052 Group 1 Latrobe Valley VIC DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 23 No No No No 13 2152 Group 2 Nepean NSW DMS Psychiatric Post Traumatic Stress disorder Year 10/11 M 23 No No No No 14 2853 Trial North Brisbane QLD ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 22 No No No No 48 3553 Group 1 Eureka VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 20 No No No No 37 3953 Group 2 Nepean NSW ESS FL2 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 21 No No No No 37 4054 Trial North Metro WA DMS Psychiatric Personality Disorder Year 10/11 F 21 No No No No 31 4154 Group 1 Toowoomba QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 20 No No No No 30 2454 Group 2 Inner Sydney NSW DMS Psychiatric Depression Year 10/11 F 20 No No No No 27 3955 Trial North Metro WA ESS FL1 Psychiatric Schizophrenia Non-trade vocational education F 24 No No No No 10 3955 Group 1 Central/West Metro ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) F 23 No No No No 10 2255 Group 2 Nepean NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 24 No No No No 11 40

106

56 Trial North Brisbane QLD ESS FL2 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 19 No No No No 12 2656 Group 1 Nepean NSW ESS FL2 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 19 No No No No 7 2456 Group 2 Nepean NSW ESS FL2 Specific Learning Attention Deficit/Hyperactivity disorder Secondary schooling completed (Year 12/13) M 19 No No No No 11 2557 Trial Maroondah VIC DMS Psychiatric Depression Non-trade vocational education M 24 No No Yes No 7 2057 Group 1 South Eastern NSW DMS Physical Shoulder and Upper Arm Disorder Non-trade vocational education M 22 No No Yes No 11 3357 Group 2 Inner Sydney NSW DMS Psychiatric Post Traumatic Stress disorder Trades qualification M 24 No No Yes No 7 2558 Trial North Brisbane QLD ESS FL2 Psychiatric Depression Year 10/11 M 22 No No No No 56 3558 Group 1 Latrobe Valley VIC ESS FL2 Intellectual Intellectual Disability Primary school or less than Year 10 M 21 No No No No 58 4358 Group 2 East Gippsland VIC ESS FL2 Psychiatric Anxiety Primary school or less than Year 10 M 21 No No No No 61 4259 Trial North Metro WA ESS FL1 Psychiatric Personality Disorder Secondary schooling completed (Year 12/13) F 21 No No No No 10 2659 Group 1 Central/West Metro ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 21 No No No No 9 3659 Group 2 Warwick QLD ESS FL1 Psychiatric Schizophrenia Secondary schooling completed (Year 12/13) F 21 No No No No 7 3060 Trial North Metro WA DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Year 10/11 M 23 No No Yes No 24 2760 Group 1 Inner Sydney NSW DMS Psychiatric Schizophrenia Year 10/11 M 24 No No Yes No 30 3660 Group 2 Liverpool NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No No Yes No 30 3361 Trial North Metro WA ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Year 10/11 F 22 No No No No 26 3461 Group 1 Nepean NSW ESS FL1 Autism Autism Not Stated F 21 No No No No 34 061 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Anxiety Year 10/11 F 20 No No No No 33 4462 Trial Maroondah VIC ESS FL1 Psychiatric Depression Primary school or less than Year 10 M 24 No No No No 31 4062 Group 1 Eureka VIC ESS FL1 Psychiatric Depression Year 10/11 M 24 No No No No 35 4062 Group 2 Liverpool NSW ESS FL1 Psychiatric Depression Non-trade vocational education M 23 No No No No 36 3563 Trial Maroondah VIC ESS FL2 Psychiatric Depression Secondary schooling completed (Year 12/13) M 23 No No No No 30 4063 Group 1 Eureka VIC ESS FL2 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 23 No No No No 46 4063 Group 2 East Gippsland VIC ESS FL2 Psychiatric Psychotic Secondary schooling completed (Year 12/13) M 23 No No No No 32 3764 Trial Maroondah VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 22 No No No No 46 3764 Group 1 Warwick QLD DMS Psychiatric Depression Non-trade vocational education M 20 No No No No 37 2864 Group 2 Nepean NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 21 No No No No 47 3065 Trial Maroondah VIC DMS Psychiatric Anxiety Non-trade vocational education M 22 No No No No 42 3465 Group 1 Liverpool NSW DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 22 No No No No 40 1665 Group 2 Liverpool NSW DMS Physical Ulcerative Colitis Year 10/11 M 21 No No No No 38 3466 Trial North Metro WA ESS FL1 Physical Klinefelter's Syndrome Non-trade vocational education M 19 No No No No 4 2366 Group 1 Toowoomba QLD ESS FL1 Autism Autism Non-trade vocational education M 18 No No No No 5 1966 Group 2 South Eastern NSW ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Non-trade vocational education M 18 No No No No 1 3167 Trial North Metro WA DMS Psychiatric Schizophrenia Year 10/11 M 23 No No No No 27 3367 Group 1 Toowoomba QLD DMS Psychiatric Anxiety Year 10/11 M 24 No No No No 35 2967 Group 2 Central/West Metro DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 23 No No No No 27 1668 Trial North Brisbane QLD DMS Specific Learning Attention Deficit/Hyperactivity disorder Secondary schooling completed (Year 12/13) M 18 No No No No 9 2168 Group 1 Toowoomba QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 19 No No No No 11 1668 Group 2 Toowoomba QLD DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 19 No No No No 10 3569 Trial North Metro WA DMS Psychiatric Depression Primary school or less than Year 10 M 22 No Yes No Yes 49 4369 Group 1 Toowoomba QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Special school or special support unit M 20 No Yes No Yes 57 4569 Group 2 Nepean NSW DMS Psychiatric Anxiety Primary school or less than Year 10 M 21 No Yes No Yes 61 4770 Trial North Metro WA ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 21 No No No No 30 3970 Group 1 Nepean NSW ESS FL2 Psychiatric Schizophrenia Year 10/11 M 20 No No No No 34 3370 Group 2 Eureka VIC ESS FL2 Psychiatric Schizophrenia Year 10/11 M 20 No No No No 32 40

107

71 Trial Maroondah VIC ESS FL1 Autism Aspergers Syndrome Non-trade vocational education M 21 No No No No 13 3471 Group 1 South Eastern NSW ESS FL1 Psychiatric Depression Non-trade vocational education M 22 No No No No 13 2471 Group 2 Nepean NSW ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Non-trade vocational education M 21 No No No No 16 2272 Trial Maroondah VIC ESS FL2 Autism Autism Non-trade vocational education M 24 No No No No 8 3772 Group 1 Central/West Metro ESS FL2 Intellectual Intellectual Disability Year 10/11 M 24 No No No No 12 2372 Group 2 Nepean NSW ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No No No No 7 3573 Trial Maroondah VIC DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) M 23 No No No No 13 3173 Group 1 Latrobe Valley VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 23 No No No No 22 2373 Group 2 Liverpool NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Bachelor Degree or equivalent M 24 No No No No 20 2274 Trial North Metro WA ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 20 No No No No 16 1574 Group 1 Toowoomba QLD ESS FL1 Autism Autism Non-trade vocational education M 22 No No No No 16 3674 Group 2 Eastern Adelaide SA ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 22 No No No No 18 3575 Trial North Brisbane QLD ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 20 No No No No 23 3175 Group 1 Eureka VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 20 No No No No 24 4175 Group 2 Eureka VIC ESS FL2 Intellectual Intellectual Disability Non-trade vocational education F 22 No No No No 19 4276 Trial Maroondah VIC ESS FL1 Psychiatric Depression Non-trade vocational education F 24 No No No No 83 3976 Group 1 Toowoomba QLD ESS FL1 Psychiatric Depression Non-trade vocational education F 23 No No No No 75 2676 Group 2 Eureka VIC ESS FL1 Psychiatric Depression Non-trade vocational education F 24 No No No No 102 4477 Trial Maroondah VIC DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 24 No No No No 17 3177 Group 1 Toowoomba QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 23 No No No No 17 3777 Group 2 Eastern Adelaide SA DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 24 No No No No 14 2378 Trial Maroondah VIC ESS FL2 Psychiatric Anxiety Year 10/11 M 23 No No No No 48 4378 Group 1 Eureka VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No No No No 79 3378 Group 2 Goldfields/Esperance ESS FL2 Autism Autism Primary school or less than Year 10 M 23 No No No No 81 3779 Trial North Brisbane QLD ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Year 10/11 F 18 No No Yes No 3 4279 Group 1 Eastern Adelaide SA DMS Psychiatric Post Traumatic Stress disorder Year 10/11 F 19 No No Yes No 7 4079 Group 2 Liverpool NSW ESS FL1 Psychiatric Depression Non-trade vocational education F 18 No No Yes No 13 3780 Trial North Brisbane QLD DMS Psychiatric Post Traumatic Stress disorder Year 10/11 F 22 No No No No 45 3380 Group 1 Nepean NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 20 No No No No 37 2780 Group 2 Eastern Adelaide SA DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 22 No No No No 37 3681 Trial North Brisbane QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 20 No No No No 22 2181 Group 1 Toowoomba QLD DMS Psychiatric Anxiety Non-trade vocational education M 20 No No No No 23 2081 Group 2 Eastern Adelaide SA DMS Psychiatric Depression Non-trade vocational education M 22 No No No No 23 3982 Trial North Brisbane QLD DMS Psychiatric Phobias Non-trade vocational education M 18 Yes No No No 12 2982 Group 1 Nepean NSW DMS Psychiatric Schizophrenia Year 10/11 M 18 Yes No No No 14 2282 Group 2 Inner Sydney NSW DMS Psychiatric Depression Non-trade vocational education M 18 Yes No No No 3 3483 Trial North Brisbane QLD DMS Psychiatric Anxiety Non-trade vocational education M 24 No No No No 19 3883 Group 1 Toowoomba QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 23 No No No No 20 2383 Group 2 Central/West Metro DMS Psychiatric Depression Diploma or equivalent M 23 No No No No 20 2384 Trial North Brisbane QLD ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) F 24 No No No No 48 4284 Group 1 East Gippsland VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 23 No No No No 48 3284 Group 2 Latrobe Valley VIC ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) F 23 No No No No 40 2685 Trial North Metro WA ESS FL2 Intellectual Intellectual Disability Year 10/11 F 18 No No No No 20 3885 Group 1 Toowoomba QLD ESS FL2 Psychiatric Anxiety Year 10/11 F 18 No No No No 23 2585 Group 2 Eureka VIC ESS FL2 Hearing Hearing Loss- Complete Year 10/11 F 19 No No No No 19 35

108

86 Trial North Brisbane QLD ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) M 20 No No No No 9 2586 Group 1 Toowoomba QLD ESS FL1 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 20 No No No No 9 1986 Group 2 Liverpool NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 22 No No No No 7 2287 Trial North Metro WA ESS FL2 Specific Learning Learning Disability Special school or special support unit M 19 No Yes No No 1 3887 Group 1 Eastern Adelaide SA ESS FL2 Psychiatric Depression Year 10/11 M 24 No Yes No No 2 4687 Group 2 Nepean NSW ESS FL2 Psychiatric Anxiety Primary school or less than Year 10 M 21 No Yes No No 14 4088 Trial North Brisbane QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 19 No No No No 12 1888 Group 1 Toowoomba QLD DMS Specific Learning Attention Deficit/Hyperactivity disorder Year 10/11 M 19 No No No No 15 2988 Group 2 Latrobe Valley VIC DMS Psychiatric Anxiety Not Stated M 19 No No No No 18 089 Trial North Metro WA DMS Psychiatric Anxiety Non-trade vocational education M 21 No No No No 32 2989 Group 1 Latrobe Valley VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 20 No No No No 29 2989 Group 2 Eureka VIC DMS Psychiatric Anxiety Non-trade vocational education M 21 No No No No 36 3690 Trial North Metro WA ESS FL2 Specific Learning Learning Disability Non-trade vocational education F 20 No No No No 7 3690 Group 1 Toowoomba QLD ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 20 No No No No 12 3990 Group 2 Latrobe Valley VIC ESS FL2 Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 21 No No No No 10 4291 Trial North Brisbane QLD DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 20 No No No No 11 3791 Group 1 South Eastern NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 21 No No No No 7 2791 Group 2 Toowoomba QLD DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 20 No No No No 12 2692 Trial North Metro WA DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No No No No 38 3692 Group 1 Eastern Adelaide SA DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No No No No 42 3692 Group 2 Toowoomba QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 24 No No No No 37 3493 Trial North Brisbane QLD DMS Psychiatric Anxiety Year 10/11 M 17 No No No No 4 3593 Group 1 Warwick QLD DMS Neurological Epilepsy Not Stated M 18 No No No No 1 093 Group 2 Eureka VIC DMS Autism Aspergers Syndrome Not Stated M 18 No No No No 0 094 Trial Maroondah VIC DMS Specific Learning Learning Disability Secondary schooling completed (Year 12/13) F 21 No No No No 22 3794 Group 1 Nepean NSW DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 20 No No No No 22 3294 Group 2 Toowoomba QLD DMS Psychiatric Personality Disorder Secondary schooling completed (Year 12/13) F 21 No No No No 22 2295 Trial North Metro WA ESS FL2 Autism Autism Non-trade vocational education M 21 No No No No 17 3695 Group 1 South Eastern NSW ESS FL2 Autism Autism Year 10/11 M 21 No No No No 15 3895 Group 2 Toowoomba QLD ESS FL2 Specific Learning Learning Disability Secondary schooling completed (Year 12/13) M 21 No No No No 17 3596 Trial North Brisbane QLD DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 22 No No Yes No 5 3596 Group 1 East Gippsland VIC DMS Specific Learning Attention Deficit/Hyperactivity disorder Year 10/11 M 22 No No Yes No 0 4796 Group 2 Inner Sydney NSW DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 20 No No Yes No 10 3197 Trial North Brisbane QLD DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 19 No No No No 24 1897 Group 1 Liverpool NSW DMS Neurological Epilepsy - complex seizure Secondary schooling completed (Year 12/13) M 19 No No No No 13 2397 Group 2 Central/West Metro DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 19 No No No No 14 3298 Trial North Metro WA DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 23 No No No No 36 3098 Group 1 Inner Sydney NSW DMS Psychiatric Depression Non-trade vocational education M 24 No No No No 28 3998 Group 2 Eastern Adelaide SA DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 24 No No No No 30 1999 Trial North Brisbane QLD ESS FL1 Psychiatric Anxiety Not Stated F 20 No No No No 35 099 Group 1 Eastern Adelaide SA ESS FL1 Physical Cancer - Leukaemia (Chronic) Year 10/11 F 20 No No No No 36 3599 Group 2 Toowoomba QLD ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Primary school or less than Year 10 F 20 No No No No 31 34100 Trial North Metro WA ESS FL1 Psychiatric Anxiety Non-trade vocational education M 20 No No No No 7 29100 Group 1 Inner Sydney NSW ESS FL1 Psychiatric Anxiety Non-trade vocational education M 21 No No No No 8 0100 Group 2 Central/West Metro ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 21 No No No No 10 16

109

101 Trial North Brisbane QLD ESS FL1 Psychiatric Psychotic Secondary schooling completed (Year 12/13) M 21 No No No No 16 24101 Group 1 Nepean NSW ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 20 No No No No 15 21101 Group 2 Liverpool NSW ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) M 20 No No No No 17 21102 Trial North Metro WA DMS Psychiatric Anxiety Not Stated M 18 No No No No 1 0102 Group 1 Central/West Metro DMS Psychiatric Anxiety Not Stated M 18 No No No No 1 0102 Group 2 Goldfields/Esperance DMS Psychiatric Depression Year 10/11 M 19 No No No No 5 17103 Trial North Metro WA DMS Psychiatric Anxiety Diploma or equivalent F 22 No No No No 13 16103 Group 1 Inner Sydney NSW DMS Psychiatric Psychotic Secondary schooling completed (Year 12/13) F 22 No No No No 18 32103 Group 2 Eureka VIC DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 21 No No No No 18 37104 Trial North Brisbane QLD ESS FL1 Psychiatric Depression Non-trade vocational education M 24 No No No No 23 36104 Group 1 Central/West Metro ESS FL1 Psychiatric Depression Non-trade vocational education M 24 No No No No 19 28104 Group 2 Central/West Metro ESS FL1 Psychiatric Anxiety Non-trade vocational education M 23 No No No No 24 27105 Trial Maroondah VIC ESS FL1 Autism Aspergers Syndrome Year 10/11 M 24 No No No No 15 29105 Group 1 Eureka VIC ESS FL1 Intellectual Intellectual Disability Year 10/11 M 23 No No No No 21 41105 Group 2 East Gippsland VIC ESS FL1 Psychiatric Anxiety Not Stated M 23 No No No No 11 0106 Trial North Brisbane QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No No Yes No 72 36106 Group 1 Latrobe Valley VIC DMS Psychiatric Schizophrenia Year 10/11 M 23 No No Yes No 79 38106 Group 2 Inner Sydney NSW DMS Psychiatric Psychotic Year 10/11 M 24 No No Yes No 45 43107 Trial North Brisbane QLD ESS FL2 Psychiatric Anxiety Non-trade vocational education M 23 No No No No 62 35107 Group 1 Inner Sydney NSW ESS FL2 Psychiatric Anxiety Non-trade vocational education M 24 No No No No 64 41107 Group 2 Eastern Adelaide SA ESS FL2 Autism Aspergers Syndrome Non-trade vocational education M 24 No No No No 74 40108 Trial North Brisbane QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 22 No No No No 35 33108 Group 1 Latrobe Valley VIC DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 21 No No No No 28 23108 Group 2 South Eastern NSW DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) M 21 No No No No 26 33109 Trial North Metro WA ESS FL2 Intellectual Intellectual Disability Secondary schooling completed (Year 12/13) F 20 No No No No 13 37109 Group 1 Nepean NSW ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 19 No No No No 8 32109 Group 2 Eastern Adelaide SA ESS FL2 Autism Aspergers Syndrome Non-trade vocational education F 20 No No No No 14 40110 Trial North Metro WA ESS FL2 Autism Aspergers Syndrome Year 10/11 M 19 No No No No 5 19110 Group 1 Latrobe Valley VIC ESS FL2 Intellectual Intellectual Disability Primary school or less than Year 10 M 18 No No No No 10 46110 Group 2 Latrobe Valley VIC ESS FL2 Specific Learning Learning Disability Year 10/11 M 19 No No No No 11 26111 Trial North Brisbane QLD ESS FL2 Autism Autism Secondary schooling completed (Year 12/13) F 21 No No No No 15 39111 Group 1 Latrobe Valley VIC ESS FL2 Specific Learning Learning Disability Year 10/11 F 21 No No No No 16 48111 Group 2 Nepean NSW ESS FL2 Intellectual Intellectual Disability Year 10/11 F 21 No No No No 14 29112 Trial Maroondah VIC DMS Psychiatric Depression Non-trade vocational education F 24 No No No No 42 38112 Group 1 Eastern Adelaide SA DMS Psychiatric Obsessive Compulsive Disorder Non-trade vocational education F 24 No No No No 48 38112 Group 2 Latrobe Valley VIC DMS Psychiatric Depression Non-trade vocational education F 23 No No No No 40 25113 Trial North Metro WA DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 20 No No No No 17 31113 Group 1 Inner Sydney NSW DMS Psychiatric Psychotic Secondary schooling completed (Year 12/13) M 22 No No No No 14 22113 Group 2 Toowoomba QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 20 No No No No 13 20114 Trial North Brisbane QLD ESS FL2 Autism Aspergers Syndrome Non-trade vocational education M 21 No No No No 9 35114 Group 1 Liverpool NSW ESS FL2 Autism Autism Year 10/11 M 20 No No No No 12 36114 Group 2 Warwick QLD ESS FL2 Specific Learning Attention Deficit/Hyperactivity disorder Non-trade vocational education M 22 No No No No 12 37115 Trial North Brisbane QLD ESS FL2 Psychiatric Anxiety Year 10/11 M 18 No No No No 20 21115 Group 1 Warwick QLD ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 18 No No No No 23 39115 Group 2 Central/West Metro ESS FL2 Specific Learning Learning Disability Not Stated M 19 No No No No 24 0

110

116 Trial North Brisbane QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 24 No No No No 36 33116 Group 1 Eastern Adelaide SA DMS Psychiatric Depression Bachelor Degree or equivalent F 24 No No No No 30 26116 Group 2 Nepean NSW DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 24 No No No No 36 24117 Trial Maroondah VIC ESS FL1 Neurological Epilepsy - Grand Mal (Tonic-Clonic) Non-trade vocational education F 19 No No No No 10 39117 Group 1 Central/West Metro ESS FL1 Autism Aspergers Syndrome Non-trade vocational education F 18 No No No No 11 22117 Group 2 Nepean NSW ESS FL1 Psychiatric Anxiety Non-trade vocational education F 19 No No No No 10 23118 Trial North Brisbane QLD ESS FL2 Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 21 No No No Yes 15 41118 Group 1 Toowoomba QLD ESS FL2 Intellectual Intellectual Disability Non-trade vocational education M 24 No No No Yes 15 38118 Group 2 Toowoomba QLD ESS FL2 Physical Lower Limb Deficiencies Secondary schooling completed (Year 12/13) M 20 No No No Yes 7 48119 Trial North Metro WA ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Diploma or equivalent M 22 No No No No 0 25119 Group 1 Central/West Metro ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Secondary schooling completed (Year 12/13) M 22 No No No No 0 17119 Group 2 Liverpool NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Diploma or equivalent M 20 No No No No 5 26120 Trial North Metro WA DMS Psychiatric Anxiety Non-trade vocational education M 24 No No No No 22 23120 Group 1 Nepean NSW DMS Psychiatric Depression Secondary schooling completed (Year 12/13) M 24 No No No No 23 38120 Group 2 Eureka VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 23 No No No No 20 28121 Trial North Metro WA ESS FL2 Psychiatric Anxiety Year 10/11 M 20 No No No No 45 36121 Group 1 East Gippsland VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Primary school or less than Year 10 M 22 No No No No 46 44121 Group 2 Eureka VIC ESS FL2 Specific Learning Learning Disability Primary school or less than Year 10 M 21 No No No No 44 41122 Trial North Metro WA ESS FL2 Psychiatric Anxiety Non-trade vocational education F 24 No No No No 40 37122 Group 1 Nepean NSW ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 24 No No No No 48 37122 Group 2 Inner Sydney NSW ESS FL2 Autism Aspergers Syndrome Non-trade vocational education F 24 No No No No 39 40123 Trial Maroondah VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 23 No No No No 57 43123 Group 1 Nepean NSW ESS FL1 Psychiatric Depression Non-trade vocational education F 24 No No No No 56 38123 Group 2 Nepean NSW ESS FL1 Physical Shoulder and Upper Arm Disorder Non-trade vocational education F 24 No No No No 63 38124 Trial Maroondah VIC DMS Physical Lower Limb Deficiencies Non-trade vocational education F 21 No No Yes No 21 41124 Group 1 South Eastern NSW DMS Psychiatric Depression Primary school or less than Year 10 F 20 No No Yes No 26 40124 Group 2 Inner Sydney NSW DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 22 No No Yes No 35 38125 Trial Maroondah VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Primary school or less than Year 10 M 22 No No No No 7 30125 Group 1 Toowoomba QLD ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Year 10/11 M 21 No No No No 10 35125 Group 2 Nepean NSW ESS FL1 Psychiatric Depression Not Stated M 22 No No No No 10 0126 Trial Maroondah VIC ESS FL1 Psychiatric Anxiety Non-trade vocational education F 24 No No No No 33 28126 Group 1 South Eastern NSW ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) F 23 No No No No 26 28126 Group 2 Warwick QLD ESS FL1 Psychiatric Obsessive Compulsive Disorder Bachelor Degree or equivalent F 23 No No No No 28 38127 Trial North Metro WA ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Secondary schooling completed (Year 12/13) F 21 No No No No 37 42127 Group 1 Nepean NSW ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Secondary schooling completed (Year 12/13) F 21 No No No No 46 39127 Group 2 Nepean NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 22 No No No No 48 42128 Trial Maroondah VIC ESS FL1 Psychiatric Anxiety Non-trade vocational education F 17 No No No No 10 26128 Group 1 Central/West Metro ESS FL1 Psychiatric Depression Non-trade vocational education F 19 No No No No 11 37128 Group 2 Eastern Adelaide SA ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 19 No No No No 8 21129 Trial North Brisbane QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 23 No No No No 56 30129 Group 1 Inner Sydney NSW DMS Psychiatric Depression Diploma or equivalent F 24 No No No No 59 37129 Group 2 Toowoomba QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 23 No No No No 56 38130 Trial North Brisbane QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 23 No No No No 22 27130 Group 1 Nepean NSW ESS FL1 Psychiatric Depression Non-trade vocational education M 24 No No No No 21 20130 Group 2 Eastern Adelaide SA ESS FL1 Psychiatric Depression Non-trade vocational education M 24 No No No No 23 30

111

131 Trial North Brisbane QLD DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 24 No No No No 48 34131 Group 1 Nepean NSW DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 24 No No No No 41 23131 Group 2 Eureka VIC DMS Psychiatric Anxiety Diploma or equivalent F 24 No No No No 38 26132 Trial North Brisbane QLD ESS FL2 Psychiatric Depression Year 10/11 F 24 No No No No 49 42132 Group 1 Central/West Metro ESS FL2 Psychiatric Depression Year 10/11 F 23 No No No No 49 44132 Group 2 Warwick QLD ESS FL2 Psychiatric Depression Year 10/11 F 23 No No No No 52 38133 Trial North Metro WA ESS FL1 Psychiatric Anxiety Year 10/11 M 19 No No No No 27 29133 Group 1 Latrobe Valley VIC ESS FL1 Psychiatric Psychotic Non-trade vocational education M 19 No No No No 32 27133 Group 2 Toowoomba QLD ESS FL1 Psychiatric Obsessive Compulsive Disorder Non-trade vocational education M 19 No No No No 29 20134 Trial North Metro WA ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Primary school or less than Year 10 M 24 Yes No No No 30 42134 Group 1 Nepean NSW ESS FL2 Psychiatric Anxiety Year 10/11 M 22 Yes No No No 26 30134 Group 2 Nepean NSW ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 Yes No No No 45 38135 Trial Maroondah VIC ESS FL1 Psychiatric Post Traumatic Stress disorder Not Stated M 18 No No No No 2 0135 Group 1 Central/West Metro ESS FL1 Psychiatric Depression Not Stated M 18 No No No No 1 0135 Group 2 Central/West Metro ESS FL1 Autism Autism Not Stated M 18 No No No No 3 0136 Trial North Brisbane QLD ESS FL2 Psychiatric Depression Year 10/11 M 22 No No No No 62 37136 Group 1 South Eastern NSW ESS FL2 Intellectual Intellectual Disability Primary school or less than Year 10 M 22 No No No No 70 43136 Group 2 Eureka VIC ESS FL2 Psychiatric Depression Primary school or less than Year 10 M 21 No No No No 60 42137 Trial North Brisbane QLD DMS Psychiatric Psychotic Diploma or equivalent M 22 No Yes No No 3 24137 Group 1 Nepean NSW DMS Psychiatric Psychotic Non-trade vocational education M 23 No Yes No No 10 9137 Group 2 South Eastern NSW DMS Psychiatric Obsessive Compulsive Disorder Secondary schooling completed (Year 12/13) M 20 No Yes No No 1 36138 Trial North Metro WA ESS FL2 Psychiatric Schizophrenia Secondary schooling completed (Year 12/13) F 23 No No No No 26 31138 Group 1 Latrobe Valley VIC ESS FL2 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) F 21 No No No No 35 31138 Group 2 Warwick QLD ESS FL2 Specific Learning Learning Disability Secondary schooling completed (Year 12/13) F 21 No No No No 25 35139 Trial Maroondah VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 22 No No No No 60 36139 Group 1 Toowoomba QLD ESS FL1 Psychiatric Anxiety Non-trade vocational education F 22 No No No No 54 37139 Group 2 Liverpool NSW ESS FL1 Psychiatric Anxiety Non-trade vocational education F 22 No No No No 81 45140 Trial North Brisbane QLD DMS Psychiatric Depression Non-trade vocational education F 24 No No No No 15 36140 Group 1 Toowoomba QLD DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education F 24 No No No No 14 14140 Group 2 Eureka VIC DMS Psychiatric Depression Non-trade vocational education F 23 No No No No 14 30141 Trial North Metro WA ESS FL2 Psychiatric Anxiety Year 10/11 M 21 No No No No 43 0141 Group 1 Liverpool NSW ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 21 No No No No 48 38141 Group 2 Nepean NSW ESS FL2 Psychiatric Anxiety Year 10/11 M 20 No No No No 48 27142 Trial North Brisbane QLD DMS Psychiatric Anxiety Non-trade vocational education M 24 No No No No 82 36142 Group 1 Warwick QLD DMS Psychiatric Depression Year 10/11 M 23 No No No No 56 35142 Group 2 Central/West Metro DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 23 No No No No 50 18143 Trial Maroondah VIC ESS FL1 Intellectual Intellectual Disability Non-trade vocational education F 24 No No No No 6 43143 Group 1 Latrobe Valley VIC ESS FL1 Neurological Cerebral Palsy Non-trade vocational education F 23 No No No No 1 44143 Group 2 Warwick QLD ESS FL1 Psychiatric Schizophrenia Non-trade vocational education F 24 No No No No 4 30144 Trial North Brisbane QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 22 No No No No 16 35144 Group 1 Latrobe Valley VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Diploma or equivalent M 21 No No No No 13 30144 Group 2 Central/West Metro ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 22 No No No No 14 28145 Trial North Brisbane QLD ESS FL1 Psychiatric Psychotic Secondary schooling completed (Year 12/13) M 20 No No No No 15 36145 Group 1 Central/West Metro ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 20 No No No No 15 17145 Group 2 Liverpool NSW ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) M 22 No No No No 9 38

112

146 Trial North Metro WA ESS FL1 Psychiatric Schizophrenia Year 10/11 M 19 No No No No 22 20146 Group 1 Eureka VIC ESS FL1 Psychiatric Depression Year 10/11 M 18 No No No No 20 24146 Group 2 Toowoomba QLD ESS FL1 Psychiatric Anxiety Year 10/11 M 18 No No No No 19 34147 Trial Maroondah VIC DMS Psychiatric Anxiety Year 10/11 F 24 No No No No 19 33147 Group 1 Central/West Metro DMS Psychiatric Depression Year 10/11 F 23 No No No No 24 31147 Group 2 Latrobe Valley VIC DMS Psychiatric Depression Non-trade vocational education F 24 No No No No 23 39148 Trial North Brisbane QLD DMS Psychiatric Depression Non-trade vocational education F 22 No No No No 16 36148 Group 1 Nepean NSW DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 22 No No No No 14 19148 Group 2 Central/West Metro DMS Psychiatric Personality Disorder Year 10/11 F 21 No No No No 17 25149 Trial Maroondah VIC DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 19 No No No No 18 17149 Group 1 Latrobe Valley VIC DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 18 No No No No 10 32149 Group 2 Toowoomba QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 18 No No No No 11 20150 Trial Maroondah VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 20 No No No No 27 46150 Group 1 Central/West Metro ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 21 No No No No 32 33150 Group 2 Goldfields/Esperance ESS FL1 Psychiatric Post Traumatic Stress disorder Non-trade vocational education F 20 No No No No 35 40151 Trial Maroondah VIC ESS FL1 Specific Learning Attention Deficit/Hyperactivity disorder Secondary schooling completed (Year 12/13) M 22 Yes No No No 33 38151 Group 1 Toowoomba QLD DMS Psychiatric Depression Bachelor Degree or equivalent M 21 Yes No No No 26 33151 Group 2 Liverpool NSW DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 22 Yes No No No 22 26152 Trial North Metro WA DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) F 20 No No No No 36 34152 Group 1 Inner Sydney NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 22 No No No No 30 43152 Group 2 Eureka VIC DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) F 20 No No No No 25 26153 Trial North Brisbane QLD ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 22 No No No No 10 38153 Group 1 Central/West Metro ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education M 20 No No No No 11 20153 Group 2 South Eastern NSW ESS FL1 Intellectual Intellectual Disability Non-trade vocational education M 21 No No No No 9 22154 Trial North Metro WA ESS FL2 Intellectual Intellectual Disability Special school or special support unit M 18 No No No No 34 27154 Group 1 Eureka VIC ESS FL2 Autism Aspergers Syndrome Year 10/11 M 18 No No No No 36 23154 Group 2 South Eastern NSW ESS FL2 Psychiatric Depression Year 10/11 M 19 No No No No 28 28155 Trial Maroondah VIC ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 25 No No No No 63 39155 Group 1 Liverpool NSW ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) F 23 No No No No 51 42155 Group 2 Toowoomba QLD ESS FL1 Psychiatric Personality Disorder Non-trade vocational education F 23 No No No No 60 34156 Trial Maroondah VIC ESS FL1 Psychiatric Obsessive Compulsive Disorder Non-trade vocational education M 23 No No No No 17 29156 Group 1 Liverpool NSW ESS FL1 Psychiatric Psychotic Secondary schooling completed (Year 12/13) M 24 No No No No 17 35156 Group 2 Eastern Adelaide SA ESS FL1 Psychiatric Depression Graduate Certificate or equivalent M 23 No No No No 14 34157 Trial North Metro WA DMS Psychiatric Depression Special school or special support unit M 23 No Yes No No 47 39157 Group 1 Eureka VIC DMS Psychiatric Depression Year 10/11 M 23 No Yes No No 43 32157 Group 2 Eureka VIC DMS Psychiatric Anxiety Year 10/11 M 23 No Yes No No 37 42158 Trial North Brisbane QLD ESS FL2 Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) F 22 No No No Yes 58 40158 Group 1 Toowoomba QLD ESS FL2 Psychiatric Depression Non-trade vocational education F 21 No No No Yes 53 49158 Group 2 Liverpool NSW ESS FL2 Psychiatric Anxiety Primary school or less than Year 10 F 21 No No No Yes 61 44159 Trial Maroondah VIC ESS FL1 Psychiatric Depression Secondary schooling completed (Year 12/13) M 24 No No No No 3 31159 Group 1 South Eastern NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 24 No No No No 0 31159 Group 2 South Eastern NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 23 No No No No 3 21160 Trial North Brisbane QLD DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 20 No No Yes No 24 31160 Group 1 Gladstone QLD DMS Psychiatric Depression Year 10/11 F 19 No No Yes No 23 37160 Group 2 Eureka VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education F 21 No No Yes No 29 37

113

161 Trial North Metro WA ESS FL2 Psychiatric Personality Disorder Non-trade vocational education F 19 No No No No 16 23161 Group 1 Liverpool NSW ESS FL2 Specific Learning Learning Disability Year 10/11 F 19 No No No No 13 44161 Group 2 Liverpool NSW ESS FL2 Psychiatric Depression Not Stated F 19 No No No No 15 0162 Trial Maroondah VIC ESS FL2 Psychiatric Schizophrenia Primary school or less than Year 10 F 20 No No No No 25 46162 Group 1 Inner Sydney NSW ESS FL2 Psychiatric Anxiety Year 10/11 F 20 No No No No 34 45162 Group 2 Liverpool NSW ESS FL2 Specific Learning Learning Disability Year 10/11 F 20 No No No No 26 34163 Trial North Brisbane QLD ESS FL1 Psychiatric Obsessive Compulsive Disorder Secondary schooling completed (Year 12/13) M 21 No No No No 49 27163 Group 1 Central/West Metro ESS FL1 Psychiatric Psychotic Non-trade vocational education M 21 No No No No 52 21163 Group 2 Central/West Metro ESS FL1 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 22 No No No No 49 34164 Trial Maroondah VIC DMS Psychiatric Anxiety Primary school or less than Year 10 M 18 No No No No 24 31164 Group 1 Nepean NSW DMS Specific Learning Learning Disability Not Stated M 18 No No No No 20 0164 Group 2 Inner Sydney NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Not Stated M 18 No No No No 20 0165 Trial Maroondah VIC DMS Physical Shoulder and Upper Arm Disorder Non-trade vocational education M 23 No Yes No No 38 25165 Group 1 Eastern Adelaide SA DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 24 No Yes No No 40 39165 Group 2 Inner Sydney NSW DMS Psychiatric Depression Year 10/11 M 24 No Yes No No 89 32166 Trial Maroondah VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 23 No No No No 32 7166 Group 1 Toowoomba QLD DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 23 No No No No 41 30166 Group 2 Liverpool NSW DMS Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 24 No No No No 40 35167 Trial Maroondah VIC DMS Psychiatric Anxiety Year 10/11 M 22 No Yes No No 59 45167 Group 1 Eastern Adelaide SA DMS Psychiatric Anxiety Year 10/11 M 24 No Yes No No 55 31167 Group 2 Nepean NSW DMS Psychiatric Depression Primary school or less than Year 10 M 24 No Yes No No 59 44168 Trial Maroondah VIC ESS FL1 Neurological Hemiplegia (Paralysis) Non-trade vocational education M 23 No No No No 13 38168 Group 1 Toowoomba QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 24 No No No No 13 31168 Group 2 Nepean NSW ESS FL1 Psychiatric Anxiety Secondary schooling completed (Year 12/13) M 23 No No No No 18 30169 Trial North Metro WA DMS Specific Learning Attention Deficit/Hyperactivity disorder Not Stated M 22 No No No No 1 0169 Group 1 Central/West Metro DMS Psychiatric Anxiety Non-trade vocational education M 21 No No No No 4 28169 Group 2 South Eastern NSW DMS Psychiatric Depression Non-trade vocational education M 21 No No No No 3 35170 Trial North Brisbane QLD ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Non-trade vocational education M 24 No No No Yes 73 41170 Group 1 Nepean NSW ESS FL2 Psychiatric Anxiety Non-trade vocational education M 24 No No No Yes 75 45170 Group 2 Nepean NSW DMS Psychiatric Anxiety Year 10/11 M 24 No No No Yes 83 24171 Trial North Metro WA ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Year 10/11 M 19 No Yes No No 11 35171 Group 1 Nepean NSW ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education M 21 No Yes No No 1 19171 Group 2 Warwick QLD ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Year 10/11 M 22 No Yes No No 5 41172 Trial Maroondah VIC ESS FL1 Psychiatric Personality Disorder Non-trade vocational education M 19 No No No No 10 32172 Group 1 Nepean NSW ESS FL1 Psychiatric Anxiety Not Stated M 19 No No No No 8 0172 Group 2 Latrobe Valley VIC ESS FL1 Neurological Cerebral Palsy Not Stated M 19 No No No No 11 0173 Trial North Metro WA DMS Psychiatric Depression Non-trade vocational education F 25 No No No No 21 25173 Group 1 Toowoomba QLD DMS Psychiatric Bi Polar Affective Disorder (Manic Depression) Secondary schooling completed (Year 12/13) F 24 No No No No 24 27173 Group 2 Nepean NSW DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 23 No No No No 20 27174 Trial North Metro WA ESS FL1 Psychiatric Depression Year 10/11 M 20 No No No No 2 5174 Group 1 Nepean NSW ESS FL1 Specific Learning Learning Disability Not Stated M 22 No No No No 2 0174 Group 2 Nepean NSW ESS FL1 Psychiatric Anxiety Not Stated M 20 No No No No 3 0175 Trial North Metro WA ESS FL1 Psychiatric Anxiety Not Stated F 23 No No No No 6 0175 Group 1 South Eastern NSW ESS FL1 Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 23 No No No No 1 36175 Group 2 Toowoomba QLD ESS FL1 Physical Irritable Bowel Syndrome Not Stated F 24 No No No No 2 0

114

176 Trial Maroondah VIC ESS FL1 Psychiatric Depression Year 10/11 F 21 No No No No 13 45176 Group 1 Eastern Adelaide SA ESS FL1 Psychiatric Anxiety Year 10/11 F 20 No No No No 14 33176 Group 2 Central/West Metro ESS FL1 Autism Autism Not Stated F 20 No No No No 16 0177 Trial North Metro WA DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 21 No Yes No No 14 29177 Group 1 Toowoomba QLD DMS Psychiatric Anxiety Year 10/11 F 22 No Yes No No 10 25177 Group 2 Liverpool NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) F 21 No Yes No No 10 33178 Trial North Metro WA DMS Neurological Cerebral Palsy Secondary schooling completed (Year 12/13) F 20 No No No No 32 25178 Group 1 Eastern Adelaide SA DMS Psychiatric Depression Secondary schooling completed (Year 12/13) F 20 No No No No 31 28178 Group 2 Latrobe Valley VIC DMS Psychiatric Anxiety Diploma or equivalent F 22 No No No No 35 40179 Trial Maroondah VIC ESS FL2 Psychiatric Psychol/Psychiatric Disorder - Other Secondary schooling completed (Year 12/13) M 20 No No No No 8 26179 Group 1 Nepean NSW ESS FL2 Autism Autism Secondary schooling completed (Year 12/13) M 22 No No No No 2 32179 Group 2 South Eastern NSW ESS FL2 Autism Aspergers Syndrome Secondary schooling completed (Year 12/13) M 22 No No No No 5 40180 Trial Maroondah VIC DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 M 21 No No No No 12 10180 Group 1 South Eastern NSW DMS Psychiatric Psychol/Psychiatric Disorder - Other Primary school or less than Year 10 M 22 No No No No 12 43180 Group 2 Latrobe Valley VIC DMS Psychiatric Anxiety Year 10/11 M 20 No No No No 11 27181 Trial Maroondah VIC DMS Psychiatric Anxiety Year 10/11 F 22 No No No No 36 41181 Group 1 Central/West Metro DMS Psychiatric Psychol/Psychiatric Disorder - Other Year 10/11 F 22 No No No No 36 39181 Group 2 East Gippsland VIC DMS Psychiatric Anxiety Year 10/11 F 20 No No No No 31 45182 Trial North Metro WA ESS FL2 Psychiatric Anxiety Year 10/11 M 19 No No No No 7 29182 Group 1 East Gippsland VIC ESS FL2 Psychiatric Anxiety Year 10/11 M 18 No No No No 9 14182 Group 2 East Gippsland VIC ESS FL2 Specific Learning Attention Deficit/Hyperactivity disorder Primary school or less than Year 10 M 18 No No No No 11 37183 Trial Maroondah VIC ESS FL1 Psychiatric Personality Disorder Non-trade vocational education F 23 No No No No 18 32183 Group 1 Nepean NSW ESS FL1 Psychiatric Bi Polar Affective Disorder (Manic Depression) Non-trade vocational education F 24 No No No No 15 29183 Group 2 Central/West Metro ESS FL1 Psychiatric Psychotic Non-trade vocational education F 23 No No No No 15 21184 Trial North Metro WA ESS FL1 Psychiatric Anxiety Non-trade vocational education F 21 No No No No 57 32184 Group 1 Nepean NSW ESS FL1 Psychiatric Depression Non-trade vocational education F 20 No No No No 52 29184 Group 2 Central/West Metro ESS FL1 Psychiatric Anxiety Non-trade vocational education F 22 No No No No 53 35

115

12.Appendix H: Power Analysis

Colmar Brunton conducted a Power Analysis to determine whether there was an appropriate number of subjects within the trial. This analysis was conducted for 13 week outcomes to ascertain if the number in the trial was sufficient to obtain a significant finding. We reject the null hypothesis that the population means of the trial and control groups are equal with probability (power) 0.96 (The Type I error probability associated with this test of this null hypothesis is 0.05). This Power Analysis is able to validate the sample size from a statistical standpoint, however it is not an indication of robustness in control for other potential confounding variables.

Figure 14: Power Analysis calculations

Calculations:

# 13 week Job outcomes for trial 11

# 13 week Job outcomes for Control 65

% difference in outcomes -11.7

Odds Ratio 0.33

Detectable at Power 0.95

Sample Size trial 184

Sample Size Control 368

116

Colmar Brunton Social Research

CANBERRA ACT 2601

Ph. (02) 6249 8566

ACN NO: 003 748 981

ABN NO: 22 003 748 981

This document takes into account the particular instructions and requirements of our Client. It is not intended for and should not be relied upon by any third party and no responsibility is undertaken to

any third party.

117

Documents

Executive summary. - DSS Web viewxxviii. 10/11/2016des youth mental health trial - evaluation report . xxviii. xxviii. 10/11/2016des youth mental health trial - evaluation report