ORIGINAL PAPER

A multifaceted intervention to improve mental health literacyin students of a multicampus university: a cluster randomised trial

Nicola J. Reavley • Terence V. McCann •

Stefan Cvetkovski • Anthony F. Jorm

Received: 2 September 2013 / Accepted: 14 April 2014

� Springer-Verlag Berlin Heidelberg 2014

Abstract

Purpose The aim of the current study was to assess

whether a multifaceted intervention could improve mental

health literacy, facilitate help seeking and reduce psycho-

logical distress and alcohol misuse in students of a multi-

campus university in Melbourne, Australia.

Methods In this cluster randomized trial, nine university

campuses were paired (some pairs included more than one

campus), with one of each pair randomly assigned to either

the intervention or control condition. The interventions

were designed to be whole-of-campus and to run over

2 academic years with their effectiveness assessed through

recruitment of a monitoring sample of students from each

campus. Interventions included emails, posters, campus

events, factsheets/booklets and mental health first aid

training courses. Participants had a 20-min telephone

interview at baseline and at the end of academic years 1

and 2. This assessed mental health literacy, help seeking,

psychological distress and alcohol use. The primary out-

comes were depression and anxiety levels and alcohol use

and pertained to the individual level.

Results There were no effects on psychological distress

and alcohol use. Recall of intervention elements was

greater in the intervention group at the end of year 2.

Students in the intervention group were more likely to say

they would go to a drug and alcohol centre for alcohol

problems at the end of 6 months.

Conclusion Although education and awareness may play

a role in improving mental health literacy, it is likely that,

to achieve changes in psychological distress, interventions

would need to be more personalized and intensive.

Keywords Mental health literacy � Students �Depression � Anxiety � Alcohol misuse

Background

Mental and substance use disorders have their first onset

before age 24 in 75 % of cases [1, 2]. At this age, many

young people are in tertiary education and while many

enjoy and effectively manage their responsibilities, ana-

lysis of data from Australian national surveys reveals that

tertiary students are at higher risk of moderate (but not

high) psychological distress as compared to non-students

[3], particularly those from lower socioeconomic back-

grounds [4]. In addition, excessive alcohol consumption

and substance misuse are very common among students [5,

6], with a recent study conducted in an Australian uni-

versity reporting that 90 % of students had consumed

alcohol in the last 12 months and 34 % met criteria for

hazardous drinking [7].

Very high levels of psychological distress in students have

been associated with a reduced capacity for work [8], and

mental health problems have been shown to affect both exam

performance and tertiary education dropout rates [9–11].

N. J. Reavley (&) � S. Cvetkovski � A. F. Jorm

Orygen Youth Health Research Centre, Centre for Youth Mental

Health and Melbourne School of Population and Global Health,

University of Melbourne, 207 Bouverie St, Parkville,

VIC 3010, Australia

e-mail: nreavley@unimelb.edu.au

T. V. McCann

College of Health and Biomedicine, Victoria University,

St Albans Campus, Building 4, Level 3,

PO Box 14428, Melbourne 8001, Australia

123

Soc Psychiatry Psychiatr Epidemiol

DOI 10.1007/s00127-014-0880-6

Such educational impacts may have lifelong consequences,

particularly if students do not complete their courses.

As with other young people, tertiary students need to

know how to take action to deal with mental health prob-

lems, whether that is professional help seeking or appro-

priate self-help behaviours. Taking appropriate action is

influenced by a number of interacting individual and

structural factors [12]. Individual factors, include stigma

and mental health literacy, which may be defined as

‘‘knowledge and beliefs about mental disorders that aid

their recognition, management or prevention’’ [13]. Struc-

tural factors include family, educational institution or

community support systems, and health system structures

[14]. In an effort to support students with mental health

problems, Australian tertiary education institutions typi-

cally offer a number of services, including counselling

services and disability liaison units. However, the great

majority of young people with depression and related dis-

orders either do not seek or delay seeking professional

help, and there is some evidence that they are also reluctant

to access formal disability support services in tertiary

education institutions [15, 16].

Evidence of the relatively low levels of mental health

literacy in members of the public has led to community

campaigns to improve this in a number of countries and

such campaigns have tended to focus on mental illness

broadly, or specifically on depression, and centre around

raising awareness of signs and symptoms and promotion of

professional help seeking [17–20]. Despite this, our

knowledge about the impact of these campaigns is rela-

tively limited and there is no evidence that such campaigns

help to reduce psychological distress, increase help seeking

or decrease suicidal behaviour [20]. It can be argued that

there is a need for a new generation of community cam-

paigns that aim for a more intensive level of intervention,

incorporating a broader range of mental health-related

messages and such campaigns could potentially produce a

cultural shift in knowledge and attitudes and may even lead

to improvements in mental health. Ideally, such campaigns

need to be directed at the stage in the lifespan where first

onset of disorders is most likely, namely youth and their

supporters, and should also be rigorously evaluated.

Because they often encompass several aspects of stu-

dents’ lives, including educational activities, health ser-

vices, residences, social networks and extracurricular

activities, educational institutions offer a unique setting for

prevention of and early intervention for mental health

problems. A number of mental health promotion inter-

ventions have been carried out in high schools, but tertiary

education institutions have typically received less attention

[21, 22]. An exception to this is in the area of prevention or

early intervention for alcohol misuse, for which evidence

of effectiveness is strongest for brief motivational

interventions and for personalised normative feedback

interventions delivered using computers or in individual

face-to-face sessions [23, 24].

There have been very few interventions aiming to pre-

vent or intervene early with depression or anxiety disorders

in tertiary students [25]. There is some evidence in support

of face-to-face cognitive behavioural/skills-based inter-

ventions [26, 27], and one social marketing intervention to

raise awareness of depression and treatments showed some

evidence of effectiveness [28].

The aim of the current study was to assess whether

MindWise, a multifaceted intervention, could improve

mental health literacy, facilitate help seeking and reduce

psychological distress and alcohol misuse in staff and

students of Victoria University (VU) in metropolitan

Melbourne, Australia. This paper describes the findings

from the intervention aimed at students. Findings from the

staff intervention have been described in an accompanying

paper [29].

Methods

Study design

VU is a multicampus institution and offers a broad mix of

courses, including a large number of vocational education

courses (including technical and trades), as well as higher

education courses. The MindWise study was a cluster

randomised trial with campuses as clusters and individual

students as participants. A cluster design was used because

it was not feasible to randomly assign individual students

studying on the same campus, as there may have been

contamination of information provided across groups

within the same campus. The trial was registered with the

Australian and New Zealand Clinical Trials Registry

(ACTRN12610001027000).

Participants

Clusters

Eligible clusters were all VU campuses. Senior VU staff

were informed of and agreed to the intervention prior to

implementation.

Individuals

Eligible participants were all students of VU who were

willing to participate in the study. To monitor the effec-

tiveness of the intervention, a sample of individuals was

recruited from each campus. Students were invited to

participate through emails and postcards handed out by

Soc Psychiatry Psychiatr Epidemiol

123

researchers on the nine largest VU campuses. Participants

were able to indicate agreement to participate via a website

or by returning the postcards to the researchers. Those who

did so were contacted by the survey company The Social

Research Centre and participated in a computer-assisted

telephone interview between March and May 2010 (the

baseline/Wave 1 interview). However, only those who

planned to study for at least 6 months were asked to con-

tinue the interview, as those attending for shorter periods

would have little or no exposure to the intervention. Fol-

low-up interviews were conducted in October–November

2010 (Wave 2) and October–November 2011 (Wave 3).

The results from the baseline survey have been reported

previously [30–32]. At each interview point, participants

received a free coffee voucher and also had a 1-in-12

chance of winning a voucher worth AU$50.

Interventions

The MindWise interventions were designed to be whole-of-

campus and to run over 2 academic years (January/Feb-

ruary to November/December in the majority of courses).

The interventions, which were designed with input from

student and staff focus groups and a student advisory

group, incorporated the following key messages: (1)

depression and related disorders are common in young

people; (2) there are recognizable signs of depression and

related disorders in young people; (3) early help seeking

leads to better outcomes; (4) there are several sources of

professional help available; (5) there are useful types of

self-help available; and (6) there are helpful first aid actions

that staff and peers can take. Other messages included

information on safe consumption of alcohol and NHMRC

guidelines for alcohol consumption (defined as no more

than two drinks per day to reduce lifetime risk of harm, and

no more than four drinks on any one occasion to reduce

risk of injury arising from that occasion) [33].

These messages were delivered in a variety of ways,

through: website/Facebook pages, Twitter activities; fact-

sheets/booklets (primarily provided by beyondblue: the

national depression initiative (http://www.beyondblue.org.

au); emails to student email addresses; booklets; campus

special events including stalls during orientation and at

exam times (at which students were also offered the

opportunity to complete online personalized feedback

questionnaires about their alcohol consumption); posters on

the back of campus toilet doors; student-designed projects

(including awareness-raising events by youth work stu-

dents); information on the electronic course information

and notes delivery system; and mental health first aid

(MHFA) training (http://www.mhfa.com.au) provided by

staff of the student counseling service. The intervention

was aimed at both staff and students.

Objectives

For students, the hypotheses tested were that the MindWise

intervention improves the following: psychological dis-

tress, alcohol misuse, recognition of mental health prob-

lems, help-seeking intentions and actions, beliefs about the

helpfulness of interventions, stigmatizing attitudes, per-

ceptions of how well the VU community supports staff and

students with mental health problems, knowledge of

NHMRC guidelines for the safe consumption of alcohol

and beliefs about help seeking for alcohol misuse. The

primary outcome measures for the trial were psychological

distress and alcohol use. All hypotheses pertained to the

individual rather than the cluster level.

Outcomes

Respondents were asked questions about their sociode-

mographic characteristics, their job role, their main campus

of study and whether they regularly visited a campus

other than the one nominated as their principal campus at

Wave 1.

The following outcomes were measured at the individ-

ual level:

Recognition of depression in a vignette

Interviews were based around a vignette of a 21-year-old

person (‘John’ or ‘Jenny,’ depending on the gender of the

respondent) with depression [13]. Respondents were asked

an open-ended question about what they thought was

wrong with the person. Responses which mentioned

‘‘depression’’ were scored as correct.

Help-seeking intentions

Respondents were asked if they would seek help if they had

a problem similar to that described in the vignette and an

open-ended question about what they would do to seek

help. Respondents were assigned to the category ‘Would

seek professional source of help’ if they nominated one or

more of VU student counsellor, GP, psychiatrist, psychol-

ogist, other counsellor or other professional.

Beliefs about the helpfulness of interventions

Students were given a list of 36 categories of people,

medicines or other interventions and asked whether each of

them is likely to be helpful, harmful or neither for the

person described in the vignette. Respondents were asses-

sed according to whether or not they nominated as helpful

all 14 of a list of items rated as helpful by 70 % of clini-

cians in a survey of health professionals [34]. These

Soc Psychiatry Psychiatr Epidemiol

123

included a GP, a student counsellor, a counsellor not

employed by VU, a psychologist, a psychiatrist, antide-

pressants, becoming more physically active, getting relax-

ation training, receiving counselling, receiving cognitive

behaviour therapy (CBT), going to a local mental health

service, cutting down on alcohol, cutting down on smoking

cigarettes and cutting down on marijuana.

Stigmatizing attitudes

Stigmatizing attitudes were assessed using a social distance

scale modified to be appropriate to the age group [35] and

the Depression Stigma Scale (personal subscale) [36]. For

analysis of these scales, scale scores were dichotomised at

the 50th percentile.

Help-seeking actions taken

Respondents were asked whether or not they had a problem

like the person in the vignette and if so, what actions they

took for the problem. Responses were assessed against a

list of 14 items rated as helpful by 70 % of clinicians in a

survey of health professionals (see above) [34].

Mental health first aid given to family or friends

Respondents were asked if anyone in their family or close

circle of friends had a problem similar to the person in the

vignette in the last 12 months. Those that did were asked if

they did anything to help. Possible responses were ‘Yes’ or

‘No’.

Psychological distress

Respondents were asked about their level of psychological

distress using the Kessler 6 (K6), a 6-item screening scale

(scored from 6 to 30) with strong psychometric properties

that is able to discriminate DSM-IV cases from non-cases

and has been used widely in general-purpose health surveys

[37]. For these analyses, scale scores were dichotomised

using a cut off C15 to assess moderate-to-high psycho-

logical distress.

Knowledge of NHMRC guidelines for safe alcohol

consumption

Participants were asked about their knowledge of the

NHMRC guidelines for alcohol consumption (defined as no

more than two drinks per day to reduce lifetime risk of

harm, and no more than four drinks on any one occasion to

reduce risk of injury arising from that occasion) [33].

Alcohol use

Those who had drunk alcohol at some point in their lives

were administered the Alcohol Use Disorders Identification

Test (AUDIT) [38]. The AUDIT is a 10-item questionnaire

designed to screen for early stage problem drinking [39,

40]. For these analyses, a cut off C8 was used to assess

risky or hazardous levels of drinking.

Help-seeking intentions and actions taken for alcohol

problems

Participants who drank alcohol regularly were asked about

help-seeking intentions for alcohol problems. For these

analyses, respondents were included in the category

‘Would seek professional help for alcohol problems’ if

they nominated one or more of: VU student counsellor, GP,

psychiatrist, psychologist, other counsellor, drug and

alcohol service or other professional. They were included

in the category ‘‘Would go to a mental health professional

for alcohol problem’’ if they nominated one or more of a

counsellor, psychiatrist or psychologist.

Perceptions of VU support for staff and students

with mental health problems

Participants were also asked about how well Victoria

University supports students with mental health problems.

Possible responses were ‘Very well’, ‘Well’, ‘Poorly’ and

‘Very poorly’.

Follow-up interview

At follow-up, this interview was repeated and students

were also asked a series of questions about their exposure

to specific MindWise intervention elements, including

website/Facebook pages, Twitter activities; factsheets/

booklets; emails; booklets; campus special events; partic-

ipation in competitions, posters on the back of campus

toilet doors; student-designed projects (including aware-

ness-raising events by youth work students); information

on the electronic course information and notes delivery

system; and MHFA training. Possible responses were ‘yes’,

‘no’ and ‘don’t know’.

Sample size estimation

Required sample size was estimated using software for

power analysis in cluster randomized trials [41]. In the

absence of relevant data for the primary outcomes of

changes in psychological distress and alcohol misuse,

likely effect sizes were based on those effect sizes which

Soc Psychiatry Psychiatr Epidemiol

123

equal or exceed Cohen’s definition of a ‘small’ (h C 0.20)

effect size [42]. To detect these effects in an unclustered

trial with 80 % power at the 0.05 significance level,

required n = 200. In the current study, six clusters were

used for the purpose of sample size and power calculations.

With a sample of 200 persons per cluster, power = 80 %

and alpha = 0.05, it was estimated that the study would

detect a standardized effect size of 0.32 if the intraclass

correlation is 0.01, and one of 0.56 if the intraclass cor-

relation is 0.05. Thus, the study would have had good

power to detect medium effect sizes (h = 0.50). Although

we aimed for a sample of 200 per cluster (1,200 overall) we

managed to recruit and randomise 767 students at Wave 1

in the period of time before the intervention started. The

reasons for the lower level of recruitment remain unclear,

although they may include refusal to answer phone calls

from the survey company after submitting contact details,

provision of incorrect contact details, and subsequent lack

of interest in the study.

Randomization

Nine campuses were paired on the basis of their size

(number of staff and students) and the type of courses

offered. Owing to their small sizes, in one case, two

campuses were combined to make one of pair and in

another case; three campuses were combined to make one

of a pair. Using the random integers option of Random.org,

two researchers (NR and AFJ) randomly assigned one of

each pair to either the intervention or control condition by

generating a 1 or a 2 for each pair (1 = intervention,

2 = control). Overall, there were three pairs, with six

campuses allocated to the control group and three cam-

puses allocated to the intervention group. There was no

allocation concealment, as allocation was based on clusters

rather than individuals, so that all students at a campus

received the same intervention.

Ethics

This study was approved by VU Human Research

Ethics Committee.

Statistical analysis

To assess whether there were any significant differences at

Wave 1, descriptive statistics were used to examine the

distributions of the sociodemographic characteristics

between the intervention and control groups and differ-

ences were tested using t tests, Pearson’s v2 tests and

Wald tests. To test the hypotheses of the study, group

(intervention vs. control) by measurement occasion (pre-,

post- and follow-up) interactions were examined using

logistic and Poisson mixed models for binary and count

outcomes, respectively. All results are reported as odds

ratios (OR) or incidence rate ratios (IRR), with 95 %

confidence intervals (95 % CIs). The strength of these

mixed models is that they can account for the correlation of

individual responses over time and the correlation of

individual responses within campuses.

Missing data were handled using uncongenial, multi-

variate imputation chained equations and were assumed to

be missing at random (MAR) [43, 44]. A series of logistic

regression models revealed that six covariates predicted

dropout from the monitoring sample: employment status,

recognition of depression, personal stigmatizing attitudes

towards others with mental health problems, awareness of

Australia’s guidelines on safe levels of drinking, the

maximum number of drinks recommended on one occasion

and the AUDIT. These covariates along with the covariates

of interest in the substantive analyses were included in the

conditional imputation models, which accounted for the

measurement level of the variables imputed (e.g., logistic

and Poisson regressions for binary and count outcomes,

respectively), with the exception of two outcomes that

related to a subsample of students, who had mental health

problems or who had family or friends with mental health

problems. It was necessary to exclude the maximum

number of drinks recommended on one occasion and the

AUDIT covariates from the multivariate imputation

chained equations for these models to converge. All mul-

tiple imputations were conducted in wide format to take

into consideration the dependence of measurement occa-

sions within respondents and then reshaped into long for-

mat for the mixed-model analyses. Twenty imputed

datasets were generated for each substantive analysis.

Although the MAR assumption is more plausible due to

the inclusion of predictors of dropout along with the other

covariates of interest in the substantive analyses in the

imputation models [45], we conducted sensitivity analyses

of significant results by introducing assumptions about

personal stigmatizing attitudes towards others with mental

health problems as the missing not at random (MNAR)

mechanism in the multiple imputation process [43, 46].

Given this is a binary variable, we introduced two MNAR

assumptions into the models in turn: (1) that all dropouts

had stigmatizing attitudes and (2) all dropouts had non-

stigmatizing attitudes towards others with mental health

problems. This is one of the more important predictors of

dropout and is arguably a plausible reason for why

respondents may have lost interest in remaining in the

study. All multiple imputations and substantive analyses

were performed using Stata IC/13.1 (StataCorp LP, TX

USA).

Soc Psychiatry Psychiatr Epidemiol

123

Results

Participant flow

Figure 1 shows the flow of participants at each stage of the

trial.

Numbers analysed

Wave 1 interviews were completed by 767 students from

the 9 randomized campuses: 426 from intervention cam-

puses and 341 from control campuses. The patterns of

completed interviews by students were as follows: for the

intervention group, 45.3 % completed all 3 waves of

interviews, 27.9 % completed only the Wave 1 interview,

22.3 % completed Waves 1 and 2 interviews, and 4.5 %

completed Waves 1 and 3 interviews; for the control group,

38.1 % completed all 3 waves of interviews, 35.2 %

completed only the Wave 1 interview, 23.8 % completed

Waves 1 and 2 interviews, and 2.9 % completed Waves 1

and 3 interviews. Of the total sample of 767 students, 499

completed Wave 2 interviews and 352 completed Wave 3

interviews, representing a 34.9 and 54.1 % loss to follow-

up, respectively. The proportions of students lost to follow-

up at Waves 2 and 3 were similar between the intervention

and control groups.

Table 1 shows the sociodemographic characteristics of

students from the Wave 1 interview. Comparison of

intervention and control groups revealed that the inter-

vention group had a significantly larger proportion of

female students than the control group (69.3 vs. 52.5 %,

v2ð1Þ = 22.53, p \ 0.001), a larger proportion of students

born in Australia than the control group (73.7 vs. 67.2 %,

v21ð Þ = 3.93, p = 0.047) and a smaller proportion of stu-

dents working full time than the control group (6.1 vs.

13.8 %, F(1,766) = 12.18, p = 0.003). To adjust for these

differences between the groups, gender (female vs. male),

country of birth (Australia vs. other) and employment

status (casual work and other vs. full time work) were

included in all models as covariates.

Fig. 1 Flow of participants at

each stage of the trial

Soc Psychiatry Psychiatr Epidemiol

123

Outcomes

Exposure to MindWise intervention elements

Comparison of exposure to MindWise intervention elements

is given in Table 2. When asked about their recall of specific

MindWise intervention elements, those in the intervention

group had higher odds of seeing information in course unit

guides or on web CT/blackboard, receiving emails and see-

ing factsheets or wallet cards at Wave 2 only. Odds of

recalling seeing posters, attending campus events were

greater at Wave 2 and Wave 3. The intervention group had

greater recall of receiving MindWise merchandise at Wave

3. Overall, those in the intervention group had higher odds of

recalling exposure to MindWise at both Waves 2 and 3.

Sensitivity analyses showed that these results were robust

to assumptions about personal stigmatizing attitudes towards

others with mental health problems as the MNAR mecha-

nisms, with the exception of the greater odds of recalling

receiving MindWise merchandise, which was a Wave 3 only

measure, where the results became marginal under both

assumptions: (1) that dropouts had stigmatizing attitudes

(OR = 1.88; 95 % CI 0.99–3.59; p = 0.055) and (2) non-

stigmatizing attitudes towards others with mental health

problems (OR = 1.83, 95 % CI 0.99–3.41; p = 0.055).

Table 1 Sociodemographic characteristics of respondents in the

intervention and control groups

Sociodemographiccharacteristics

Intervention(n = 426)

Control(n = 341)

Age: M (SD) 24.89 (8.02) 23.96 (8.89)

Gender

Male (%) 30.8 47.5

Female (%) 69.3 52.5

Australian citizens (%) 88.0 84.8

Country of birth Australia (%) 73.7 67.2

Education

Studying full time (%) 89.8 87.6

Bachelor degree (%) 54.2 55.4

Diploma (%) 21.3 24.9

Certificate I–IV (%) 14.6 13.5

Postgraduate (%) 6.3 1.5

Other (%) 3.8 4.7

Employment

Full time (%) 6.1 13.8

Part time (%) 22.7 16.7

Casual (%) 34.7 27.9

Contract (%) 1.2 0.3

Not working (%) 35.2 41.4

Looking for work (%) 60.4 67.4

Table 2 Comparison of exposures to MindWise between interven-

tion and control groups

Exposures Intervention Control OR (95 % CI)a

Recalls seeing MindWise information in course unit guides or on web

CT/blackboard: % yes

Wave 2 33.3 21.8 1.70 (1.07–2.70)*

Wave 3 33.5 25.7 1.47 (0.83–2.59)

Recalls seeing posters around campus: % yes

Wave 2 51.7 33.2 2.10 (1.38–3.18)**

Wave 3 76.9 62.1 1.94 (1.13–3.31)*

Recalls attending MindWise related events on campus: % yes

Wave 2 33.0 18.5 2.02 (1.31–3.11)**

Wave 3 26.4 12.1 2.86 (1.29–6.32)*

Recalls receiving emails: % yes

Wave 2 25.0 14.7 1.98 (1.14–3.43)*

Wave 3 29.3 30.7 0.92 (0.53–1.58)

Recalls receiving books or booklets about mental health: % yes

Wave 2 11.5 8.5 1.25 (0.67–2.33)

Wave 3 22.2 12.9 1.75 (0.86–3.54)

Recalls completing an online questionnaire about alcohol or mental

health: % yes

Wave 2 5.2 3.3 1.29 (0.55–3.02)

Wave 3 16.0 17.9 0.96 (0.48–1.91)

Recalls viewing a website related to MindWise: % yes

Wave 2 10.4 7.6 1.40 (0.71–2.76)

Wave 3 19.8 12.9 1.65 (0.83–3.29)

Recalls seeing MindWise factsheets or wallet cards: % yes

Wave 2 30.9 17.5 1.97 (1.26–3.07)**

Wave 3 46.2 35.0 1.56 (0.98–2.48)

Measures of exposure only at Wave 3

Recalls doing a

mental health first

aid course: % yes

6.6 4.3 2.10 (0.69–6.40)

Participated in a

competition run by

MindWise: % yes

19.8 13.6 1.74 (0.92–3.27)

Recalls receiving

MindWise

merchandise:

% yes

23.1 14.3 1.97 (1.03–3.75)*

Summary of specific MindWise exposures: % high exposureb

Wave 2 53.8 33.7 2.15 (1.31–3.51)**

Wave 3 54.7 35.7 2.21 (1.39–3.49)**

All models were adjusted for gender, country of birth (Australia vs.

other) and employment status (casual work and other vs. full time

work)

* p \ 0.05; ** p \ 0.01; *** p \ 0.001a Odds ratio and 95 % confidence intervalb For these analyses, the number of specific exposures that respon-

dents recalled were summed and then dichotomized at the 50th

percentile

Soc Psychiatry Psychiatr Epidemiol

123

Mental health literacy

There were no differences between intervention and con-

trol groups at any time in regards to recognition of

depression, intentions to seek help for depression, inten-

tions to see a VU counsellor, intentions to seek professional

help, beliefs about the helpfulness of interventions, desire

for social distance from the person described in the vignette

and personally held stigmatizing attitudes (Table 3).

Help seeking for mental health problems

There were no significant differences between the groups at

any time point in help-seeking actions taken by students

who said they had a similar problem to John/Jenny in the

past 12 months (Table 3).

Mental health first aid given to family or friends

There were no differences at any time point between

intervention and control groups in mental health first aid

given to family and friends (Table 3).

VU support of students with mental health problems

There were no differences at any time point between

intervention and control groups in regards to opinions

about VUs support of students with mental health prob-

lems (Table 3).

Psychological distress

There were no differences between intervention and con-

trol groups in levels of psychological distress at any time

point (Table 3).

Alcohol misuse and NHMRC guidelines

There were no differences at any time point in the likeli-

hood of having heard about NHMRC alcohol guidelines for

safe levels of drinking, or in awareness of the recommen-

dation to consume no more than two standard drinks a day,

or no more than four on any one occasion to avoid the risk

of harm (Table 3). There were no differences in the

intentions to seek professional help, help from a GP or help

from a mental health professional for alcohol problems.

Those in the intervention group had higher odds of

intending to go to a drug and alcohol service for alcohol

problems at Wave 2 only. This result was robust to sensi-

tivity analyses. Levels of risky drinking were not signifi-

cantly different between intervention and control groups at

any time point.

Discussion

The results of this study, a cluster randomized trial which

specifically aimed to improve mental health literacy in

tertiary students, showed few benefits. Although recall of

intervention elements was greater in the intervention group

at the end of the 2-year assessment period, there were very

few improvements in mental health literacy and no effects

on mental health. Students in the intervention group were

more likely to respond that they would go to a drug and

alcohol centre for alcohol problems.

The major limitations of the study relate to the small

sample size, relatively high attrition and the contamination

across campuses. Although every effort was made to avoid

this, it is possible that students travelling to other campuses

and sharing information with friends from other courses

and campuses may have led to information being dissem-

inated to control campuses. In addition, we did not succeed

in achieving a sample size with sufficient power to detect

effect sizes equalling or exceeding Cohen’s definition of a

‘small’ (h C 0.2) effect size [42]. Given our achieved

sample size and an intraclass correlation of 0.03, and

assuming a power of 0.80 and alpha of 0.05, the minimum

detectable effect size of the study was medium (h = 0.55).

It is possible that there may have been smaller effects

which were not detected. A further limitation relates to the

challenges of doing cluster randomized trials in tertiary

institutions. Typically the number of campuses in such

institutions is relatively small, thus limiting the number of

potential clusters. Furthermore, compared to secondary

schools, the size of the clusters is quite large, leading to

confounding of the intervention effect with the cluster

effect. In order to ensure a valid analysis, the minimum

number of clusters per arm is recommended to be four [47].

However, it is likely that this would be difficult for a single

tertiary institution and prohibitively expensive and logis-

tically difficult for such a trial to be run in eight separate

institutions.

These results provide some insight into the types of

intervention elements that may be useful in studies such as

these. When prompted to recall specific intervention ele-

ments, intervention and control group differences were

greatest for recall of posters and campus events. The

posters, which were mainly placed on the backs of toilet

doors, were also recalled relatively well and are also likely

to be a useful way of disseminating health information,

supporting findings seen in other studies [48]. Although

campus events are likely to be memorable, they only reach

those who happen to be there on the day, which is likely to

be a relatively small number of people. Scaling-up such

events may bring more benefit.

Some aspects of mental health literacy, which at Wave 1

were comparable with those in the general population,

Soc Psychiatry Psychiatr Epidemiol

123

increased in both intervention and control groups. For

example, accurate labelling of depression, which at 78.3 %

in the intervention group and 69.5 % in the control group is

very similar to that seen in the general population (73.7 %;

Table 3 Comparison of outcomes between intervention and control

groups

Outcomes Intervention Control OR/IRR

(95 % CI)a,b

Recognition of depression: % yes

Wave 1 (baseline) 78.2 70.1 1.47 (0.67–3.25)

Wave 2 vs. 1 83.7 72.0 1.56 (0.73–3.33)

Wave 3 vs. 1 91.0 81.4 1.53 (0.48–4.83)

Help-seeking intentions for depression: % that would seek help

Wave 1 (baseline) 84.5 80.9 1.36 (0.75–2.49)

Wave 2 vs. 1 89.2 86.3 1.42 (0.58–3.46)

Wave 3 vs. 1 90.1 90.0 0.86 (0.32–2.34)

Would see a VU counsellor: % yes

Wave 1 (baseline) 12.8 12.3 1.02 (0.47–2.24)

Wave 2 vs. 1 12.5 13.2 0.79 (0.32–1.95)

Wave 3 vs. 1 12.0 11.9 1.02 (0.39–2.70)

Would seek professional source of help: % yes

Wave 1 (baseline) 61.9 59.1 1.01 (0.52–1.98)

Wave 2 vs. 1 61.5 59.3 0.89 (0.44–1.79)

Wave 3 vs. 1 67.5 62.7 1.17 (0.54–2.52)

Interventions believed to be helpful: % that selected all 14 items as helpful

Wave 1 (baseline) 22.8 16.1 1.64 (0.93–2.89)

Wave 2 vs. 1 31.3 27.0 0.67 (0.33–1.35)

Wave 3 vs. 1 41.0 29.3 1.02 (0.44–2.39)

Social distance: % lower social distance

Wave 1 (baseline) 55.6 45.2 1.62 (0.85–3.11)

Wave 2 vs. 1 64.6 48.8 1.45 (0.80–2.63)

Wave 3 vs. 1 65.6 50.0 1.34 (0.63–2.83)

Personal stigmatizing attitudes: % lower stigma

Wave 1 (baseline) 40.0 35.1 1.03 (0.46–2.33)

Wave 2 vs. 1 44.8 36.8 1.20 (0.63–2.27)

Wave 3 vs. 1 50.7 41.5 1.13 (0.51–2.47)

Help seeking actions for respondents’ own mental health problem: M

(SD)b

Wave 1 (baseline) 4.59 (3.06) 4.61 (2.93) 1.04 (0.84–1.28)

Wave 2 vs. 1 4.26 (2.87) 4.14 (2.77) 0.90 (0.70–1.17)

Wave 3 vs. 1 5.40 (3.56) 4.00 (2.64) 1.23 (0.91–1.67)

Mental health first aid given to family or close friends: % yes

Wave 1 (baseline) 90.9 89.5 1.14 (0.50–2.62)

Wave 2 vs. 1 94.0 89.5 1.24 (0.30–5.13)

Wave 3 vs. 1 93.8 93.1 0.95 (0.24–3.74)

How well students supported by VU teaching staff: % Cwell

Wave 1 (baseline) 64.8 55.7 1.58 (0.84–2.98)

Wave 2 vs. 1 69.8 65.4 0.81 (0.42–1.57)

Wave 3 vs. 1 75.5 70.0 0.72 (0.33–1.58)

How well students supported by VU counselling staff: % Cwell

Wave 1 (baseline) 56.6 56.9 0.99 (0.63–1.56)

Wave 2 vs. 1 69.8 64.5 1.56 (0.87–2.79)

Wave 3 vs. 1 72.6 72.1 1.02 (0.50–2.10)

How well students supported by other students at VU: % Cwell

Wave 1 (baseline) 60.3 59.2 0.90 (0.59–1.37)

Wave 2 vs. 1 67.4 62.1 1.38 (0.77–2.46)

Wave 3 vs. 1 65.6 62.9 1.04 (0.54–1.98)

Table 3 continued

Outcomes Intervention Control OR/IRR

(95 % CI)a,b

Psychological distress (K6): % moderate to high distress

Wave 1 (baseline) 21.9 21.3 0.83 (0.44–1.55)

Wave 2 vs. 1 25.4 14.8 1.84 (0.92–3.65)

Wave 3 vs. 1 17.5 14.6 0.94 (0.42–2.13)

Heard about Australia’s guidelines for safe levels of drinking: % yes

Wave 1 (baseline) 66.7 63.6 0.88 (0.48–1.60)

Wave 2 vs. 1 76.4 76.3 0.89 (0.43–1.83)

Wave 3 vs. 1 86.3 82.9 1.66 (0.71–3.86)

Knowledge of guidelines on number of standard drinks per day: % B2

Wave 1 (baseline) 72.1 65.7 1.27 (0.80–2.01)

Wave 2 vs. 1 77.8 74.4 0.85 (0.45–1.61)

Wave 3 vs. 1 88.2 86.2 0.87 (0.34–2.24)

Knowledge of guidelines on maximum number of drinks on one occasion:

% B4

Wave 1 (baseline) 59.9 51.0 1.40 (0.95–2.05)

Wave 2 vs. 1 63.9 58.3 0.80 (0.46–1.41)

Wave 3 vs. 1 64.1 64.2 0.73 (0.39–1.35)

Would seek professional help for alcohol problems: % yes

Wave 1 (baseline) 68.5 67.4 0.96 (0.50–1.84)

Wave 2 vs. 1 71.6 65.8 1.30 (0.63–2.66)

Wave 3 vs. 1 72.8 67.9 1.26 (0.54–2.92)

Would seek help from GP for alcohol problems: % yes

Wave 1 (baseline) 29.4 28.4 0.88 (0.43–1.79)

Wave 2 vs. 1 40.2 34.2 1.30 (0.58–2.88)

Wave 3 vs. 1 42.4 37.6 1.19 (0.52–2.73)

Would go to a drug and alcohol service for alcohol problems: % yes

Wave 1 (baseline) 22.0 23.7 0.89 (0.52–1.55)

Wave 2 vs. 1 30.4 18.1 2.55 (1.21–5.35)*

Wave 3 vs. 1 23.4 26.6 1.02 (0.50–2.08)

Would go to a mental health professional for alcohol problems: % yes

Wave 1 (baseline) 22.7 20.9 1.02 (0.61–1.73)

Wave 2 vs. 1 19.6 20.8 0.69 (0.32–1.47)

Wave 3 vs. 1 19.0 17.4 0.99 (0.44–2.24)

AUDIT: % risky or hazardous levels of drinking

Wave 1 (baseline) 38.4 41.7 0.90 (0.43–1.87)

Wave 2 vs. 1 27.7 31.7 0.93 (0.37–2.32)

Wave 3 vs. 1 32.9 27.4 1.32 (0.51–3.42)

All models were adjusted for gender, country of birth (Australia vs. other)

and employment status (casual work and other vs. full time work)

* p \ 0.05, ** p \ 0.01, *** p \ 0.001a Odds ratio and 95 % confidence intervalsb Incidence rate ratios and 95 % confidence intervals. A count of items

selected by participants assessed against the list of intervention elements

considered helpful by 70 % of clinicians. Means and standard deviations

are also reported for the ease of interpretation

Soc Psychiatry Psychiatr Epidemiol

123

see [49]), increased to 91.0 % in the intervention group and

81.4 % in the control group. A similar pattern was seen for

the intention to seek help if the respondent had a problem

similar to that described in the vignette and for beliefs

about the helpfulness of interventions. It is possible that

assessment effects, that is, being in the monitoring sample

and participating in the telephone interview contributed to

these changes, thus biasing the effect estimates towards the

null hypothesis [50]. It is also possible that, even though

participants who changed from an intervention to a control

campus (and vice versa) were excluded from these analy-

ses, those in control campuses were exposed to the inter-

vention. The relatively high level of exposure in the control

group supports this possibility, particularly the high par-

ticipation rate in the MHFA course, which as a 2-day

training course, participants in the control condition are

less likely to over report; thus, the study is more likely to

be comparing two interventions of differing dose rather

than an intervention and a non-intervention control and

such findings point to the difficulty of assessing the added

value of the intervention over and above what else is

happening in a large institution such as VU.

These results may be compared with other interven-

tions that have aimed to improve attitudes toward or

knowledge of mental health problems in young adults [28,

51]. In a cluster randomized trial, Merritt et al. [28] used

postcards and posters in an effort to improve awareness

that depression can be treated effectively in over 3,000

undergraduates. Although the postcards were read by

69 % of students, there was no evidence of a difference

between intervention and control groups in reports that

depression could be treated effectively. However, there

were some improvements in the recognition of depressive

symptoms and beliefs about antidepressants. Livingston

et al. [51] evaluated the effect of an online social media

campaign to raise mental health awareness and reduce

stigma. However, awareness of the website increased,

attitudes towards mental health issues were similar

between exposed and unexposed respondents. Thus, like

the MindWise project, awareness of intervention elements

in these studies led only to minimal changes in attitudes

and knowledge.

Although education and awareness are likely to play a

role in improving mental health literacy, it is likely that to

achieve changes in stigmatising attitudes and psychological

distress, interventions would need to be more personalized

and intensive. This is even more likely to be the case for

interventions to prevent or intervene early for alcohol

misuse in tertiary students. There is relatively strong evi-

dence for brief motivational interventions and for per-

sonalised normative interventions delivered using

computers or in individual face-to-face sessions, while

evidence for informational approaches is lacking [25].

Conclusions

Few improvements in student mental health literacy were

seen as a result of the MindWise intervention. While

education and awareness may play a role in improving

mental health literacy, it is likely that, to reduce stigma,

improve help seeking and achieve changes in psychologi-

cal distress, interventions would need to be more person-

alized and intensive.

Acknowledgments Funding for the study was provided by be-

yondblue and by the NHMRC Australia Fellowship awarded to AFJ.

We would particularly like to acknowledge Fiona Blee for her work

on implementing the MindWise interventions. We would also like to

thank Prof John McCallum for championing the trial at Victoria

University, Dr Darko Hajzler for his assistance in implementing the

interventions and Prof Dan Lubman for his advice on the assessment

of alcohol misuse and the implementation of interventions to address

this.

Conflict of interest None declared.

References

1. Kessler RC, Angermeyer M, Anthony JC, De Graaf R, Demyt-

tenaere K, Gasquet I, De Girolamo G, Gluzman S, Gureje O,

Haro JM, Kawakami N, Karam A, Levinson D, Medina Mora

ME, Oakley Browne MA, Posada-Villa J, Stein DJ, Adley Tsang

CH, Aguilar-Gaxiola S, Alonso J, Lee S, Heeringa S, Pennell BE,

Berglund P, Gruber MJ, Petukhova M, Chatterji S, Ustun TB

(2007) Lifetime prevalence and age-of-onset distributions of

mental disorders in the World Health Organization’s World

Mental Health Survey Initiative. World Psychiatry 6(3):168–176

2. Slade T, Johnston A, Oakley Browne MA, Andrews G, Whiteford

H (2009) 2007 National Survey of Mental Health and Wellbeing:

methods and key findings. Aust N Z J Psychiatry 43(7):594–605

3. Cvetkovski S, Reavley NJ, Jorm AF (2012) The prevalence and

correlates of psychological distress in Australian tertiary students

compared to their community peers. Aust N Z J Psychiatry

45(6):457–467

4. Said D, Kypri K, Bowman J (2013) Risk factors for mental dis-

order among university students in Australia: findings from a

web-based cross-sectional survey. Soc Psychiatry Psychiatr Epi-

demiol 48(6):935–944. doi:10.1007/s00127-012-0574-x

5. Karam E, Kypri K, Salamoun M (2007) Alcohol use among

college students: an international perspective. Curr Opin Psy-

chiatry 20(3):213–221

6. Hingson RW, Zha W, Weitzman ER (2009) Magnitude of and

trends in alcohol-related mortality and morbidity among U.S.

college students ages 18–24, 1998–2005. J Stud Alcohol Drugs

Suppl 16:12–20

7. Hallett J, Howat PM, Maycock BR, McManus A, Kypri K,

Dhaliwal SS (2012) Undergraduate student drinking and related

harms at an Australian university: web-based survey of a large

random sample. BMC Public Health 12:37. doi:10.1186/1471-

2458-12-37

8. Stallman HM (2008) Prevalence of psychological distress in

university students—implications for service delivery. Aust Fam

Physician 37(8):673–677

9. Andrews B, Wilding JM (2004) The relation of depression and

anxiety to life-stress and achievement in students. Br J Psychol

95(Pt 4):509–521

Soc Psychiatry Psychiatr Epidemiol

123

10. Hysenbegasi A, Hass SL, Rowland CR (2005) The impact of

depression on the academic productivity of university students.

J Ment Health Policy Econ 8(3):145–151

11. Kessler RC, Foster CL, Saunders WB, Stang PE (1995) Social

consequences of psychiatric disorders, I: educational attainment.

Am J Psychiatry 152(7):1026–1032

12. Rickwood DJ, Deane FP, Wilson CJ (2007) When and how do

young people seek professional help for mental health problems?

Med J Aust 187(7 Suppl):S35–S39

13. Jorm AF, Korten AE, Jacomb PA, Christensen H, Rodgers B,

Pollitt P (1997) ‘‘Mental health literacy’’: a survey of the public’s

ability to recognise mental disorders and their beliefs about the

effectiveness of treatment. Med J Aust 166(4):182–186

14. Barker G, Olukoya A, Aggleton P (2005) Young people, social

support and help-seeking. Int J Adolesc Med Health 17(4):

315–335

15. Salzer MS, Wick LC, Rogers JA (2008) Familiarity with and use

of accommodations and supports among postsecondary students

with mental illnesses. Psychiatr Serv 59(4):370–375. doi:10.

1176/appi.ps.59.4.370

16. Slade T, Johnston A, Teesson M, Whiteford H, Burgess P, Pirkis

J, Saw S (2009) The mental health of Australians 2. Report on the

2007 National Survey of Mental Health and Wellbeing. Depart-

ment of Health and Ageing, Canberra

17. Dietrich S, Mergl R, Freudenberg P, Althaus D, Hegerl U (2009)

Impact of a campaign on the public’s attitudes towards depres-

sion. Health Educ Res 25(1):135–150. doi:10.1093/her/cyp050

18. Evans-Lacko S, London J, Little K, Henderson C, Thornicroft G

(2010) Evaluation of a brief anti-stigma campaign in Cambridge:

do short-term campaigns work? BMC Public Health 10:339.

doi:10.1186/1471-2458-10-339

19. Henderson C, Thornicroft G (2009) Stigma and discrimination in

mental illness: time to change. Lancet 373(9679):1928–1930.

doi:10.1016/S0140-6736(09)61046-1

20. Dumesnil H, Verger P (2009) Public awareness campaigns about

depression and suicide: a review. Psychiatr Serv 60(9):1203–1213.

doi:10.1176/appi.ps.60.9.1203

21. Cuijpers P, van Straten A, Smits N, Smit F (2006) Screening and

early psychological intervention for depression in schools : sys-

tematic review and meta-analysis. Eur Child Adolesc Psychiatry

15(5):300–307

22. Patton GC, Glover S, Bond L, Butler H, Godfrey C, Di Pietro G,

Bowes G (2000) The Gatehouse Project: a systematic approach to

mental health promotion in secondary schools. Aust N Z J Psy-

chiatry 34(4):586–593

23. Moreira MT, Smith LA, Foxcroft D (2009) Social norms inter-

ventions to reduce alcohol misuse in University or College stu-

dents. Cochrane Database Syst Rev 3:CD006748

24. Riper H, van Straten A, Keuken M, Smit F, Schippers G, Cuijpers

P (2009) Curbing problem drinking with personalized-feedback

interventions: a meta-analysis. Am J Prev Med 36(3):247–255

25. Reavley N, Jorm AF (2010) Prevention and early intervention to

improve mental health in higher education students: a review.

Early Interv Psychiatry 4(2):132–142. doi:10.1111/j.1751-7893.

2010.00167.x

26. Seligman ME, Schulman P, Tryon AM (2007) Group prevention

of depression and anxiety symptoms. Behav Res Ther 45(6):

1111–1126

27. Steinhardt M, Dolbier C (2008) Evaluation of a resilience inter-

vention to enhance coping strategies and protective factors and

decrease symptomatology. J Am Coll Health 56(4):445–453

28. Merritt RK, Price JR, Mollison J, Geddes JR (2007) A cluster

randomized controlled trial to assess the effectiveness of an

intervention to educate students about depression. Psychol Med

37(3):363–372

29. Reavley NJ, McCann TV, Cvetkovski S, Jorm AF (2014) A

multifaceted intervention to improve mental health literacy in

employees of a multi-campus university: a cluster randomised

trial. J Public Ment Health 13(1):25–39. doi:10.1108/JPMH-03-

2013-0010

30. Reavley NJ, McCann TV, Jorm AF (2012) Mental health literacy

in higher education students. Early Interv Psychiatry 6(1):45–52

31. Reavley NJ, Jorm AF, McCann TV, Lubman DI (2011) Alcohol

consumption in tertiary education students. BMC Public Health

11:545. doi:10.1186/1471-2458-11-545

32. Reavley NJ, McCann TV, Jorm AF (2012) Actions taken to deal

with mental health problems in Australian higher education stu-

dents. Early Interv Psychiatry 6(2):159–165. doi:10.1111/j.1751-

7893.2011.00294.x

33. NHMRC (2009) Australian guidelines to reduce the health risks

from drinking alcohol. National Health and Medical Research

Council, Canberra

34. Jorm AF, Korten AE, Jacomb PA, Rodgers B, Pollitt P, Chris-

tensen H, Henderson S (1997) Helpfulness of interventions for

mental disorders: beliefs of health professionals compared with

the general public. Br J Psychiatry 171:233–237

35. Link BG, Phelan JC, Bresnahan M, Stueve A, Pescosolido BA

(1999) Public conceptions of mental illness: labels, causes, dan-

gerousness, and social distance. Am J Public Health 89(9):

1328–1333

36. Wright A, Jorm AF, Mackinnon AJ (2011) Labeling of mental

disorders and stigma in young people. Soc Sci Med

73(4):498–506. doi:10.1016/j.socscimed.2011.06.015

37. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK,

Normand SL, Walters EE, Zaslavsky AM (2002) Short screening

scales to monitor population prevalences and trends in non-spe-

cific psychological distress. Psychol Med 32(6):959–976

38. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M

(1993) Development of the Alcohol Use Disorders Identification

Test (AUDIT): WHO collaborative project on early detection of

persons with harmful alcohol consumption—II. Addiction 88(6):

791–804

39. Hermansson U, Helander A, Brandt L, Huss A, Ronnberg S

(2010) Screening and brief intervention for risky alcohol con-

sumption in the workplace: results of a 1-year randomized con-

trolled study. Alcohol Alcohol 45(3):252–257. doi:10.1093/

alcalc/agq021

40. McPherson TL, Goplerud E, Derr D, Mickenberg J, Courte-

manche S (2010) Telephonic screening and brief intervention for

alcohol misuse among workers contacting the employee assis-

tance program: a feasibility study. Drug Alcohol Rev

29(6):641–646. doi:10.1111/j.1465-3362.2010.00249.x

41. Campbell MK, Thomson S, Ramsay CR, MacLennan GS,

Grimshaw JM (2004) Sample size calculator for cluster ran-

domized trials. Comput Biol Med 34(2):113–125

42. Cohen J (1992) A power primer. Psychol Bull 112:155–159

43. Kenward MG, Carpenter J (2007) Multiple imputation: current

perspectives. Stat Methods Med Res 16:199–218

44. White IR, Royston P, Wood AM (2011) Multiple imputation

using chained equations: issues and guidance for practice. Stat

Med 30:377–399

45. Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward

MG, Wood AM, Carpenter JR (2009) Multiple imputation for

missing data in epidemiological and clinical research: potential

and pitfalls. BMJ 338:b2393. doi:10.1136/bmj.b2393

46. Little R, Yau L (1996) Intent-to-treat analysis for longitudinal

studies with drop-outs. Biometrics 52(4):1324–1333

47. Campbell MK, Elbourne DR, Altman DG (2004) CONSORT

statement: extension to cluster randomised trials. BMJ

328(7441):702–708

Soc Psychiatry Psychiatr Epidemiol

123

48. Schmidt E, Kiss SM, Lokanc-Diluzio W (2009) Changing social

norms: a mass media campaign for youth ages 12–18. Can J

Public Health 100(1):41–45

49. Reavley NJ, Jorm AF (2011) Recognition of mental disorders and

beliefs about treatment and outcome: findings from an Australian

National Survey of Mental Health Literacy and Stigma. Aust N Z

J Psychiatry 45(11):947–956

50. McCambridge J, Kypri K (2011) Can simply answering research

questions change behaviour? Systematic review and meta

analyses of brief alcohol intervention trials. PLoS One

6(10):e23748. doi:10.1371/journal.pone.0023748

51. Livingston JD, Tugwell A, Korf-Uzan K, Cianfrone M, Coni-

glio C (2012) Evaluation of a campaign to improve awareness

and attitudes of young people towards mental health issues.

Soc Psychiatry Psychiatr Epidemiol. doi:10.1007/s00127-012-

0617-3

Soc Psychiatry Psychiatr Epidemiol

123