View
215
Download
2
Category
Preview:
Citation preview
ORIGINAL PAPER
A multifaceted intervention to improve mental health literacyin students of a multicampus university: a cluster randomised trial
Nicola J. Reavley • Terence V. McCann •
Stefan Cvetkovski • Anthony F. Jorm
Received: 2 September 2013 / Accepted: 14 April 2014
� Springer-Verlag Berlin Heidelberg 2014
Abstract
Purpose The aim of the current study was to assess
whether a multifaceted intervention could improve mental
health literacy, facilitate help seeking and reduce psycho-
logical distress and alcohol misuse in students of a multi-
campus university in Melbourne, Australia.
Methods In this cluster randomized trial, nine university
campuses were paired (some pairs included more than one
campus), with one of each pair randomly assigned to either
the intervention or control condition. The interventions
were designed to be whole-of-campus and to run over
2 academic years with their effectiveness assessed through
recruitment of a monitoring sample of students from each
campus. Interventions included emails, posters, campus
events, factsheets/booklets and mental health first aid
training courses. Participants had a 20-min telephone
interview at baseline and at the end of academic years 1
and 2. This assessed mental health literacy, help seeking,
psychological distress and alcohol use. The primary out-
comes were depression and anxiety levels and alcohol use
and pertained to the individual level.
Results There were no effects on psychological distress
and alcohol use. Recall of intervention elements was
greater in the intervention group at the end of year 2.
Students in the intervention group were more likely to say
they would go to a drug and alcohol centre for alcohol
problems at the end of 6 months.
Conclusion Although education and awareness may play
a role in improving mental health literacy, it is likely that,
to achieve changes in psychological distress, interventions
would need to be more personalized and intensive.
Keywords Mental health literacy � Students �Depression � Anxiety � Alcohol misuse
Background
Mental and substance use disorders have their first onset
before age 24 in 75 % of cases [1, 2]. At this age, many
young people are in tertiary education and while many
enjoy and effectively manage their responsibilities, ana-
lysis of data from Australian national surveys reveals that
tertiary students are at higher risk of moderate (but not
high) psychological distress as compared to non-students
[3], particularly those from lower socioeconomic back-
grounds [4]. In addition, excessive alcohol consumption
and substance misuse are very common among students [5,
6], with a recent study conducted in an Australian uni-
versity reporting that 90 % of students had consumed
alcohol in the last 12 months and 34 % met criteria for
hazardous drinking [7].
Very high levels of psychological distress in students have
been associated with a reduced capacity for work [8], and
mental health problems have been shown to affect both exam
performance and tertiary education dropout rates [9–11].
N. J. Reavley (&) � S. Cvetkovski � A. F. Jorm
Orygen Youth Health Research Centre, Centre for Youth Mental
Health and Melbourne School of Population and Global Health,
University of Melbourne, 207 Bouverie St, Parkville,
VIC 3010, Australia
e-mail: nreavley@unimelb.edu.au
T. V. McCann
College of Health and Biomedicine, Victoria University,
St Albans Campus, Building 4, Level 3,
PO Box 14428, Melbourne 8001, Australia
123
Soc Psychiatry Psychiatr Epidemiol
DOI 10.1007/s00127-014-0880-6
Such educational impacts may have lifelong consequences,
particularly if students do not complete their courses.
As with other young people, tertiary students need to
know how to take action to deal with mental health prob-
lems, whether that is professional help seeking or appro-
priate self-help behaviours. Taking appropriate action is
influenced by a number of interacting individual and
structural factors [12]. Individual factors, include stigma
and mental health literacy, which may be defined as
‘‘knowledge and beliefs about mental disorders that aid
their recognition, management or prevention’’ [13]. Struc-
tural factors include family, educational institution or
community support systems, and health system structures
[14]. In an effort to support students with mental health
problems, Australian tertiary education institutions typi-
cally offer a number of services, including counselling
services and disability liaison units. However, the great
majority of young people with depression and related dis-
orders either do not seek or delay seeking professional
help, and there is some evidence that they are also reluctant
to access formal disability support services in tertiary
education institutions [15, 16].
Evidence of the relatively low levels of mental health
literacy in members of the public has led to community
campaigns to improve this in a number of countries and
such campaigns have tended to focus on mental illness
broadly, or specifically on depression, and centre around
raising awareness of signs and symptoms and promotion of
professional help seeking [17–20]. Despite this, our
knowledge about the impact of these campaigns is rela-
tively limited and there is no evidence that such campaigns
help to reduce psychological distress, increase help seeking
or decrease suicidal behaviour [20]. It can be argued that
there is a need for a new generation of community cam-
paigns that aim for a more intensive level of intervention,
incorporating a broader range of mental health-related
messages and such campaigns could potentially produce a
cultural shift in knowledge and attitudes and may even lead
to improvements in mental health. Ideally, such campaigns
need to be directed at the stage in the lifespan where first
onset of disorders is most likely, namely youth and their
supporters, and should also be rigorously evaluated.
Because they often encompass several aspects of stu-
dents’ lives, including educational activities, health ser-
vices, residences, social networks and extracurricular
activities, educational institutions offer a unique setting for
prevention of and early intervention for mental health
problems. A number of mental health promotion inter-
ventions have been carried out in high schools, but tertiary
education institutions have typically received less attention
[21, 22]. An exception to this is in the area of prevention or
early intervention for alcohol misuse, for which evidence
of effectiveness is strongest for brief motivational
interventions and for personalised normative feedback
interventions delivered using computers or in individual
face-to-face sessions [23, 24].
There have been very few interventions aiming to pre-
vent or intervene early with depression or anxiety disorders
in tertiary students [25]. There is some evidence in support
of face-to-face cognitive behavioural/skills-based inter-
ventions [26, 27], and one social marketing intervention to
raise awareness of depression and treatments showed some
evidence of effectiveness [28].
The aim of the current study was to assess whether
MindWise, a multifaceted intervention, could improve
mental health literacy, facilitate help seeking and reduce
psychological distress and alcohol misuse in staff and
students of Victoria University (VU) in metropolitan
Melbourne, Australia. This paper describes the findings
from the intervention aimed at students. Findings from the
staff intervention have been described in an accompanying
paper [29].
Methods
Study design
VU is a multicampus institution and offers a broad mix of
courses, including a large number of vocational education
courses (including technical and trades), as well as higher
education courses. The MindWise study was a cluster
randomised trial with campuses as clusters and individual
students as participants. A cluster design was used because
it was not feasible to randomly assign individual students
studying on the same campus, as there may have been
contamination of information provided across groups
within the same campus. The trial was registered with the
Australian and New Zealand Clinical Trials Registry
(ACTRN12610001027000).
Participants
Clusters
Eligible clusters were all VU campuses. Senior VU staff
were informed of and agreed to the intervention prior to
implementation.
Individuals
Eligible participants were all students of VU who were
willing to participate in the study. To monitor the effec-
tiveness of the intervention, a sample of individuals was
recruited from each campus. Students were invited to
participate through emails and postcards handed out by
Soc Psychiatry Psychiatr Epidemiol
123
researchers on the nine largest VU campuses. Participants
were able to indicate agreement to participate via a website
or by returning the postcards to the researchers. Those who
did so were contacted by the survey company The Social
Research Centre and participated in a computer-assisted
telephone interview between March and May 2010 (the
baseline/Wave 1 interview). However, only those who
planned to study for at least 6 months were asked to con-
tinue the interview, as those attending for shorter periods
would have little or no exposure to the intervention. Fol-
low-up interviews were conducted in October–November
2010 (Wave 2) and October–November 2011 (Wave 3).
The results from the baseline survey have been reported
previously [30–32]. At each interview point, participants
received a free coffee voucher and also had a 1-in-12
chance of winning a voucher worth AU$50.
Interventions
The MindWise interventions were designed to be whole-of-
campus and to run over 2 academic years (January/Feb-
ruary to November/December in the majority of courses).
The interventions, which were designed with input from
student and staff focus groups and a student advisory
group, incorporated the following key messages: (1)
depression and related disorders are common in young
people; (2) there are recognizable signs of depression and
related disorders in young people; (3) early help seeking
leads to better outcomes; (4) there are several sources of
professional help available; (5) there are useful types of
self-help available; and (6) there are helpful first aid actions
that staff and peers can take. Other messages included
information on safe consumption of alcohol and NHMRC
guidelines for alcohol consumption (defined as no more
than two drinks per day to reduce lifetime risk of harm, and
no more than four drinks on any one occasion to reduce
risk of injury arising from that occasion) [33].
These messages were delivered in a variety of ways,
through: website/Facebook pages, Twitter activities; fact-
sheets/booklets (primarily provided by beyondblue: the
national depression initiative (http://www.beyondblue.org.
au); emails to student email addresses; booklets; campus
special events including stalls during orientation and at
exam times (at which students were also offered the
opportunity to complete online personalized feedback
questionnaires about their alcohol consumption); posters on
the back of campus toilet doors; student-designed projects
(including awareness-raising events by youth work stu-
dents); information on the electronic course information
and notes delivery system; and mental health first aid
(MHFA) training (http://www.mhfa.com.au) provided by
staff of the student counseling service. The intervention
was aimed at both staff and students.
Objectives
For students, the hypotheses tested were that the MindWise
intervention improves the following: psychological dis-
tress, alcohol misuse, recognition of mental health prob-
lems, help-seeking intentions and actions, beliefs about the
helpfulness of interventions, stigmatizing attitudes, per-
ceptions of how well the VU community supports staff and
students with mental health problems, knowledge of
NHMRC guidelines for the safe consumption of alcohol
and beliefs about help seeking for alcohol misuse. The
primary outcome measures for the trial were psychological
distress and alcohol use. All hypotheses pertained to the
individual rather than the cluster level.
Outcomes
Respondents were asked questions about their sociode-
mographic characteristics, their job role, their main campus
of study and whether they regularly visited a campus
other than the one nominated as their principal campus at
Wave 1.
The following outcomes were measured at the individ-
ual level:
Recognition of depression in a vignette
Interviews were based around a vignette of a 21-year-old
person (‘John’ or ‘Jenny,’ depending on the gender of the
respondent) with depression [13]. Respondents were asked
an open-ended question about what they thought was
wrong with the person. Responses which mentioned
‘‘depression’’ were scored as correct.
Help-seeking intentions
Respondents were asked if they would seek help if they had
a problem similar to that described in the vignette and an
open-ended question about what they would do to seek
help. Respondents were assigned to the category ‘Would
seek professional source of help’ if they nominated one or
more of VU student counsellor, GP, psychiatrist, psychol-
ogist, other counsellor or other professional.
Beliefs about the helpfulness of interventions
Students were given a list of 36 categories of people,
medicines or other interventions and asked whether each of
them is likely to be helpful, harmful or neither for the
person described in the vignette. Respondents were asses-
sed according to whether or not they nominated as helpful
all 14 of a list of items rated as helpful by 70 % of clini-
cians in a survey of health professionals [34]. These
Soc Psychiatry Psychiatr Epidemiol
123
included a GP, a student counsellor, a counsellor not
employed by VU, a psychologist, a psychiatrist, antide-
pressants, becoming more physically active, getting relax-
ation training, receiving counselling, receiving cognitive
behaviour therapy (CBT), going to a local mental health
service, cutting down on alcohol, cutting down on smoking
cigarettes and cutting down on marijuana.
Stigmatizing attitudes
Stigmatizing attitudes were assessed using a social distance
scale modified to be appropriate to the age group [35] and
the Depression Stigma Scale (personal subscale) [36]. For
analysis of these scales, scale scores were dichotomised at
the 50th percentile.
Help-seeking actions taken
Respondents were asked whether or not they had a problem
like the person in the vignette and if so, what actions they
took for the problem. Responses were assessed against a
list of 14 items rated as helpful by 70 % of clinicians in a
survey of health professionals (see above) [34].
Mental health first aid given to family or friends
Respondents were asked if anyone in their family or close
circle of friends had a problem similar to the person in the
vignette in the last 12 months. Those that did were asked if
they did anything to help. Possible responses were ‘Yes’ or
‘No’.
Psychological distress
Respondents were asked about their level of psychological
distress using the Kessler 6 (K6), a 6-item screening scale
(scored from 6 to 30) with strong psychometric properties
that is able to discriminate DSM-IV cases from non-cases
and has been used widely in general-purpose health surveys
[37]. For these analyses, scale scores were dichotomised
using a cut off C15 to assess moderate-to-high psycho-
logical distress.
Knowledge of NHMRC guidelines for safe alcohol
consumption
Participants were asked about their knowledge of the
NHMRC guidelines for alcohol consumption (defined as no
more than two drinks per day to reduce lifetime risk of
harm, and no more than four drinks on any one occasion to
reduce risk of injury arising from that occasion) [33].
Alcohol use
Those who had drunk alcohol at some point in their lives
were administered the Alcohol Use Disorders Identification
Test (AUDIT) [38]. The AUDIT is a 10-item questionnaire
designed to screen for early stage problem drinking [39,
40]. For these analyses, a cut off C8 was used to assess
risky or hazardous levels of drinking.
Help-seeking intentions and actions taken for alcohol
problems
Participants who drank alcohol regularly were asked about
help-seeking intentions for alcohol problems. For these
analyses, respondents were included in the category
‘Would seek professional help for alcohol problems’ if
they nominated one or more of: VU student counsellor, GP,
psychiatrist, psychologist, other counsellor, drug and
alcohol service or other professional. They were included
in the category ‘‘Would go to a mental health professional
for alcohol problem’’ if they nominated one or more of a
counsellor, psychiatrist or psychologist.
Perceptions of VU support for staff and students
with mental health problems
Participants were also asked about how well Victoria
University supports students with mental health problems.
Possible responses were ‘Very well’, ‘Well’, ‘Poorly’ and
‘Very poorly’.
Follow-up interview
At follow-up, this interview was repeated and students
were also asked a series of questions about their exposure
to specific MindWise intervention elements, including
website/Facebook pages, Twitter activities; factsheets/
booklets; emails; booklets; campus special events; partic-
ipation in competitions, posters on the back of campus
toilet doors; student-designed projects (including aware-
ness-raising events by youth work students); information
on the electronic course information and notes delivery
system; and MHFA training. Possible responses were ‘yes’,
‘no’ and ‘don’t know’.
Sample size estimation
Required sample size was estimated using software for
power analysis in cluster randomized trials [41]. In the
absence of relevant data for the primary outcomes of
changes in psychological distress and alcohol misuse,
likely effect sizes were based on those effect sizes which
Soc Psychiatry Psychiatr Epidemiol
123
equal or exceed Cohen’s definition of a ‘small’ (h C 0.20)
effect size [42]. To detect these effects in an unclustered
trial with 80 % power at the 0.05 significance level,
required n = 200. In the current study, six clusters were
used for the purpose of sample size and power calculations.
With a sample of 200 persons per cluster, power = 80 %
and alpha = 0.05, it was estimated that the study would
detect a standardized effect size of 0.32 if the intraclass
correlation is 0.01, and one of 0.56 if the intraclass cor-
relation is 0.05. Thus, the study would have had good
power to detect medium effect sizes (h = 0.50). Although
we aimed for a sample of 200 per cluster (1,200 overall) we
managed to recruit and randomise 767 students at Wave 1
in the period of time before the intervention started. The
reasons for the lower level of recruitment remain unclear,
although they may include refusal to answer phone calls
from the survey company after submitting contact details,
provision of incorrect contact details, and subsequent lack
of interest in the study.
Randomization
Nine campuses were paired on the basis of their size
(number of staff and students) and the type of courses
offered. Owing to their small sizes, in one case, two
campuses were combined to make one of pair and in
another case; three campuses were combined to make one
of a pair. Using the random integers option of Random.org,
two researchers (NR and AFJ) randomly assigned one of
each pair to either the intervention or control condition by
generating a 1 or a 2 for each pair (1 = intervention,
2 = control). Overall, there were three pairs, with six
campuses allocated to the control group and three cam-
puses allocated to the intervention group. There was no
allocation concealment, as allocation was based on clusters
rather than individuals, so that all students at a campus
received the same intervention.
Ethics
This study was approved by VU Human Research
Ethics Committee.
Statistical analysis
To assess whether there were any significant differences at
Wave 1, descriptive statistics were used to examine the
distributions of the sociodemographic characteristics
between the intervention and control groups and differ-
ences were tested using t tests, Pearson’s v2 tests and
Wald tests. To test the hypotheses of the study, group
(intervention vs. control) by measurement occasion (pre-,
post- and follow-up) interactions were examined using
logistic and Poisson mixed models for binary and count
outcomes, respectively. All results are reported as odds
ratios (OR) or incidence rate ratios (IRR), with 95 %
confidence intervals (95 % CIs). The strength of these
mixed models is that they can account for the correlation of
individual responses over time and the correlation of
individual responses within campuses.
Missing data were handled using uncongenial, multi-
variate imputation chained equations and were assumed to
be missing at random (MAR) [43, 44]. A series of logistic
regression models revealed that six covariates predicted
dropout from the monitoring sample: employment status,
recognition of depression, personal stigmatizing attitudes
towards others with mental health problems, awareness of
Australia’s guidelines on safe levels of drinking, the
maximum number of drinks recommended on one occasion
and the AUDIT. These covariates along with the covariates
of interest in the substantive analyses were included in the
conditional imputation models, which accounted for the
measurement level of the variables imputed (e.g., logistic
and Poisson regressions for binary and count outcomes,
respectively), with the exception of two outcomes that
related to a subsample of students, who had mental health
problems or who had family or friends with mental health
problems. It was necessary to exclude the maximum
number of drinks recommended on one occasion and the
AUDIT covariates from the multivariate imputation
chained equations for these models to converge. All mul-
tiple imputations were conducted in wide format to take
into consideration the dependence of measurement occa-
sions within respondents and then reshaped into long for-
mat for the mixed-model analyses. Twenty imputed
datasets were generated for each substantive analysis.
Although the MAR assumption is more plausible due to
the inclusion of predictors of dropout along with the other
covariates of interest in the substantive analyses in the
imputation models [45], we conducted sensitivity analyses
of significant results by introducing assumptions about
personal stigmatizing attitudes towards others with mental
health problems as the missing not at random (MNAR)
mechanism in the multiple imputation process [43, 46].
Given this is a binary variable, we introduced two MNAR
assumptions into the models in turn: (1) that all dropouts
had stigmatizing attitudes and (2) all dropouts had non-
stigmatizing attitudes towards others with mental health
problems. This is one of the more important predictors of
dropout and is arguably a plausible reason for why
respondents may have lost interest in remaining in the
study. All multiple imputations and substantive analyses
were performed using Stata IC/13.1 (StataCorp LP, TX
USA).
Soc Psychiatry Psychiatr Epidemiol
123
Results
Participant flow
Figure 1 shows the flow of participants at each stage of the
trial.
Numbers analysed
Wave 1 interviews were completed by 767 students from
the 9 randomized campuses: 426 from intervention cam-
puses and 341 from control campuses. The patterns of
completed interviews by students were as follows: for the
intervention group, 45.3 % completed all 3 waves of
interviews, 27.9 % completed only the Wave 1 interview,
22.3 % completed Waves 1 and 2 interviews, and 4.5 %
completed Waves 1 and 3 interviews; for the control group,
38.1 % completed all 3 waves of interviews, 35.2 %
completed only the Wave 1 interview, 23.8 % completed
Waves 1 and 2 interviews, and 2.9 % completed Waves 1
and 3 interviews. Of the total sample of 767 students, 499
completed Wave 2 interviews and 352 completed Wave 3
interviews, representing a 34.9 and 54.1 % loss to follow-
up, respectively. The proportions of students lost to follow-
up at Waves 2 and 3 were similar between the intervention
and control groups.
Table 1 shows the sociodemographic characteristics of
students from the Wave 1 interview. Comparison of
intervention and control groups revealed that the inter-
vention group had a significantly larger proportion of
female students than the control group (69.3 vs. 52.5 %,
v2ð1Þ = 22.53, p \ 0.001), a larger proportion of students
born in Australia than the control group (73.7 vs. 67.2 %,
v21ð Þ = 3.93, p = 0.047) and a smaller proportion of stu-
dents working full time than the control group (6.1 vs.
13.8 %, F(1,766) = 12.18, p = 0.003). To adjust for these
differences between the groups, gender (female vs. male),
country of birth (Australia vs. other) and employment
status (casual work and other vs. full time work) were
included in all models as covariates.
Fig. 1 Flow of participants at
each stage of the trial
Soc Psychiatry Psychiatr Epidemiol
123
Outcomes
Exposure to MindWise intervention elements
Comparison of exposure to MindWise intervention elements
is given in Table 2. When asked about their recall of specific
MindWise intervention elements, those in the intervention
group had higher odds of seeing information in course unit
guides or on web CT/blackboard, receiving emails and see-
ing factsheets or wallet cards at Wave 2 only. Odds of
recalling seeing posters, attending campus events were
greater at Wave 2 and Wave 3. The intervention group had
greater recall of receiving MindWise merchandise at Wave
3. Overall, those in the intervention group had higher odds of
recalling exposure to MindWise at both Waves 2 and 3.
Sensitivity analyses showed that these results were robust
to assumptions about personal stigmatizing attitudes towards
others with mental health problems as the MNAR mecha-
nisms, with the exception of the greater odds of recalling
receiving MindWise merchandise, which was a Wave 3 only
measure, where the results became marginal under both
assumptions: (1) that dropouts had stigmatizing attitudes
(OR = 1.88; 95 % CI 0.99–3.59; p = 0.055) and (2) non-
stigmatizing attitudes towards others with mental health
problems (OR = 1.83, 95 % CI 0.99–3.41; p = 0.055).
Table 1 Sociodemographic characteristics of respondents in the
intervention and control groups
Sociodemographiccharacteristics
Intervention(n = 426)
Control(n = 341)
Age: M (SD) 24.89 (8.02) 23.96 (8.89)
Gender
Male (%) 30.8 47.5
Female (%) 69.3 52.5
Australian citizens (%) 88.0 84.8
Country of birth Australia (%) 73.7 67.2
Education
Studying full time (%) 89.8 87.6
Bachelor degree (%) 54.2 55.4
Diploma (%) 21.3 24.9
Certificate I–IV (%) 14.6 13.5
Postgraduate (%) 6.3 1.5
Other (%) 3.8 4.7
Employment
Full time (%) 6.1 13.8
Part time (%) 22.7 16.7
Casual (%) 34.7 27.9
Contract (%) 1.2 0.3
Not working (%) 35.2 41.4
Looking for work (%) 60.4 67.4
Table 2 Comparison of exposures to MindWise between interven-
tion and control groups
Exposures Intervention Control OR (95 % CI)a
Recalls seeing MindWise information in course unit guides or on web
CT/blackboard: % yes
Wave 2 33.3 21.8 1.70 (1.07–2.70)*
Wave 3 33.5 25.7 1.47 (0.83–2.59)
Recalls seeing posters around campus: % yes
Wave 2 51.7 33.2 2.10 (1.38–3.18)**
Wave 3 76.9 62.1 1.94 (1.13–3.31)*
Recalls attending MindWise related events on campus: % yes
Wave 2 33.0 18.5 2.02 (1.31–3.11)**
Wave 3 26.4 12.1 2.86 (1.29–6.32)*
Recalls receiving emails: % yes
Wave 2 25.0 14.7 1.98 (1.14–3.43)*
Wave 3 29.3 30.7 0.92 (0.53–1.58)
Recalls receiving books or booklets about mental health: % yes
Wave 2 11.5 8.5 1.25 (0.67–2.33)
Wave 3 22.2 12.9 1.75 (0.86–3.54)
Recalls completing an online questionnaire about alcohol or mental
health: % yes
Wave 2 5.2 3.3 1.29 (0.55–3.02)
Wave 3 16.0 17.9 0.96 (0.48–1.91)
Recalls viewing a website related to MindWise: % yes
Wave 2 10.4 7.6 1.40 (0.71–2.76)
Wave 3 19.8 12.9 1.65 (0.83–3.29)
Recalls seeing MindWise factsheets or wallet cards: % yes
Wave 2 30.9 17.5 1.97 (1.26–3.07)**
Wave 3 46.2 35.0 1.56 (0.98–2.48)
Measures of exposure only at Wave 3
Recalls doing a
mental health first
aid course: % yes
6.6 4.3 2.10 (0.69–6.40)
Participated in a
competition run by
MindWise: % yes
19.8 13.6 1.74 (0.92–3.27)
Recalls receiving
MindWise
merchandise:
% yes
23.1 14.3 1.97 (1.03–3.75)*
Summary of specific MindWise exposures: % high exposureb
Wave 2 53.8 33.7 2.15 (1.31–3.51)**
Wave 3 54.7 35.7 2.21 (1.39–3.49)**
All models were adjusted for gender, country of birth (Australia vs.
other) and employment status (casual work and other vs. full time
work)
* p \ 0.05; ** p \ 0.01; *** p \ 0.001a Odds ratio and 95 % confidence intervalb For these analyses, the number of specific exposures that respon-
dents recalled were summed and then dichotomized at the 50th
percentile
Soc Psychiatry Psychiatr Epidemiol
123
Mental health literacy
There were no differences between intervention and con-
trol groups at any time in regards to recognition of
depression, intentions to seek help for depression, inten-
tions to see a VU counsellor, intentions to seek professional
help, beliefs about the helpfulness of interventions, desire
for social distance from the person described in the vignette
and personally held stigmatizing attitudes (Table 3).
Help seeking for mental health problems
There were no significant differences between the groups at
any time point in help-seeking actions taken by students
who said they had a similar problem to John/Jenny in the
past 12 months (Table 3).
Mental health first aid given to family or friends
There were no differences at any time point between
intervention and control groups in mental health first aid
given to family and friends (Table 3).
VU support of students with mental health problems
There were no differences at any time point between
intervention and control groups in regards to opinions
about VUs support of students with mental health prob-
lems (Table 3).
Psychological distress
There were no differences between intervention and con-
trol groups in levels of psychological distress at any time
point (Table 3).
Alcohol misuse and NHMRC guidelines
There were no differences at any time point in the likeli-
hood of having heard about NHMRC alcohol guidelines for
safe levels of drinking, or in awareness of the recommen-
dation to consume no more than two standard drinks a day,
or no more than four on any one occasion to avoid the risk
of harm (Table 3). There were no differences in the
intentions to seek professional help, help from a GP or help
from a mental health professional for alcohol problems.
Those in the intervention group had higher odds of
intending to go to a drug and alcohol service for alcohol
problems at Wave 2 only. This result was robust to sensi-
tivity analyses. Levels of risky drinking were not signifi-
cantly different between intervention and control groups at
any time point.
Discussion
The results of this study, a cluster randomized trial which
specifically aimed to improve mental health literacy in
tertiary students, showed few benefits. Although recall of
intervention elements was greater in the intervention group
at the end of the 2-year assessment period, there were very
few improvements in mental health literacy and no effects
on mental health. Students in the intervention group were
more likely to respond that they would go to a drug and
alcohol centre for alcohol problems.
The major limitations of the study relate to the small
sample size, relatively high attrition and the contamination
across campuses. Although every effort was made to avoid
this, it is possible that students travelling to other campuses
and sharing information with friends from other courses
and campuses may have led to information being dissem-
inated to control campuses. In addition, we did not succeed
in achieving a sample size with sufficient power to detect
effect sizes equalling or exceeding Cohen’s definition of a
‘small’ (h C 0.2) effect size [42]. Given our achieved
sample size and an intraclass correlation of 0.03, and
assuming a power of 0.80 and alpha of 0.05, the minimum
detectable effect size of the study was medium (h = 0.55).
It is possible that there may have been smaller effects
which were not detected. A further limitation relates to the
challenges of doing cluster randomized trials in tertiary
institutions. Typically the number of campuses in such
institutions is relatively small, thus limiting the number of
potential clusters. Furthermore, compared to secondary
schools, the size of the clusters is quite large, leading to
confounding of the intervention effect with the cluster
effect. In order to ensure a valid analysis, the minimum
number of clusters per arm is recommended to be four [47].
However, it is likely that this would be difficult for a single
tertiary institution and prohibitively expensive and logis-
tically difficult for such a trial to be run in eight separate
institutions.
These results provide some insight into the types of
intervention elements that may be useful in studies such as
these. When prompted to recall specific intervention ele-
ments, intervention and control group differences were
greatest for recall of posters and campus events. The
posters, which were mainly placed on the backs of toilet
doors, were also recalled relatively well and are also likely
to be a useful way of disseminating health information,
supporting findings seen in other studies [48]. Although
campus events are likely to be memorable, they only reach
those who happen to be there on the day, which is likely to
be a relatively small number of people. Scaling-up such
events may bring more benefit.
Some aspects of mental health literacy, which at Wave 1
were comparable with those in the general population,
Soc Psychiatry Psychiatr Epidemiol
123
increased in both intervention and control groups. For
example, accurate labelling of depression, which at 78.3 %
in the intervention group and 69.5 % in the control group is
very similar to that seen in the general population (73.7 %;
Table 3 Comparison of outcomes between intervention and control
groups
Outcomes Intervention Control OR/IRR
(95 % CI)a,b
Recognition of depression: % yes
Wave 1 (baseline) 78.2 70.1 1.47 (0.67–3.25)
Wave 2 vs. 1 83.7 72.0 1.56 (0.73–3.33)
Wave 3 vs. 1 91.0 81.4 1.53 (0.48–4.83)
Help-seeking intentions for depression: % that would seek help
Wave 1 (baseline) 84.5 80.9 1.36 (0.75–2.49)
Wave 2 vs. 1 89.2 86.3 1.42 (0.58–3.46)
Wave 3 vs. 1 90.1 90.0 0.86 (0.32–2.34)
Would see a VU counsellor: % yes
Wave 1 (baseline) 12.8 12.3 1.02 (0.47–2.24)
Wave 2 vs. 1 12.5 13.2 0.79 (0.32–1.95)
Wave 3 vs. 1 12.0 11.9 1.02 (0.39–2.70)
Would seek professional source of help: % yes
Wave 1 (baseline) 61.9 59.1 1.01 (0.52–1.98)
Wave 2 vs. 1 61.5 59.3 0.89 (0.44–1.79)
Wave 3 vs. 1 67.5 62.7 1.17 (0.54–2.52)
Interventions believed to be helpful: % that selected all 14 items as helpful
Wave 1 (baseline) 22.8 16.1 1.64 (0.93–2.89)
Wave 2 vs. 1 31.3 27.0 0.67 (0.33–1.35)
Wave 3 vs. 1 41.0 29.3 1.02 (0.44–2.39)
Social distance: % lower social distance
Wave 1 (baseline) 55.6 45.2 1.62 (0.85–3.11)
Wave 2 vs. 1 64.6 48.8 1.45 (0.80–2.63)
Wave 3 vs. 1 65.6 50.0 1.34 (0.63–2.83)
Personal stigmatizing attitudes: % lower stigma
Wave 1 (baseline) 40.0 35.1 1.03 (0.46–2.33)
Wave 2 vs. 1 44.8 36.8 1.20 (0.63–2.27)
Wave 3 vs. 1 50.7 41.5 1.13 (0.51–2.47)
Help seeking actions for respondents’ own mental health problem: M
(SD)b
Wave 1 (baseline) 4.59 (3.06) 4.61 (2.93) 1.04 (0.84–1.28)
Wave 2 vs. 1 4.26 (2.87) 4.14 (2.77) 0.90 (0.70–1.17)
Wave 3 vs. 1 5.40 (3.56) 4.00 (2.64) 1.23 (0.91–1.67)
Mental health first aid given to family or close friends: % yes
Wave 1 (baseline) 90.9 89.5 1.14 (0.50–2.62)
Wave 2 vs. 1 94.0 89.5 1.24 (0.30–5.13)
Wave 3 vs. 1 93.8 93.1 0.95 (0.24–3.74)
How well students supported by VU teaching staff: % Cwell
Wave 1 (baseline) 64.8 55.7 1.58 (0.84–2.98)
Wave 2 vs. 1 69.8 65.4 0.81 (0.42–1.57)
Wave 3 vs. 1 75.5 70.0 0.72 (0.33–1.58)
How well students supported by VU counselling staff: % Cwell
Wave 1 (baseline) 56.6 56.9 0.99 (0.63–1.56)
Wave 2 vs. 1 69.8 64.5 1.56 (0.87–2.79)
Wave 3 vs. 1 72.6 72.1 1.02 (0.50–2.10)
How well students supported by other students at VU: % Cwell
Wave 1 (baseline) 60.3 59.2 0.90 (0.59–1.37)
Wave 2 vs. 1 67.4 62.1 1.38 (0.77–2.46)
Wave 3 vs. 1 65.6 62.9 1.04 (0.54–1.98)
Table 3 continued
Outcomes Intervention Control OR/IRR
(95 % CI)a,b
Psychological distress (K6): % moderate to high distress
Wave 1 (baseline) 21.9 21.3 0.83 (0.44–1.55)
Wave 2 vs. 1 25.4 14.8 1.84 (0.92–3.65)
Wave 3 vs. 1 17.5 14.6 0.94 (0.42–2.13)
Heard about Australia’s guidelines for safe levels of drinking: % yes
Wave 1 (baseline) 66.7 63.6 0.88 (0.48–1.60)
Wave 2 vs. 1 76.4 76.3 0.89 (0.43–1.83)
Wave 3 vs. 1 86.3 82.9 1.66 (0.71–3.86)
Knowledge of guidelines on number of standard drinks per day: % B2
Wave 1 (baseline) 72.1 65.7 1.27 (0.80–2.01)
Wave 2 vs. 1 77.8 74.4 0.85 (0.45–1.61)
Wave 3 vs. 1 88.2 86.2 0.87 (0.34–2.24)
Knowledge of guidelines on maximum number of drinks on one occasion:
% B4
Wave 1 (baseline) 59.9 51.0 1.40 (0.95–2.05)
Wave 2 vs. 1 63.9 58.3 0.80 (0.46–1.41)
Wave 3 vs. 1 64.1 64.2 0.73 (0.39–1.35)
Would seek professional help for alcohol problems: % yes
Wave 1 (baseline) 68.5 67.4 0.96 (0.50–1.84)
Wave 2 vs. 1 71.6 65.8 1.30 (0.63–2.66)
Wave 3 vs. 1 72.8 67.9 1.26 (0.54–2.92)
Would seek help from GP for alcohol problems: % yes
Wave 1 (baseline) 29.4 28.4 0.88 (0.43–1.79)
Wave 2 vs. 1 40.2 34.2 1.30 (0.58–2.88)
Wave 3 vs. 1 42.4 37.6 1.19 (0.52–2.73)
Would go to a drug and alcohol service for alcohol problems: % yes
Wave 1 (baseline) 22.0 23.7 0.89 (0.52–1.55)
Wave 2 vs. 1 30.4 18.1 2.55 (1.21–5.35)*
Wave 3 vs. 1 23.4 26.6 1.02 (0.50–2.08)
Would go to a mental health professional for alcohol problems: % yes
Wave 1 (baseline) 22.7 20.9 1.02 (0.61–1.73)
Wave 2 vs. 1 19.6 20.8 0.69 (0.32–1.47)
Wave 3 vs. 1 19.0 17.4 0.99 (0.44–2.24)
AUDIT: % risky or hazardous levels of drinking
Wave 1 (baseline) 38.4 41.7 0.90 (0.43–1.87)
Wave 2 vs. 1 27.7 31.7 0.93 (0.37–2.32)
Wave 3 vs. 1 32.9 27.4 1.32 (0.51–3.42)
All models were adjusted for gender, country of birth (Australia vs. other)
and employment status (casual work and other vs. full time work)
* p \ 0.05, ** p \ 0.01, *** p \ 0.001a Odds ratio and 95 % confidence intervalsb Incidence rate ratios and 95 % confidence intervals. A count of items
selected by participants assessed against the list of intervention elements
considered helpful by 70 % of clinicians. Means and standard deviations
are also reported for the ease of interpretation
Soc Psychiatry Psychiatr Epidemiol
123
see [49]), increased to 91.0 % in the intervention group and
81.4 % in the control group. A similar pattern was seen for
the intention to seek help if the respondent had a problem
similar to that described in the vignette and for beliefs
about the helpfulness of interventions. It is possible that
assessment effects, that is, being in the monitoring sample
and participating in the telephone interview contributed to
these changes, thus biasing the effect estimates towards the
null hypothesis [50]. It is also possible that, even though
participants who changed from an intervention to a control
campus (and vice versa) were excluded from these analy-
ses, those in control campuses were exposed to the inter-
vention. The relatively high level of exposure in the control
group supports this possibility, particularly the high par-
ticipation rate in the MHFA course, which as a 2-day
training course, participants in the control condition are
less likely to over report; thus, the study is more likely to
be comparing two interventions of differing dose rather
than an intervention and a non-intervention control and
such findings point to the difficulty of assessing the added
value of the intervention over and above what else is
happening in a large institution such as VU.
These results may be compared with other interven-
tions that have aimed to improve attitudes toward or
knowledge of mental health problems in young adults [28,
51]. In a cluster randomized trial, Merritt et al. [28] used
postcards and posters in an effort to improve awareness
that depression can be treated effectively in over 3,000
undergraduates. Although the postcards were read by
69 % of students, there was no evidence of a difference
between intervention and control groups in reports that
depression could be treated effectively. However, there
were some improvements in the recognition of depressive
symptoms and beliefs about antidepressants. Livingston
et al. [51] evaluated the effect of an online social media
campaign to raise mental health awareness and reduce
stigma. However, awareness of the website increased,
attitudes towards mental health issues were similar
between exposed and unexposed respondents. Thus, like
the MindWise project, awareness of intervention elements
in these studies led only to minimal changes in attitudes
and knowledge.
Although education and awareness are likely to play a
role in improving mental health literacy, it is likely that to
achieve changes in stigmatising attitudes and psychological
distress, interventions would need to be more personalized
and intensive. This is even more likely to be the case for
interventions to prevent or intervene early for alcohol
misuse in tertiary students. There is relatively strong evi-
dence for brief motivational interventions and for per-
sonalised normative interventions delivered using
computers or in individual face-to-face sessions, while
evidence for informational approaches is lacking [25].
Conclusions
Few improvements in student mental health literacy were
seen as a result of the MindWise intervention. While
education and awareness may play a role in improving
mental health literacy, it is likely that, to reduce stigma,
improve help seeking and achieve changes in psychologi-
cal distress, interventions would need to be more person-
alized and intensive.
Acknowledgments Funding for the study was provided by be-
yondblue and by the NHMRC Australia Fellowship awarded to AFJ.
We would particularly like to acknowledge Fiona Blee for her work
on implementing the MindWise interventions. We would also like to
thank Prof John McCallum for championing the trial at Victoria
University, Dr Darko Hajzler for his assistance in implementing the
interventions and Prof Dan Lubman for his advice on the assessment
of alcohol misuse and the implementation of interventions to address
this.
Conflict of interest None declared.
References
1. Kessler RC, Angermeyer M, Anthony JC, De Graaf R, Demyt-
tenaere K, Gasquet I, De Girolamo G, Gluzman S, Gureje O,
Haro JM, Kawakami N, Karam A, Levinson D, Medina Mora
ME, Oakley Browne MA, Posada-Villa J, Stein DJ, Adley Tsang
CH, Aguilar-Gaxiola S, Alonso J, Lee S, Heeringa S, Pennell BE,
Berglund P, Gruber MJ, Petukhova M, Chatterji S, Ustun TB
(2007) Lifetime prevalence and age-of-onset distributions of
mental disorders in the World Health Organization’s World
Mental Health Survey Initiative. World Psychiatry 6(3):168–176
2. Slade T, Johnston A, Oakley Browne MA, Andrews G, Whiteford
H (2009) 2007 National Survey of Mental Health and Wellbeing:
methods and key findings. Aust N Z J Psychiatry 43(7):594–605
3. Cvetkovski S, Reavley NJ, Jorm AF (2012) The prevalence and
correlates of psychological distress in Australian tertiary students
compared to their community peers. Aust N Z J Psychiatry
45(6):457–467
4. Said D, Kypri K, Bowman J (2013) Risk factors for mental dis-
order among university students in Australia: findings from a
web-based cross-sectional survey. Soc Psychiatry Psychiatr Epi-
demiol 48(6):935–944. doi:10.1007/s00127-012-0574-x
5. Karam E, Kypri K, Salamoun M (2007) Alcohol use among
college students: an international perspective. Curr Opin Psy-
chiatry 20(3):213–221
6. Hingson RW, Zha W, Weitzman ER (2009) Magnitude of and
trends in alcohol-related mortality and morbidity among U.S.
college students ages 18–24, 1998–2005. J Stud Alcohol Drugs
Suppl 16:12–20
7. Hallett J, Howat PM, Maycock BR, McManus A, Kypri K,
Dhaliwal SS (2012) Undergraduate student drinking and related
harms at an Australian university: web-based survey of a large
random sample. BMC Public Health 12:37. doi:10.1186/1471-
2458-12-37
8. Stallman HM (2008) Prevalence of psychological distress in
university students—implications for service delivery. Aust Fam
Physician 37(8):673–677
9. Andrews B, Wilding JM (2004) The relation of depression and
anxiety to life-stress and achievement in students. Br J Psychol
95(Pt 4):509–521
Soc Psychiatry Psychiatr Epidemiol
123
10. Hysenbegasi A, Hass SL, Rowland CR (2005) The impact of
depression on the academic productivity of university students.
J Ment Health Policy Econ 8(3):145–151
11. Kessler RC, Foster CL, Saunders WB, Stang PE (1995) Social
consequences of psychiatric disorders, I: educational attainment.
Am J Psychiatry 152(7):1026–1032
12. Rickwood DJ, Deane FP, Wilson CJ (2007) When and how do
young people seek professional help for mental health problems?
Med J Aust 187(7 Suppl):S35–S39
13. Jorm AF, Korten AE, Jacomb PA, Christensen H, Rodgers B,
Pollitt P (1997) ‘‘Mental health literacy’’: a survey of the public’s
ability to recognise mental disorders and their beliefs about the
effectiveness of treatment. Med J Aust 166(4):182–186
14. Barker G, Olukoya A, Aggleton P (2005) Young people, social
support and help-seeking. Int J Adolesc Med Health 17(4):
315–335
15. Salzer MS, Wick LC, Rogers JA (2008) Familiarity with and use
of accommodations and supports among postsecondary students
with mental illnesses. Psychiatr Serv 59(4):370–375. doi:10.
1176/appi.ps.59.4.370
16. Slade T, Johnston A, Teesson M, Whiteford H, Burgess P, Pirkis
J, Saw S (2009) The mental health of Australians 2. Report on the
2007 National Survey of Mental Health and Wellbeing. Depart-
ment of Health and Ageing, Canberra
17. Dietrich S, Mergl R, Freudenberg P, Althaus D, Hegerl U (2009)
Impact of a campaign on the public’s attitudes towards depres-
sion. Health Educ Res 25(1):135–150. doi:10.1093/her/cyp050
18. Evans-Lacko S, London J, Little K, Henderson C, Thornicroft G
(2010) Evaluation of a brief anti-stigma campaign in Cambridge:
do short-term campaigns work? BMC Public Health 10:339.
doi:10.1186/1471-2458-10-339
19. Henderson C, Thornicroft G (2009) Stigma and discrimination in
mental illness: time to change. Lancet 373(9679):1928–1930.
doi:10.1016/S0140-6736(09)61046-1
20. Dumesnil H, Verger P (2009) Public awareness campaigns about
depression and suicide: a review. Psychiatr Serv 60(9):1203–1213.
doi:10.1176/appi.ps.60.9.1203
21. Cuijpers P, van Straten A, Smits N, Smit F (2006) Screening and
early psychological intervention for depression in schools : sys-
tematic review and meta-analysis. Eur Child Adolesc Psychiatry
15(5):300–307
22. Patton GC, Glover S, Bond L, Butler H, Godfrey C, Di Pietro G,
Bowes G (2000) The Gatehouse Project: a systematic approach to
mental health promotion in secondary schools. Aust N Z J Psy-
chiatry 34(4):586–593
23. Moreira MT, Smith LA, Foxcroft D (2009) Social norms inter-
ventions to reduce alcohol misuse in University or College stu-
dents. Cochrane Database Syst Rev 3:CD006748
24. Riper H, van Straten A, Keuken M, Smit F, Schippers G, Cuijpers
P (2009) Curbing problem drinking with personalized-feedback
interventions: a meta-analysis. Am J Prev Med 36(3):247–255
25. Reavley N, Jorm AF (2010) Prevention and early intervention to
improve mental health in higher education students: a review.
Early Interv Psychiatry 4(2):132–142. doi:10.1111/j.1751-7893.
2010.00167.x
26. Seligman ME, Schulman P, Tryon AM (2007) Group prevention
of depression and anxiety symptoms. Behav Res Ther 45(6):
1111–1126
27. Steinhardt M, Dolbier C (2008) Evaluation of a resilience inter-
vention to enhance coping strategies and protective factors and
decrease symptomatology. J Am Coll Health 56(4):445–453
28. Merritt RK, Price JR, Mollison J, Geddes JR (2007) A cluster
randomized controlled trial to assess the effectiveness of an
intervention to educate students about depression. Psychol Med
37(3):363–372
29. Reavley NJ, McCann TV, Cvetkovski S, Jorm AF (2014) A
multifaceted intervention to improve mental health literacy in
employees of a multi-campus university: a cluster randomised
trial. J Public Ment Health 13(1):25–39. doi:10.1108/JPMH-03-
2013-0010
30. Reavley NJ, McCann TV, Jorm AF (2012) Mental health literacy
in higher education students. Early Interv Psychiatry 6(1):45–52
31. Reavley NJ, Jorm AF, McCann TV, Lubman DI (2011) Alcohol
consumption in tertiary education students. BMC Public Health
11:545. doi:10.1186/1471-2458-11-545
32. Reavley NJ, McCann TV, Jorm AF (2012) Actions taken to deal
with mental health problems in Australian higher education stu-
dents. Early Interv Psychiatry 6(2):159–165. doi:10.1111/j.1751-
7893.2011.00294.x
33. NHMRC (2009) Australian guidelines to reduce the health risks
from drinking alcohol. National Health and Medical Research
Council, Canberra
34. Jorm AF, Korten AE, Jacomb PA, Rodgers B, Pollitt P, Chris-
tensen H, Henderson S (1997) Helpfulness of interventions for
mental disorders: beliefs of health professionals compared with
the general public. Br J Psychiatry 171:233–237
35. Link BG, Phelan JC, Bresnahan M, Stueve A, Pescosolido BA
(1999) Public conceptions of mental illness: labels, causes, dan-
gerousness, and social distance. Am J Public Health 89(9):
1328–1333
36. Wright A, Jorm AF, Mackinnon AJ (2011) Labeling of mental
disorders and stigma in young people. Soc Sci Med
73(4):498–506. doi:10.1016/j.socscimed.2011.06.015
37. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK,
Normand SL, Walters EE, Zaslavsky AM (2002) Short screening
scales to monitor population prevalences and trends in non-spe-
cific psychological distress. Psychol Med 32(6):959–976
38. Saunders JB, Aasland OG, Babor TF, de la Fuente JR, Grant M
(1993) Development of the Alcohol Use Disorders Identification
Test (AUDIT): WHO collaborative project on early detection of
persons with harmful alcohol consumption—II. Addiction 88(6):
791–804
39. Hermansson U, Helander A, Brandt L, Huss A, Ronnberg S
(2010) Screening and brief intervention for risky alcohol con-
sumption in the workplace: results of a 1-year randomized con-
trolled study. Alcohol Alcohol 45(3):252–257. doi:10.1093/
alcalc/agq021
40. McPherson TL, Goplerud E, Derr D, Mickenberg J, Courte-
manche S (2010) Telephonic screening and brief intervention for
alcohol misuse among workers contacting the employee assis-
tance program: a feasibility study. Drug Alcohol Rev
29(6):641–646. doi:10.1111/j.1465-3362.2010.00249.x
41. Campbell MK, Thomson S, Ramsay CR, MacLennan GS,
Grimshaw JM (2004) Sample size calculator for cluster ran-
domized trials. Comput Biol Med 34(2):113–125
42. Cohen J (1992) A power primer. Psychol Bull 112:155–159
43. Kenward MG, Carpenter J (2007) Multiple imputation: current
perspectives. Stat Methods Med Res 16:199–218
44. White IR, Royston P, Wood AM (2011) Multiple imputation
using chained equations: issues and guidance for practice. Stat
Med 30:377–399
45. Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward
MG, Wood AM, Carpenter JR (2009) Multiple imputation for
missing data in epidemiological and clinical research: potential
and pitfalls. BMJ 338:b2393. doi:10.1136/bmj.b2393
46. Little R, Yau L (1996) Intent-to-treat analysis for longitudinal
studies with drop-outs. Biometrics 52(4):1324–1333
47. Campbell MK, Elbourne DR, Altman DG (2004) CONSORT
statement: extension to cluster randomised trials. BMJ
328(7441):702–708
Soc Psychiatry Psychiatr Epidemiol
123
48. Schmidt E, Kiss SM, Lokanc-Diluzio W (2009) Changing social
norms: a mass media campaign for youth ages 12–18. Can J
Public Health 100(1):41–45
49. Reavley NJ, Jorm AF (2011) Recognition of mental disorders and
beliefs about treatment and outcome: findings from an Australian
National Survey of Mental Health Literacy and Stigma. Aust N Z
J Psychiatry 45(11):947–956
50. McCambridge J, Kypri K (2011) Can simply answering research
questions change behaviour? Systematic review and meta
analyses of brief alcohol intervention trials. PLoS One
6(10):e23748. doi:10.1371/journal.pone.0023748
51. Livingston JD, Tugwell A, Korf-Uzan K, Cianfrone M, Coni-
glio C (2012) Evaluation of a campaign to improve awareness
and attitudes of young people towards mental health issues.
Soc Psychiatry Psychiatr Epidemiol. doi:10.1007/s00127-012-
0617-3
Soc Psychiatry Psychiatr Epidemiol
123
Recommended