Upload
others
View
12
Download
0
Embed Size (px)
Citation preview
RESEARCH ARTICLE Open Access
Effectiveness of online and mobiletelephone applications (‘apps’) for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysisKatrina Witt1*, Matthew J. Spittal2, Gregory Carter3, Jane Pirkis2, Sarah Hetrick4, Dianne Currier2, Jo Robinson4
and Allison Milner5
Abstract
Background: Online and mobile telephone applications (‘apps’) have the potential to improve the scalability ofeffective interventions for suicidal ideation and self-harm. The aim of this review was therefore to investigate theeffectiveness of digital interventions for the self-management of suicidal ideation or self-harm.
Methods: Seven databases (Applied Science & Technology; CENTRAL; CRESP; Embase; Global Health; PsycARTICLES;PsycINFO; Medline) were searched to 31 March, 2017. Studies that examined the effectiveness of digital interventionsfor suicidal ideation and/or self-harm, or which reported outcome data for suicidal ideation and/or self-harm, within arandomised controlled trial (RCT), pseudo-RCT, or observational pre-test/post-test design were included in the review.
Results: Fourteen non-overlapping studies were included, reporting data from a total of 3,356 participants. Overall,digital interventions were associated with reductions for suicidal ideation scores at post-intervention. There was noevidence of a treatment effect for self-harm or attempted suicide.
Conclusions: Most studies were biased in relation to at least one aspect of study design, and particularly the domainsof participant, clinical personnel, and outcome assessor blinding. Performance and detection bias therefore cannot beruled out. Digital interventions for suicidal ideation and self-harm may be more effective than waitlist control. It isunclear whether these reductions would be clinically meaningful at present. Further evidence, particularly with regardsto the potential mechanisms of action of these interventions, as well as safety, is required before these interventionscould recommended.
Keywords: Digital, Application, Mobile telephone, Suicide ideation, Self-harm, Suicide
BackgroundSelf-harm, which includes intentional self-injury orself-poisoning irrespective of type of motivation and/or degree of suicidal intent [1], and attempted sui-cide, which refers to any intentionally self-inflictedself-injurious and/or self-poisoning behaviour withclear suicidal intent [2, 3], are associated with
suicidal ideation [4], non-fatal repetition of self-harm,and completed suicide [5]. Although in the UnitedStates of America (USA) distinction is frequentlymade between non-suicidal self-injury and suicidalbehavior [6], outside of the USA these terms are yetto receive widespread acceptance [7]. Instead, this re-view includes all forms of self-inflected self-injury orself-poisoning, irrespective of type of motivation ordegree of suicidal intent, which we refer to collect-ively as ‘self-harm’ [8].
* Correspondence: [email protected] Health, Turning Point, Eastern Health Clinical School, MonashUniversity, 54-62 Gertrude Street, Fitzroy, Victoria 3065, AustraliaFull list of author information is available at the end of the article
© The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Witt et al. BMC Psychiatry (2017) 17:297 DOI 10.1186/s12888-017-1458-0
Effective face-to-face treatments for self-harm and sui-cidal ideation are available [9–11]. Many effective psycho-therapeutic options for the treatment of suicidal ideationand self-harm are resource intensive and require specialistclinician training, however. Resource limitations in somelow-to-middle income countries therefore limits accessto these professionals. Negative associations have, forexample, been found between per capita availability ofmental health services and suicide rates in a number ofcountries [12–14].Even in countries where access to psychotherapy for
suicidal ideation and self-harm are available, less thanone-half of those who self-harm receive treatment [15].There are a number of barriers to treatment for thosewho experience self-harm or suicidal ideation, including:beliefs that treatment is not warranted and/or is likely tobe ineffective, stigma, shame, negative prior experienceswith mental health care providers, and financial difficulties[15]. Young people in particular also cite a preference forself-management as a major obstacle to help-seeking forself-harm from clinical services [16].Given that an estimated 85% of the global population
is covered by a commercially-available wireless signal, andfurther, over five billion persons have a mobile phone sub-scription [17], digital interventions, including both onlineprograms and mobile telephone applications (‘apps’) (col-lectively referred to here as ‘digital interventions’), havebeen proposed as one mechanism by which the scalabilityof effective treatments for self-harm and suicidal ideationmay be improved [18]. Such interventions may also helpto overcome some of the attitudinal and structural bar-riers which prevent those who engage in self-harm fromaccessing clinical services [19], and may therefore repre-sent a valuable addition to a stepped care treatment modelin which access is improved through the provision oflower intensity ‘self-help’ interventions in addition to highintensity psychotherapeutic treatments for those whosesymptoms do not resolve.To date, the effectiveness of these digital interventions
has not been routinely evaluated. This is problematicgiven that the widespread implementation of these inter-ventions without appropriate evaluation of their useabil-ity and effectiveness could lead to the development andpromulgation of ineffective, or even harmful, interven-tions [20]. We therefore present a systematic review andmeta-analysis of the characteristics and effectiveness ofdigital interventions, including both online resourcesand mobile telephone apps for suicidal ideation and self-harm.
MethodsThe reporting of this systematic review and meta-analysis conforms to the Preferred Reporting Items for
Systematic Reviews and Meta-Analyses (PRISMA)guidelines [21].
Search strategy and selection criteriaWe searched for literature indexed in seven electronicdatabases covering a wide range of disciplines, including:computing and information technology (Applied Science& Technology) clinical trials (Cochrane Central Registerof Controlled Trials [CENTRAL]), medicine (Embase;Medline), psychology (PsycARTICLES; PsycINFO), andpublic health (Global Health) as well as a database thatspecifically indexes literature on interventions for theprevention of suicide (Centre for Research Excellence inSuicide Prevention [CRESP]). Clinical trial registries werealso searched using these same keywords to identify on-going studies. All databases were searched from their re-spective start dates until 31 March, 2017.A two stage process was used to locate relevant studies.
At the first stage, keywords inclusive of digital interven-tions and platforms were combined. Next, using standardBoolean operators, these were combined with keywordsrelated to suicidal ideation and self-harm. There were norestrictions either on publication language or status. Forfurther information on this electronic search strategy,please see the Appendix.Ancestry searches were also conducted by manually
screening the reference lists of included studies and pre-vious reviews [22–26]. Where information on eitherstudy design, methodology, or results was either unclearor missing from the published study, we sought clarifica-tion from corresponding authors.
Selection criteriaStudies were eligible for inclusion if: (1) the effectivenessof a standalone digital intervention (i.e., any online ormobile telephone app) was evaluated; (2) the interventionwas designed for the self-management of suicidal ideationor self-harm; (3) data on the effectiveness of the interven-tion with respect to any suicidal outcome (i.e., suicidalideation, repetition of self-harm, attempted suicide, orcompleted suicide) were reported; and (4) either a ran-domized, pseudo-randomized, or observational pre-test/post-test design was used.Studies were excluded if: (1) the program was not a
standalone digital intervention. Thus multimodal inter-ventions, in which the digital intervention was intendedto serve either as a complement or adjunct to traditionalface-to-face psychosocial therapy or required significantinput or involvement from face-to-face treating clini-cians, were not eligible for inclusion in this review. Briefcontact-based programs delivered via text messaging ore-mail services were also excluded as no form of activepsychosocial therapy is typically provided in the contextof these programs. Studies were also excluded if: (2) the
Witt et al. BMC Psychiatry (2017) 17:297 Page 2 of 18
intervention targeted gatekeepers (i.e., carers, otherhealth care professionals, or bystanders who may comeinto contact with suicidal persons); or (3) no data on sui-cidal ideation, self-harm, attempted and/or completedsuicide were reported. Descriptions of programs withoutdata on effectiveness were also excluded.Two authors (KW and AM) independently screened
studies for inclusion. Firstly, titles of all retrieved studieswere screened. Next, only studies meeting inclusioncriteria following a full text screen were retained. Anydisagreements regarding study eligibility were resolvedfollowing consensus discussions with the broader group ofreview authors. Once again, corresponding authorswere contacted to request further information on programdesign, study design, data analysis, or methodology asrequired.
Data analysisMethodological details were extracted from included stud-ies using a standardized extraction form by two authors(KW and AM) working independently of one another.Disagreements were resolved through consensus discus-sions. Methodological details included: research design,treatment setting, and outcome ascertainment. We alsoassessed whether those evaluating the intervention wereindependent from those who developed the intervention.Data on the primary outcome, suicidal ideation, were
also extracted by KW and AM working independently ofone another. To make maximal use of the available data,information was extracted irrespective of whether theseoutcomes were measured continuously, for example asscores on a psychometrically validated measure of sui-cidal ideation or as numbers of repeat episodes of self-harm, or categorically, as the proportion of participantsreporting thoughts of suicide or number of self-harmevents. Care was taken, however, to ensure the itemsused to determine these outcomes were comparablebetween studies.Secondary outcomes included: episodes of self-harm,
attempted suicide, and completed suicide measured ac-cording to self-report and/or hospital or medical records.Where a study reported outcomes at multiple time points,for example at six and 12 months’ follow-up, only data forthe longest follow-up period were extracted, in line withrecommendations [27].
Statistical analysisData on the primary outcome (i.e., suicidal ideation)were synthesized using one of two approaches. Wherethese were reported categorically (e.g., as a difference inproportions reporting thoughts of suicide in the inter-vention group as compared to the control group), oddsratios (OR) and their accompanying 95% confidence in-tervals were calculated.
Where outcomes were reported as scores on a con-tinuous scale, such as total scores on the Beck Scale ofSuicidal Ideation (BSSI; [28]), the mean difference (MD)or the standard mean difference (SMD), with an accom-panying 95% confidence interval, was calculated as ap-propriate. Specifically, where the same scale was used toinvestigate the outcome of interest in all studies includedin a meta-analysis, the MD was used. The SMD, on theother hand, was used where outcomes were measuredusing a variety of different scales as recommended [27].Pooled ORs, MDs, or SMDs were calculated by using
the DerSimonian and Laird random effects model [29],as implemented by RevMan for Windows, version 3.5.The impact of between-study heterogeneity was quanti-fied by the I2 statistic [30]. We interpreted an I2 statisticof ≥75% as indicating substantial levels of between-studyheterogeneity. Where we found evidence of this, weundertook sensitivity analyses to explore potential causesof this between-study heterogeneity, as outlined below.
Sensitivity analysesGiven recent findings suggesting that psychological inter-ventions that directly address suicidal ideation and self-harm are more effective in reducing attempted andcompleted suicide than interventions which indirectlytarget the symptoms associated with suicidality (e.g., anx-iety, depression, hopelessness) [31], we conducted sensi-tivity analyses to investigate whether digital interventionsdeveloped specifically for the self-management of suicidalideation or self-harm would be more effective in pre-venting suicidal ideation and repetition of self-harmthan interventions developed for the self-managementof depression symptomatology more generally.
Sub-group analysesThe inclusion of RCTs and non-randomized observationalstudies within a single meta-analysis is becoming increas-ingly common as reliance on RCT evidence alone can leadto knowledge translation bias [32]. RCTs, for example,typically recruit highly selected patient populations withlower risk profiles as compared to “real world” popula-tions [33]. The inclusion of results from pre-test/post-testobservational studies together with those from RCTs,however, can also lead to over-estimation of the treatmenteffect size [34]. To balance these two concerns, all studieswere eligible for inclusion in our review, irrespective ofstudy design; however, we did not pool data from RCTstogether with data from observational studies. Instead,we calculated separate sub-group analyses by study de-sign to investigate what impact, if any, study design hadon the magnitude of the effect size observed for theseinterventions.
Witt et al. BMC Psychiatry (2017) 17:297 Page 3 of 18
Risk of biasRisk of bias was assessed using the Cochrane Collabor-ation tool for randomised and pseudo-randomised con-trolled trials [35] and, for controlled before/after designedstudies, the Risk of Bias In Non-Randomized Studies ofInterventions (ROBINS–I; [36]). The Cochrane Collabor-ation tool assesses bias in seven domains including: ad-equacy of the random sequence generation, allocationconcealment, blinding of participants, clinical personnel,and outcome assessors, as well as incomplete data, select-ive outcome reporting, and other bias. The ROBINS–I as-sesses bias in eight domains including: confounding,participant selection, classification of the intervention,departures from the intended intervention, missing data,measurement of outcomes, selection of the reported re-sults, and overall bias.
Role of the funding sourceThe funder had no role in study design, data collection,data analysis, data interpretation, or writing of the re-port. KW and AM had full access to all the data in thisstudy, and all authors had final responsibility for the de-cision to submit for publication.
ResultsThe electronic search strategy outlined in the Appendixinitially identified a total of 9033 potentially relevant re-cords. Four additional records were retrieved following
ancestry searching. After excluding duplicates, this figurereduced to 6382 records. Of these, 6063 records werescreened out after a review of their titles, whilst 305 re-cords were omitted following full text screening. A totalof 14 independent, non-overlapping studies were there-fore included in the review, reporting data on a total of3356 participants (Fig. 1; Table 1).
Study characteristicsThe majority of these studies had been conducted eitherin Australia (four studies: [37–40]) or the USA (four stud-ies: [41–44]). Two studies were conducted in Germany[45, 46], one in the Netherlands [47], one in Sweden [48],and one in Switzerland [49]. One further study recruitedparticipants through online forums and therefore in-cluded participants from a number of different coun-tries, including countries in North America, Europe, andAustralasia [50].Most studies evaluated the effectiveness of online pro-
grams [37–39, 41–49]. Only two evaluated the effective-ness of mobile telephone apps [40, 50]. Most programswere developed for the self-management of depression[37, 38, 42, 43, 45, 46, 48, 49]. However, as these studiesassessed the effectiveness of these programs on to at leastone suicide-related primary or secondary outcome (i.e.,suicidal ideation, self-harm, attempted and/or completedsuicide) they were nonetheless eligible for inclusion in thisreview. Only five programs were developed specifically for
Fig. 1 PRISMA flow diagram of included and excluded studies
Witt et al. BMC Psychiatry (2017) 17:297 Page 4 of 18
Table
1Stud
ycharacteristics,metho
dologicald
etails,and
riskof
bias
assessmen
tforthe14
stud
iesinclud
edin
thisreview
Stud
yand
Reference
Num
ber
Stud
yDesign
Cou
ntry
NParticipantSource
Interven
tionCon
ditio
nCon
trol
Con
ditio
nMeasures
Risk
ofBias
INV
CTL
Christensen
,2013
[37]
RCT
Australia
3835
Callersto
Lifeline,a
charitable24
hteleph
one
coun
selling
serviceproviding
crisissupp
ortandsuicide
preven
tion.Person
swho
werehigh
lydistressed
and/
orsuicidalat
intake
were
exclud
ed.
MoodG
YM:Internetself-guide
dpsycho
educationprog
ram
coup
ledwith
self-gu
ided
iCBT
consis-tingof
sixmod
ules
delivered
once
aweekover
asixweekpe
riod.
Participantswere
allocatedto
await-listcontrol
cond
ition
.
Suicidalideation:scores
onfour
suicidalideatio
nitemsfro
mthe28-item
Gen
eralHealth
Questionn
aire
(GHQ–28).
Neither
participantsno
rclinicalpe
rson
nelw
ere
blindto
treatm
ent
allocatio
n.Prop
ortio
ncompletingtreatm
ent
notstated
.
Franklin,
2016
[50]
Recruitedfro
mon
line
forumsforself-injury.
Participantshadto
self-
repo
rttw
oor
moreep
isod
esof
self-cuttingin
pastmon
thto
beeligiblefor
participation.
Therapeutic
Evaluative
Condition
ing:App
-based
gamified
cond
ition
ing
prog
ram
inwhich
self-harm
relatedstim
uliw
ere
sequ
entially
pairedwith
adversestim
uli.Participants
wereen
couraged
tointeract
with
theprog
ram
asoftenas
necessaryover
aon
emon
thpe
riod.
Participantswere
allocatedto
anattentionalcon
trol
cond
ition
.
Suicidalideation:scores
ontheSelf-Injurio
usThou
ghtsand
Behaviou
rsInterview,
indicatin
gnu
mbe
rof
days
inwhich
the
participanthadhad
thou
ghtsof
suicide.
Self-ha
rm:scoreson
the
Self-Injurio
usThou
ghts
andBehaviou
rsInterview,
indicatin
gthenu
mbe
rof
days
inwhich
the
participanthaden
gage
din
self-cuttingor
NSSI.
Neither
participantsno
rclinicalpe
rson
nelw
ere
blindto
treatm
ent
allocatio
n.Stud
y1
RCT
International55
59
Stud
y2
RCT
International62
69
Stud
y3
RCT
International75
84
Guille,2015
[41]
RCT
USA
100
99Med
icalinternscommen
cing
theirreside
ncyyear
recruited
from
oneof
twoho
spitals.
MoodG
YM:Internetself-guide
dpsycho
educationprog
ram
coup
ledwith
self-gu
ided
iCBT
consis-tingof
four
mod
ules
delivered
once
aweekover
afour
weekpe
riod.
Participantswere
allocatedto
anattentionalcon
trol
cond
ition
.
Suicidalideation:
prop
ortio
nof
participants
self-repo
rtingany
thou
ghtsof
suicide
accordingto
scores
onitem
nine
ofthePatient
Health
Questionn
aire–9
(PHQ–9).
Neither
participantsno
rclinicalpe
rson
nelw
ere
blindto
treatm
ent
allocatio
n.One
-half
(49%
)did
notcomplete
treatm
ent.
Hed
man,
2014
[48]
Pre-
test/
Post-
test
Swed
en1203
Recruitedfro
maun
iversity
hospitalp
sychia-tric
clinic.
iCBT:con
sistingof
10mod
ules
ofbe
haviou
ralactivationand
cogn
itive
therapyde
livered
over
a12
weekpe
riod.
N/A.
Suicidalideation:scores
onitem
9of
theMon
tgom
ery
ÅsbergDep
ressionRatin
gScale(M
ADRS).
App
ropriate
adjustmen
twas
madeforpo
tential
time-varyingconfou
nding.
Neither
participantsno
rclinicalpe
rson
nelw
ere
blindto
treatm
ent
allocatio
n.One
-quarter
(25%
)did
notcomplete
treatm
ent.
Witt et al. BMC Psychiatry (2017) 17:297 Page 5 of 18
Table
1Stud
ycharacteristics,metho
dologicald
etails,and
riskof
bias
assessmen
tforthe14
stud
iesinclud
edin
thisreview
(Con
tinued)
Hill,2016
[44]
RCT
USA
4040
Recruitedfollowingan
advertisingcampaign.All
participantshadto
self-
repo
rtsign
ificant
levelsof
perceivedbu
rden
somen
ess
(i.e.,score
of17
orgreater
ontheInter-pe
rson
alNeeds
Question-naire
Perceived
Burden
-som
enesssub-scale).
LEAP:Interne
tself-gu
ided
prog
ram
inform
edby
the
InterpersonalThe
oryof
Suicide(prin
-cipallype
rceived
burden
-som
eness)consistin
gof
twomod
ules
delivered
over
atw
oweekpe
riod.
Participantswere
allocatedto
psycho
education.
Suicidalideation:scores
ontheBeck
ScaleforSuicidal
Ideatio
n.
Participantswereno
tblindto
treatm
ent
allocatio
n.Con
trol
cond
ition
didno
taccoun
tforinterven
tion
timeor
engage
men
t(i.e.,
didno
tsatisfyattentional
controlreq
uiremen
ts).
Overon
e-third
(39.0%
)didno
tcompletetreatm
ent.
Kordy,2016
[46]
RCT
Germany
7780
Recruitedfro
mon
eof
six
psychiatric
departmen
ts.A
llpatientshadto
meet
diagno
sticcriteria
formajor
depression
accord-in
gto
the
Structured
Clinical
Interven
tionforDSM
-IVand
toself-repo
rtahistoryof
atleastthreepriorep
isod
esof
depression
tobe
eligiblefor
participation.
SUMMIT:Interne
tself-gu
ided
emotioncoping
prog
ram.The
numbe
rof
treatm
entmod
ules
was
notstated
,but
participants
wereen
couraged
toaccess
the
interven
tionas
oftenas
necessary
over
a12
mon
thpe
riod.
Participantswere
allocatedto
usual
psychiatric
care,
includ
ingph
armaco-
therapyand/or
psycho
therapyas
necessary.
Self-ha
rman
d/or
attempted
suicide:re-hospitalisation
followingan
episod
eof
self-injury
and/or
asuicide
attempt.
Neither
participantsno
rclinicalpe
rson
nelw
ere
blindto
treatm
entallocatio
n.Prop
ortio
ncompleting
treatm
entno
tstated
.
Mew
ton,
2015
[38]
Pre-
test/
Post-
test
Australia
484
Patientsdiagno
sedwith
depression
who
were
prescribed
acourse
ofiCBT
bytheirprim
arycare
provider.
iCBT:C
onsistingof
sixmod
ules
delivered
over
sixweekpe
riod.
N/A.
Suicidalideation:scores
onitem
9of
thePatient
Health
Questionn
aire–9
(PHQ–9).
App
ropriate
adjustmen
twas
madeforpresen
ceof
missing
data
andfor
potentialtim
e-varying
confou
nding.
Neither
participantsno
rclinical
person
nelw
ereblindto
treatm
entallocatio
n.Alm
oston
e-half(43.2%
)didno
tcompletetreatm
ent.
Mortiz,2012
[45]
Pseudo
-RC
TGermany
105
105
Recruitedfro
mweb
sites
providingsupp
ortto
person
swith
depression
.
Deprexis:self-gu
ided
iCBT
prog
ram
consistin
gof
10mod
ules
delivered
over
aneigh
tweekpe
riod.
Participantswere
allocatedto
await-listcontrol
cond
ition
.
Suicidalideation:scores
ontheGerman
lang
uage
versionof
theSuicide
BehaviorsQuestionn
aire-
Revised(SBQ
–R).
Neither
participantsno
rclinicalpe
rson
nelw
ereblind
totreatm
entallocatio
n.Participantswerealternately
allocatedto
theinterven
tion
andcontrolg
roup
s.One
-fifth
(21.9%
)didno
tcomplete
treatm
ent.
Robinson
,2016
[39]
Pre-
test/
Post-
test
Australia
21Second
aryscho
olstud
ents
presen
tingto
scho
ol-based
coun
sellorswith
suicidal
ideatio
nin
thepastmon
th.
Refra
me-IT:Interne
tself-gu
ided
iCBT
prog
ram
consistin
gof
eigh
tmod
ules
delivered
over
aneigh
tweekpe
riod.
N/A.
Suicidalideation:scores
ontheSuicidalIdeatio
nQuestionn
aire-Jun
ior
(SIQ-JR)
ortheAdu
ltSuicidalIdeatio
nQuestionn
aire
(ASIQ)
(asapprop
riate
tothe
participant’s
age).
Analysesbasedon
treatm
ent
completerson
ly(N
=21).
Neither
participantsno
rclinicalpe
rson
nelw
ereblind
totreatm
entallocatio
n.
Witt et al. BMC Psychiatry (2017) 17:297 Page 6 of 18
Table
1Stud
ycharacteristics,metho
dologicald
etails,and
riskof
bias
assessmen
tforthe14
stud
iesinclud
edin
thisreview
(Con
tinued)
Tigh
e,2017
[40]
RCT
Australia
3130
Recruitedthou
gha
commun
ity-based
suicide
preven
tionorganisatio
n.All
participantshadto
iden
tify
asAbo
riginaland/orTorres
StraitIsland
er.A
llparticipants
also
hadto
repo
rtclinically
sig-nificantlevelsof
depres-
sion
(i.e.,scoresof
10or
greateron
thePatient
Health
Questionn
aire-9),psycho
-logicald
istress(i.e.,scoresof
25or
greateron
theKessler
Psycho
logicalD
istressScale),
andhadto
self-repo
rtsuicidalthou
ghtsin
thepast
twoweeks.
iBobbly:App
-based
self-gu
ided
CBT
prog
ram
consistin
gof
three
mod
ules
completed
over
asix
weekpe
riod.
Partici-p
antswere
also
encouraged
tocomplete
self-assessmen
tson
functio
ning
,suicidalthinking
,mind-fulness
exercises,self-soothing
activities,
andcultu
raleng
agem
ent
activities.
Participantswere
allocatedto
await-listcontrol
cond
ition
.
Suicidalideation:scores
ontheDep
ressive
Symptom
sInventory-
Suicidality
Subscale
(DSI-SS).
Due
tochange
sin
inclusion
criteria
afterthe
commen
cemen
tof
thetrial,
one-qu
arter(26.2%
)of
the
includ
edparticipantsdid
notmeetthecriterio
nfor
frequ
ency
ofsuicidal
thou
ghts.Participants,
clinicalpe
rson
nel,and
outcom
eassessorswere
notblindto
treatm
ent
allocatio
n.Dataon
usage
was
availablefor65.6%
oftheinclud
edparticipants.
Ofthese,15.0%
didno
tcompletetreatm
ent.
VanSpijker,
2014
[47]
RCT
Nethe
rland
s116
120
Recruitedfollowingan
advertisingcampaign.All
participantshadto
repo
rtmild
tomod
eratesuicidal
ideatio
nto
beeligiblefor
participation.Participants
self-repo
rtingsevere
depression
wereexclud
ed.
iCBT:C
onsistingof
sixmod
ules
delivered
over
asixweekpe
riod
with
adde
dactivities
from
DBT,
PST,andmindfulne
ss-based
cogn
itive
therapy.
Participantswere
allocatedto
anattentionaland
onlinebibliotherapy.
Suicidalideation:scores
ontheBeck
Suicide
Ideatio
nScale.
Neither
participantsno
rclinicalpe
rson
nelw
ereblind
totreatm
entallocatio
n.Aroun
don
e-half(44.0%
)did
notcom-plete
atleastthree
treatm
entmod
ules.
Van
Voorhe
es,
2009
[42]
Pre-
test/
Post-
testa
USA
83Ado
lescen
tsreceiving
treatm
entforde
pression
atprim
arycare
facilities
registered
with
oneof
five
major
Health
Care
Organisations
(HMOs).
CATCH-IT:Internetself-gu
ided
prog
ram
consistin
gof
14mod
-ules
ofCBT,IPT,and
comm-unity
resiliencyactivities
delivered
over
a12
weekpe
riod.
N/A.
Suicidalideation:scores
onitem
9of
thePatient
Health
Questionn
aire–9
(PHQ–9).
Alm
ostall(96.4%)
participantscompleted
treatm
ent.Asdata
were
analysed
aspre-test/post-test
compare-son
sin
thisreview
,no
approp
riate
adjustmen
tcouldbe
madeforpo
tential
time-varyingconfou
nding.
Neither
participantsno
rclin-
icalpe
rson
nelw
ereblindto
treatm
entallocatio
n.
Wagne
r,2014
[49]
RCT
Switzerland
3230
Recruitedfro
mun
iversity
web
sites,ne
wspapers,po
ster
adverts,andfro
mreferralsto
ade
pression
self-he
lpgrou
p.Participantshadto
scoreat
least12
ontheBeck
Dep
res-
sion
Inventory-IIat
the
screen
inginterview.
iCBT:C
onsistingof
seven
mod
ules
delivered
over
aneigh
tweekpe
riod.
Participantswere
allocatedto
eigh
tweeks
ofmanualised
face-to-face
CBT
delivered
bya
registered
psycho
logist.
Suicidalideation:scores
ontheBeck
Suicide
Ideatio
nScale.
Neither
participantsno
rclinicalpe
rson
nelw
ereblind
totreatm
entallocatio
n.Despite
rand
-omisation,a
sign
ificantlyhigh
ernu
mbe
rof
females
wereallocatedto
theinterven
tion.One
-fifth
(22.0%
)didno
tcomplete
treatm
ent.
Witt et al. BMC Psychiatry (2017) 17:297 Page 7 of 18
Table
1Stud
ycharacteristics,metho
dologicald
etails,and
riskof
bias
assessmen
tforthe14
stud
iesinclud
edin
thisreview
(Con
tinued)
Whiteside
,2014
[43]
Pre-
test/
Post-
test
USA
39Prim
arycare
patients
receivingtreatm
entfora
new
episod
eof
depression
andwho
wereno
talready
receivingeither
pharm-
acothe
rapy
orpsycho
-therapy.Thosewho
had
received
treatm
entfora
depressive
episod
ein
the
lastsixmon
thswereno
teligibleto
participate.
Thrive:iCBT
prog
ram
consistin
gof
threemod
ules
delivered
over
athreeweekpe
riod.
Partici-p
ants
werealso
encouraged
toaccess
asecure
messaging
platform
atleaston
ceaweekforeigh
tweeks
toreceiveamoti-vational
interven
tionfro
maqu
alified
iCBT
coach.
N/A.
Suicidalideation:scores
onitem
13of
the
Symptom
Che
cklist-20.
Noapprop
riate
adjustmen
twas
madeforpo
tential
time-varyingconfou
nding.
Neither
participantsno
rclinicalpe
rson
nelw
ere
blindto
treatm
ent
allocatio
n.One
-fifth
(22.0%
)didno
tcomplete
treatm
ent.
Abb
reviations:C
TLcontrol,CB
Tcogn
itive
beha
viou
ralthe
rapy
,iCB
Tinternet-based
cogn
itive
beha
viou
ralthe
rapy
,DBT
dialectical
beha
viou
rtherap
y,DSM
-IVDiagn
ostic
andStatistical
Man
ualo
fMen
talD
isorde
rs,fou
rth
revision
,INVinterven
tion,
IPTinterpersona
lthe
rapy
,NSSIn
on-suicida
lself-injury,P
STprob
lem-solving
therap
y,RC
Trand
omised
controlledtrial,TA
Utreatm
entas
usua
l,UKUnitedKing
dom,U
SAUnitedStates
ofAmerica
a Alth
ough
participan
tswererand
omised
totheinterven
tionan
dcontrolcon
ditio
ns,allpa
rticipan
tsin
thisstud
yreceived
access
tothedigitalintervention.
Forthepu
rposes
ofthisreview
,the
refore,d
atafrom
thisRC
Twas
treatedas
pre-test/post-test
compa
rison
sinstead
Witt et al. BMC Psychiatry (2017) 17:297 Page 8 of 18
the self-management of suicidal ideation [39–41, 44, 47],and only one was developed for the self-management ofself-harm [50].In terms of study design, eight were randomised con-
trolled trials [37, 40, 41, 44, 46, 47, 49, 50], four were ob-servational pre-test/post-test studies [38, 39, 43, 48], andone was a pseudo-randomised controlled trial in whichsequential participants were alternately allocated to theintervention and control conditions [45]. For one RCT,although participants were randomised to the interven-tion and control conditions, all participants received ac-cess to the digital intervention [42]. For the purposes ofthis review, data from this RCT was therefore treated aspre-test/post-test comparisons instead. For two add-itional RCTs, outcomes at pre-test/post-test were alsoreported for the intervention group, enabling their inclu-sion in pre-test/post-test comparisons as well as in RCTcomparisons [45, 49]. However, to ensure results fromthese studies were not double-counted, analyses werepooled separately by study design for all outcomes re-ported in this review. In one further RCT, participantsassigned to the wait-list control condition crossed overto receive the intervention after six weeks [40]. To avoidcontamination from any ‘carry-over’ effects, we only ex-tracted data for the first six week period prior to cross-over as recommended [51].
Types of digital interventionsMost programs were developed by clinical psychologistsand/or psychiatrists with experience treating suicidalideation and/or self-harm [37, 39–45, 47–50], and wereevaluated by those who developed the intervention [37–40, 42, 44, 46–50]. In terms of therapeutic approach,most programs were based on the principles of cognitivebehavioural therapy (CBT). Some also included elementsof ‘third wave’ CBT, such as mindfulness [45], dialecticalbehaviour therapy [47], or mentalisation-based cognitivetherapy [47]; all of which hold the rationale that challen-ging thoughts, a principle feature of CBT, is less import-ant than understanding and accepting thoughts in anon-judgemental manner. Other programs included avariety of treatment approaches, including: acceptance-based therapy [40], problem-solving therapy [47], inter-personal therapy [42], mood monitoring [46], and crisisplanning [46]. Only one program utilised gamification inwhich participants were presented with a series of visualstimuli pairs designed to condition aversive reactions toself-harming thoughts or behaviours [50].
Types of control conditionsFor the eight RCTs and one pseudo-RCT designed trials,the interventions were compared against a number oftypes of control conditions, including: wait-list control[37, 40, 45], attentional control [41, 47, 50],
psychoeducation [44], treatment as usual [46], or face-to-face psychotherapy [49].
Ongoing studiesAn additional 10 ongoing studies were identified at thetime of the systematic search [52–59]. Further details ofthese ongoing trials are reported in Table 2.
Suicidal ideationFour studies reported data on the number of participantsself-reporting suicidal ideation. At post-intervention,there was some suggestion of a reduction in the propor-tion of participants self-reporting suicidal ideation in threeobservational pre-test/post-test studies (OR 0.36, 95% CI0.27 to 0.49, 3 studies, I2 = 0%, p < 0.0001; Fig. 2). How-ever, by the final follow-up assessment, data from oneRCT suggested no evidence of a treatment effect for theseinterventions (Fig. 2). There was no evidence of a signifi-cant difference in the magnitude of the treatment effect bystudy design for this outcome (χ2 = 0.61, df = 1, p = 0.44).As all four studies included in this analysis investigatedthe effectiveness of digital interventions specifically devel-oped for the self-management of depression symptoms(rather than suicidal ideation or self-harm specifically),sensitivity analyses could not be undertaken.One study covering three related RCTs reported infor-
mation on the frequency of self-reported episodes of sui-cidal ideation at post-intervention (for all three studies)and at the conclusion of the final follow-up assessment(for one of these three studies) [50]. However, no evi-dence of a treatment effect was found for these interven-tions at either time point (Fig. 3). As only one RCT wasincluded in this analysis, neither tests for subgroup dif-ferences nor sensitivity analyses could be undertaken.Eight studies reported data on suicidal ideation scores
using a number of different psychometric instruments,including: the BSSI [44, 47, 49], the Depressive SymptomInventory–Suicidality Subscale (DSI-SS; [60]) [40], theSuicide Behaviors Questionnaire–Revised (SBQ–R; [61])[45], the Suicidal Ideation Questionnaire–Junior (SIQ–J;[62]) [39], the four item suicidal ideation sub-scale ofthe General Health Questionnaire (GHQ–28; [63]) [37],and the suicidal ideation item of the Hopkins SymptomChecklist (HSCL–20; [64]) [43]. As raw means andstandard deviations were not reported for suicidal idea-tion scores in one RCT, data were instead estimatedfrom graphics in the original report for this study [37].At post-intervention, there was evidence these inter-
ventions were associated with a significant reduction insuicidal ideation scores in five RCTs (SMD -0.26, 95% CI-0.44 to −0.08, 5 studies, I2 = 0.0%, p = 0.005; Fig. 4).There was no evidence of a significant treatment effectfor these interventions in either one pseudo-randomisedcontrolled trial or four observational studies, however
Witt et al. BMC Psychiatry (2017) 17:297 Page 9 of 18
Table
2Stud
ycharacteristics,metho
dologicalinformation,andtrialreg
istrationinform
ationforthe10
ongo
ingstud
iesiden
tifiedby
thisreview
Stud
yand
Reference
Num
ber
Stud
yDesign
Cou
ntry
Target
NParticipantSource
Interven
tionCon
ditio
nCon
trol
Con
ditio
nMeasures
TrialR
egistration
Num
ber
INV
CTL
Boele,2014
[52]
RCT
Nethe
rland
s63
63Adu
ltswith
gradeII,III,orIV
glioma
(assessedaccordingto
WHOcriteria)
scoring12
orgreateron
theCen
ter
forEpidem
iologicalStudies
De-
pression
Scale.
Internet
self-gu
ided
prob
lem
solving
therapyconsistin
gof
fivemod
ules
delivered
once
aweekover
afive
weekpe
riod.
Waitlist.
Suicidalideation:scores
ontheBeck
ScaleforSuicidalIdeatio
n(BSSI).
Nethe
rland
sTrial
Register:N
TR3223.
Eylem,2015
[53]
RCT
Nethe
rland
sandUK
100
100
Com
mun
ity-dwellingadultseither
born
inTurkey,orwith
oneor
both
parentsbo
rnin
Turkey
with
ascore
ofon
eor
greateron
theBeck
SuicidalIdeatio
nScale.
iCBT:C
onsistingof
betw
eensixand
eigh
tmod
ules
delivered
once
aweekover
asixweekpe
riod.
Waitlist.
Suicidalideation:scores
ontheBeck
ScaleforSuicidalIdeatio
n(BSSI)and
theSuicidalIdeatio
nAttrib
uteScale
(SIAS).
Not
stated
.
Gum
pert,
2016
[Not
Publishe
d]
Pre-
test/
Post-
test
Swed
en43
Ado
lescen
tsdiagno
sedwith
NSSI,
defined
accordingto
sectionthree
oftheDSM
-5,w
ithat
leaston
eep
i-sode
ofNSSIo
ccurrin
gwith
inthe
pastmon
th.
Internet
emotionregu
latio
ntraining
.Noinform
ationon
thenu
mbe
rof
mod
ules
orthedu
ratio
nof
therapy
wereprovided
.
N/A.
Self-Harm:self-rep
ortedself-harm
ac-
cordingto
theDeliberateSelf-Harm
Inventory(DSH
I).Suicidalideation:scores
onthe
SuicidalIdeatio
nQuestionn
aire-
Junior
(SIQ-JR).
ClinicalTrialsRegister:
NCT02697019.
Kaslow
,2016
[Not
Publishe
d]
Pre-
test/
Post-
test
USA
25Referralsforbe
haviou
ralh
ealth
treatm
entthroug
han
outpatient
men
talh
ealth
servicefollowinga
suicideattempt
orclinically
sign
ificant
suicidalideatio
n.
RefliefLink:A
pp-based
moo
dtracking
andstress
managem
entinterven
tion
consistin
gof
mod
ules
ofrelaxatio
nexercises,gu
ided
med
itatio
n,pro-
gressive
re-laxatio
n,andmindfulne
ssskillstraining
.Noinform
ationon
the
numbe
rof
mod
ules
orthedu
ratio
nof
therapywereprovided
.
N/A.
Suicidalideationan
dbeha
viours:
measuredaccordingto
scores
ontheColum
biaSuicideSeverity
Ratin
gScaleScreen
er(C-SSRS)
aswellasthenu
mbe
rof
participants
self-repo
rtingep
isod
esof
self-injury.
ClinicalTrialsRegister:
NCT02691221.
Müh
lmann,
2017
[54]
RCT
Den
mark
175
175
Com
mun
ity-dwellingadultsscoring
threeor
greateron
theBeck
Scale
forSuicidalIdeatio
n.
iCBT:C
onsistingof
sixmod
ules
delivered
once
aweekover
asix
weekpe
riod.
Waitlist.
Suicidalideation:scores
ontheBeck
ScaleforSuicidalIdeatio
n(BSSI)and
theSuicidalIdeatio
nAttrib
uteScale
(SIAS).
Self-Harm:n
umbe
rof
participants
with
aho
spitaland
/ormed
ical
record
forself-harm
.Suicide:nu
mbe
rof
suicidede
aths
accordingto
anatio
nalm
ortality
register.
ClinicalTrialsRegister:
NCT02872610.
Perry,2015
[55]
cRCT
Australia
1600
(30
scho
ols)
Com
mun
ity-dwellingadoles-cen
tsen
rolledin
theirfinalyear
ofhigh
scho
ol.
SPAR
X-R:Internet
gamified
iCBT
prog
ram
consistin
gof
seven
mod
ules
delivered
once
aweekover
sevenweeks.
Atten
tion
placeb
o.Suicidalideationan
d/or
attempted
suicide:scores
onthesuicidal
thou
ghts,p
lans,and
attempted
suicideitemsof
theYo
uthRisk
Behaviou
rSurvey
(YRBS).
AustralianandNew
ZealandClinicalTrials
Register:
ACTRN12614000316606.
Robinson
,2014
[56]
cRCT
Australia
170
(28
scho
ols)
Com
mun
ity-dwellingadoles-cen
tsen
rolledin
high
scho
olself-
repo
rtinganysuicidalideatio
nover
thepastmon
th.
Refra
me-IT:iCBT
prog
ram
consistin
gof
eigh
tmod
ules
delivered
over
a10
to12
weekpe
riod.
TAU.
Suicidalideation:scores
onthe
SuicidalIdeatio
nQuestionn
aire
(SIQ).
Self-Harm
andAttempted
Suicide:
scores
onaspecifically-develop
edinstrumen
t.
AustralianandNew
ZealandClinicalTrials
Register:
ACTRN12613000864729.
USA
>2500
TAU.
Witt et al. BMC Psychiatry (2017) 17:297 Page 10 of 18
Table
2Stud
ycharacteristics,metho
dologicalinformation,andtrialreg
istrationinform
ationforthe10
ongo
ingstud
iesiden
tifiedby
thisreview
(Con
tinued)
Simon
,2016
[57]
Zelen
RCT
Referralsto
anou
tpatient
men
tal
health
and/or
gene
ralm
edical
centre
who
repo
rtdaily
oralmost
daily
suicidalideatio
naccordingto
scores
onitem
9of
thePatient
Health
Questionn
aire-9.
Internet
DBT
with
aspecificfocuson
thede
velopm
entof
mindfulne
ssskills,no
n-judg
emen
talaccep
tance
ofem
otions,opp
osite
actio
n,and
pacedbreathing.
Thisprog
ram
will
consistof
four
self-pacedmod
ules.
Suicidalideation:scores
ona
specifically
develope
dsix-item
ver-
sion
oftheColum
biaSuicideSever-
ityRatin
gScaleScreen
er(C-SSRS).
ClinicalTrialsRegister:
NCT02326883.
Stallard,
2016
[58]
Pre-
test/
Post-
test
UK
50Com
mun
ity-dwellingadoles-cen
tswith
ahistoryof
repe
ated
self-harm
who
arecurren
tlyreceivingon
-go
ingcare
inou
t-patient
child
and
adolescent
men
talh
ealth
services.
BlueIce:App
-based
toolbo
x,de
vel-
oped
inconsultatio
nwith
youn
gpe
oplewho
self-harm
,including
amoo
ddiary,moo
d-liftin
gexercises
draw
nfro
mCBT
andDBT,p
hysical
activities,and
mindfulne
ssexercises.
Part-icipantsarealso
encouraged
toup
load
andshareinspiratio
nalp
ho-
tosandmusicfiles.Participantswill
useBlueIcedaily
for10
weeks.
N/A.
Self-Harm:m
easuredaccordingto
self-repo
rt,and
GPrepo
rt,and
/or
med
icalrecords.
Not
stated
.
VanSpijker,
2015
[59]
RCT
Australia
570
Com
mun
ity-dwellingadults
repo
rtingcurren
tsuicidalideatio
naccordingto
item
fiveof
the
Colum
bia-SuicideSeverityRatin
gScale,bu
twho
dono
tself-repo
rtattemptingsuicidewith
inthepast
mon
th.
iCBT:C
onsistingof
sixmod
ules
delivered
once
aweekover
asix
weekpe
riod.
Atten
tion
placeb
o.Suicidalideation:scores
onthe
Intensity
ofSuicidalIdeatio
nsub-
scaleof
theColum
biaSuicideSever-
ityRatin
gScaleScreen
er(C-SSRS)
andtheSuicidalIdeatio
nAttrib
ute
Scale(SIAS).
AustralianandNew
ZealandClinicalTrials
Register:
ACTRN12613000410752.
Abb
reviations:C
TLcontrol,CB
Tcogn
itive
beha
viou
ralthe
rapy
,iCB
Tinternet-based
cogn
itive
beha
viou
ralthe
rapy
,cRC
Tclusterrand
omised
controlledtrial,DBT
dialectical
beha
viou
rtherap
y,DSM
-IVDiagn
ostic
andStatis-
tical
Man
ualo
fMen
talD
isorde
rs,fou
rthrevision
,DSM
-5Diagn
ostic
andStatistical
Man
ualo
fMen
talD
isorde
rs,fifthrevision
,GPge
neralp
ractition
er,INVinterven
tion,
IPTinterpersona
lthe
rapy
,NSSIn
on-suicida
lself-
injury,P
STprob
lem-solving
therap
y,RC
Trand
omised
controlledtrial,TA
Utreatm
entas
usua
l,UKUnitedKing
dom,U
SAUnitedStates
ofAmerica
Witt et al. BMC Psychiatry (2017) 17:297 Page 11 of 18
(Fig. 4). The test for subgroup differences was non-significant suggesting there was no difference in magni-tude of the effect size by study design for this outcome(χ2 = 0.25, df = 2, p = 0.88). Sensitivity analyses includ-ing only those studies in which the intervention was spe-cifically developed for the self-management of suicidalideation or self-harm also did not materially affect theseresults (results not shown).Two RCTs reported data on suicidal ideation at the
final follow-up assessment [37, 44]. For one of these tri-als, however, data on suicidal ideation scores at finalfollow-up had to be estimated by review authors fromgraphics presented in the original trial report [37]. Datafrom these trials suggested these digital interventionswere not associated with a significant treatment effect
for suicidal ideation by this time point (SMD -0.34, 95%CI –0.70 to 0.01, 2 studies, I2 = 0.0%, p = 0.06; Fig. 4).Given that both studies included in this analysis wereRCTs, neither tests for subgroup differences nor sen-sitivity analyses could be undertaken. One furtherRCT presented data on outcomes at final follow-upfor the intervention group only [65]. Analyzing thesedata as pre-test/post-test comparisons suggested asignificant reduction in suicidal ideation scores atthe final follow-up assessment in this trial (MD-4.90, 95% CI -7.07 to −2.73, 1 study, I2 = not ap-plicable, p < 0.001). As only one RCT reported dataat the final follow-up assessment, neither tests forsubgroup differences nor sensitivity analyses couldbe undertaken.
Fig. 2 Random effects odds ratio (OR) and accompanying 95% confidence interval (CI) for digital interventions on the proportion of participantsreaching defined clinical thresholds for suicidal ideation
Fig. 3 Random effects mean difference (MD) and accompanying 95% confidence interval (CI) for digital interventions on frequency of self-reported suicidal ideation
Witt et al. BMC Psychiatry (2017) 17:297 Page 12 of 18
Fig. 4 Random effects standard mean difference (SMD) and accompanying 95% confidence interval (CI) for digital interventions of suicidalideation scores
Fig. 5 Random effects mean difference (MD) and accompanying 95% confidence interval (CI) for digital interventions on frequency of self-reportedself-cutting and non-suicidal self-injury (NSSI)
Witt et al. BMC Psychiatry (2017) 17:297 Page 13 of 18
Self-harmOne study, covering three related RCTs, evaluated theeffectiveness of digital interventions on frequency ofself-harm episodes [50]. At post-intervention, there wasno indication of a treatment effect for these interventionson either self-reported frequency of self-cutting or non-suicidal self-injury in these three RCTs (Fig. 5). There wasalso no indication of a treatment effect for this interven-tion on frequency of self-reported self-cutting or non-suicidal self-injury at the final follow-up assessment (atone month) in one of these RCTs (Fig. 5). Sensitivity ana-lyses could not be undertaken for this outcome as onlyone study investigated outcomes relating to repetition ofself-harm. As all three studies were RCTs, subgroup ana-lyses to investigate the impact of study design on the mag-nitude of the effect size could also not be undertaken.
Combined self-harm and attempted suicideOne RCT evaluated the effectiveness of a digital inter-vention on attempted suicide and self-harm [46]. As re-sults for self-harm could not be disaggregated from thatfor attempted suicide, they are instead analysed here as acombined outcome. There was no evidence of a reduc-tion in the proportion of participants who attemptedsuicide and/or engaged in self-harm over a 24 monthfollow-up period in this study (OR 2.11, 95% CI 0.19 to23.81, 1 study, I2 = not applicable, p = 0.55). Once again,as only one study investigated outcomes relating to com-bined attempted suicide and/or self-harm, sub-groupand sensitivity analyses could not be undertaken.
Attempted suicideOne RCT evaluated the effectiveness of a digital interven-tion on attempted suicide [47]; however, no evidence of areduction in the proportion of participants self-reporting asuicide attempt was noted by the post-intervention assess-ment in this study (OR 0.58, 95% CI 0.16 to 2.02, 1 study,I2 = not applicable, p = 0.39). As only one study investi-gated outcomes relating to combined attempted sui-cide, sub-group and sensitivity analyses could not beundertaken.
DiscussionThis systematic review and meta-analysis evaluated theeffectiveness of digital interventions, including both onlineand mobile telephone applications (‘apps’) for the self-management and/or treatment of suicidal ideation or be-haviours. A total of 14 studies were included in thepresent review, reporting data for a total of 3356participants.Overall, this review found some evidence that digital
interventions may be associated with reductions in sui-cidal ideation, particularly at the post-intervention as-sessment. It is notable, however, that where these
interventions were associated with significant treatmentbenefits for suicidal ideation, these effects tended to bestronger in observational pre-test/post-test-designed stud-ies as compared with RCTs. Most of the included studiesconducted to date, however, have utilised a pre-test/post-test observational design. There was no evidence to sug-gest these interventions are associated with reductions inself-harm or attempted suicide, although only three stud-ies investigated these outcomes [46, 47, 50]. Few of thesestudies would have been adequately powered to evaluaterare outcomes, including repetition of self-harm and sui-cide reattempts.Adherence was poor in majority of these studies; al-
though this was not clearly related to the number oftreatment modules. Of those studies that reported infor-mation on adherence [38–50], for example, up to one-half of participants allocated to the intervention groupdid not complete all treatment modules. Adherence dur-ing the long-term follow-up period, which was reportedin one RCT [50], was also poor; over one-half (64%) ofparticipants allocated to the intervention group in thistrial did not access the intervention at all during the onemonth follow-up period in this RCT [50]. This suggeststhat the use of digital innovations, including gamifica-tion, may be insufficient to keep these interventions en-gaging over the longer term.
Limitations of the included studiesMost studies were of low to moderate quality, as assessedusing the Cochrane Collaboration’s tool for randomisedand pseudorandomised controlled trials [35], or theROBINS–I tool for controlled pre-test/post-test studies[36], with biases most apparent for the domains of partici-pant, clinical personnel, and outcome assessor blinding.Performance and detection bias therefore cannot be ruledout.Of those studies utilising an RCT design, most com-
pared the intervention to either waitlist control [37, 45]or attentional control [41, 47, 50] conditions. Variabilityin the control condition has been found to be associatedwith the magnitude of the treatment effect in office-based psychosocial interventions for depression and anx-iety [66, 67]. Yet, despite this, variability in the controlcondition is a rarely investigated source of heterogeneityin meta-analyses of digital interventions.The included studies report data on a variety of differ-
ent outcomes. Although all studies included at least onesuicidal ideation and/or behaviour relevant outcome,these were measured in a variety of different ways, in-cluding from psychometric scales, self-report, or accord-ing to hospital or medical records. Given that theseoutcomes were measured using the same methodologywithin individual studies, the effect of outcome measuredefinition on case identification at the post-intervention
Witt et al. BMC Psychiatry (2017) 17:297 Page 14 of 18
and follow-up assessments could be expected to affectthe intervention and control groups equally. Betweenstudies, however, outcome measure definition may haveaffected our results in light of research findings suggest-ing that self-reported self-harm underestimates hospital-treated self-harm [68]. We were unable to assesswhether variability in the magnitude of the treatment ef-fect size was related to the way in which the outcomemeasure was assessed in this review, however, owing tothe small number of studies included in any one meta-analysis. Further work will be necessary to pick apart theinfluence of outcome definition on the apparent effect-iveness of interventions, including digital interventions,for the prevention of self-harm.Most studies (71.4%) also reported information on de-
pression [38–40, 42–45, 47–49]. Fewer studies reporteddata on other clinically relevant outcomes, such as hope-lessness [40], and none reported information onproblem-solving following the completion of the inter-vention. The National Institute for Health and ClinicalExcellence (NICE) guidelines recommend that all inter-ventions for self-harm should, at a minimum, investigatethe effect of psychological interventions for self-harm onpotential mechanisms of action, including depression,hopelessness, and problem-solving [69], echoing morerecent calls to this effect in the international scholarship[70, 71]. Online resources may normalize self-harm andmay provide vulnerable individuals with access to self-harm content and imagery, including information onmethods of self-harm [72]. Additionally, digital interven-tions that provide some form of active psychosocial ther-apy, and particularly those that include a focus on moodmonitoring, may lead to increased negative affectivityand rumination [73]. Outcomes relating to negativeaffect and rumination should therefore also be reportedfor all evaluations of digital interventions for this patientgroup in future.Most programs (57.1%) were developed for the self-
management of depression [37, 38, 42, 43, 45, 46, 48,49]. Recent findings for psychosocial interventions, how-ever, suggest that those developed specifically for themanagement of suicidal thinking are associated withgreater impacts for the reduction of attempted and com-pleted suicide as compared to those targeting indirectsymptoms associated with suicidal behaviour, such asanxiety, depression, or hopelessness [31]. Future studiesin this area should evaluate the degree to which thesedigital interventions lead to meaningful change in theproposed mechanism(s) of action in order to identify thetreatment module(s) associated with the greatest impactsin reducing suicidal ideation and self-harm in thispopulation.The included studies also examined interventions of
varying intensity. Whilst it could be expected that
treatments of greater intensity will have greater impactson reducing suicidal ideation and self-harm, a recentmeta-regression review found no evidence to suggestthat treatment intensity, measured as the total number ofavailable treatment sessions, was associated with greatereffectiveness for office-based psychosocial therapy for self-harm repetition [74].Finally, it is also likely that these populations will have
a very low risk for suicidal behaviour as all studies re-cruited participants from the community. Despite this, al-most all (78.6%) studies included indicated samples,such as callers to telephone or online counselling ser-vices [37, 45, 50], or those already in contact with pri-mary care [38, 42, 43], counselling [39, 40, 49] orpsychiatric services [46, 48].
Strengths and limitations of the present reviewThe majority of these interventions were based on theprinciples of standard cognitive behavioural therapy(CBT), which has been found to have efficacy in redu-cing repetition of self-harm in clinical populations in arecent systematic review of psychosocial interventionsfor the treatment of self-harm [10]. The findings of thepresent review significantly extend this by suggestingthat digital interventions which incorporate the princi-ples of standard CBT may have promising effects in re-ducing suicidal ideation in non-clinical populations, atleast in the short-term.Whilst we utilized a comprehensive search to locate all
relevant trials of digital interventions for the self-management of self-harm, we identified only two mobiletelephone apps [40, 50]. Given that a recent Australianstudy identified a total of 24 apps for the prevention ofsuicidal behaviour are currently available for downloadfrom the Australian Google Play and iOS store [75], thiswould suggest that a large number of these apps have noevidence to support their effectiveness. It is likely that asimilar proportion of online interventions would havelittle evidence to support their effectiveness.
ConclusionsAlthough a growing number of both online and mobiletelephone applications (‘apps’) for the self-managementand treatment of suicidal thinking and behaviours arenow available, few of these have been evaluated for theireffectiveness in reducing these outcomes. We identifiedjust 14 in this review. Overall, while there is some prom-ise of these interventions to reduce suicidal ideation,how this translates into reductions in self-harm and/orattempted suicide is unclear at present. Given the preva-lence of suicidal ideation in clinical populations, add-itionally, it is unclear whether these reductions would beclinically meaningful at present.
Witt et al. BMC Psychiatry (2017) 17:297 Page 15 of 18
AppendixElectronic Search Strategy for the Medline and theCochrane LibraryProvides keywords used and electronic search strategyfor the Medline and the Cochrane Library. Current to 31March, 2017.Electronic search strategy
AbbreviationsApp: Application; BDI–II: Beck Depression Inventory, version two; BSSI: Beck Scaleof Suicidal Ideation; CBT: Cognitive behavioural therapy; CDRS–R: Children’sDepression Rating Scale–Revised; CENTRAL: Cochrane Central Register ofControlled Trials; CES–D: Center for Epidemiologic Studies Depression scale;CI: Confidence Interval; CRCT: Cluster randomised controlled trial; CRESP: Centrefor Research Excellence in Suicide Prevention; DBT: Dialectical behaviour therapy;DSI–SS: Depressive Symptom Inventory-Suicidality Subscale; DSM-5: Diagnosticand Statistical Manual of Mental Disorders, fifth revision; GHQ–28: General HealthQuestionnaire, 28 item; HSCL–20: Hopkins Symptom Checklist, 20 item;MADRS: Montgomery-Åsberg Depression Rating Scale; ME: Mean difference;OR: Odds ratio; PHQ–9: Patient Health Questionnaire, 9 item; PRISMA: PreferredReporting Items for Systematic Reviews and Meta-Analyses; RADS–2: ReynoldsAdolescent Depression Scale-Version 2; RCT: Randomised controlled trial; ROBINS–I: Risk of Bias in Non-Randomized Studies of Interventions; SBQ–R: SuicideBehaviors Questionnaire-Revised; SIQ–J: Suicidal Ideation Questionnaire-Junior;SMD: Standard mean difference; TAU: Treatment as usual; UK: United Kingdom;USA: United States of America; WHO: World Health Organization
AcknowledgementsThe authors wish to thank Constance Guille, Joe Franklin, Bregje van Spijker, andBen van Voorhees for providing raw data and/or methodological clarificationrelating to their studies.
Table 3 Cochrane Library
Keyword Number of ‘hits’
#1 ‘mhealth’ 4
#2 ‘ehealth’ 2
#3 web 790
#4 online 178
#5 internet 126
#6 ‘mobile device*’ 10
#7 ‘mobile phone’ 23
#8 ‘cell* phone’ 18
#9 smartphone 4
#10 ‘phone app*’ 39
#11 ‘mobile app*’ 36
#12 ‘cell* app*’ 1134
#13 ‘tablet computer*’ 6
#14 ipad* 0
#15 iphone* 0
#16 samsung 0
#17 android 0
#18 ‘windows phone*’ 0
#19 #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7OR #8 OR #9 OR #10 OR #11 OR #12 OR #13OR #14 OR #15 OR #16 OR #17 OR #18
2007
#20 ‘auto$mutilat*’ 0
#21 ‘self$harm’ 19
#22 ‘self$injur*’ 11
#23 ‘self$mutilat*’ 1
#24 ‘self$cutt*’ 0
#25 ‘self$poison*’ 5
#26 ‘drug overdos*’ 16
#27 overdos* 22
#28 ‘self$destructive behavio*’ 0
#29 ‘suicid* attempt*’ 14
#30 suicid* 69
#31 #20 OR #21 OR #22 OR #23 OR #24 OR #25OR #26 OR #27 OR #28 OR #29 OR #30
102
#32 #19 AND #31 32
Dates searched: Start Date to 30.06.2016
Table 4 MEDLINE (OVID interface)
Keyword Number of ‘hits’
#1 ‘mhealth’.mp 741
#2 ‘ehealth’.mp 1199
#3 web.mp 61,883
#4 online.mp 56,082
#5 internet.mp 76,766
#6 ‘mobile device*’.mp 1159
#7 ‘mobile phone’.mp 2999
#8 ‘cell* phone’.mp 1540
#9 smartphone.mp 2325
#10 ‘phone app*’.mp 229
#11 ‘mobile app*’.mp 2291
#12 ‘cell* app*’.mp 19,603
#13 ‘tablet computer*’.mp 340
#14 ipad*.mp 696
#15 iphone*.mp 400
#16 samsung,mp 527
#17 android.mp 1057
#18 ‘windows phone*’.mp 11
#19 #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7OR #8 OR #9 OR #10 OR #11 OR #12 OR #13OR #14 OR #15 OR #16 OR #17 OR #18
186,347
#20 ‘auto?mutilat*’.mp 123
#21 suicide, attempted/ or self-injurious behavior/or‘self?harm’.mp.
22,723
#22 suicidal ideation/ or suicide, attempted/ orsuicide/or suicid*.mp.
75,063
#23 overdos*’.mp 16,949
#24 #20 OR #21 OR #22 OR #23 92,822
#25 #19 AND #24 1265
Dates searched: Start Date to 31.03.2017
Witt et al. BMC Psychiatry (2017) 17:297 Page 16 of 18
FundingThis work was supported by grants awarded to AM by beyondblue and theMovember Foundation. No other specific funding was received for theremaining authors. Funders had no role in either the design, interpretationof the findings, or writing of this manuscript.
Availability of data and materialsThis review involves analysis of previously published data available in thestudies included in this review. The final dataset is available from thecorresponding author upon reasonable request.
Authors’ contributionsKW had the idea for the paper, conducted the systematic review, extractedand analysed the data. AM and MS assisted with data extraction and analysis.All authors contributed to the writing of the manuscript and approved thefinal version of the manuscript for publication.
Ethics approval and consent to participateEthical approval and participant consent were not required for this review,since the study involved review and analysis of previously published data.
Consent for publicationNot applicable.
Competing interestsTwo of the authors of this review (SH, JR) were authors of one of theincluded studies. The remaining authors have no other competing interestto declare.
Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.
Author details1Population Health, Turning Point, Eastern Health Clinical School, MonashUniversity, 54-62 Gertrude Street, Fitzroy, Victoria 3065, Australia. 2MelbourneSchool of Population and Global Health, University of Melbourne, Melbourne,Victoria, Australia. 3Centre for Translational Neuroscience and Mental Health,Faculty of Health and Medicine, University of Newcastle, Callaghan, NewSouth Wales, Australia. 4Orygen, the National Centre of Excellence in YouthMental Health and the Centre for Youth Mental Health, University ofMelbourne, Melbourne, Victoria, Australia. 5Centre for Health Equity,Melbourne School of Population and Global Health, University of Melbourne,Melbourne, Victoria, Australia.
Received: 15 May 2017 Accepted: 8 August 2017
References1. Hawton K, Zhal D, Weatherall R. Suicide following deliberate self-harm: long
term follow-up of patients who presented to a general hospital. Br J Psychiatry.2003;182:537–42.
2. Logan C. Managing Clinical Risk: A Guide to Effective Practice. In: Logan C,Johnstone L, editors. Suicide and self-harm: Clinical risk assessment andmanagement using a structured professional judgement approach.Abingdon: United Kingdom: Routledge; 2013. p. 115–25.
3. McLean J, Maxwell M, Platt S, Harris F, Jepson R. Risk and protective factorsfor suicide and suicidal behaviour: a literature review. Edinbrugh, UnitedKingdom: Scottish Government; 2008.
4. Paul E, Tsypes A, Eidlitz L, Ernhout C, Whitlock J. Frequency and functions ofnon-suicidal self-injury: associations with suicidal thoughts and behaviors.Psychiatry Res. 2015;255:276–82.
5. Carroll R, Metcalfe C, Gunnell D. Hospital presenting self-harm and risk offatal and non-fatal repetition: systematic review and meta-analysis. PLoSOne. 2014;9(2):e89944.
6. Franklin J, Nock M. The Oxford Handbook of Behavioral Emergencies andCrises. In: Keleespies P, editor. Nonsuicidal self-injury and its relation tosuicidal behavior. New York, NY: Oxford University Press; 2017.
7. Kapur N, Cooper J, O'Connor R, Hawton K. Non-suicidal self-injury v.Attempted suicide: new diagnosis or false dichotomy? Br J Psychiatry. 2013;202(5):326–8.
8. Hawton K, Harriss L, Hall S, Simkin S, Bale E, Bond A. Deliberate self-harm inOxford, 1990-2000: A time of change in patient characteristics. Psychol Med.2003;33:987–95.
9. Hawton K, Witt K, Taylor Sailsbury T, Arensman E, Gunnell D, Townsend E,van Heeringen K, Hazell P. Interventions for self-harm in children andadolescents. Cochrane Database Syst Rev. 2015;9:CD012013.
10. Hawton K, Witt K, Taylor Salisbury T, Arensman E, Gunnell D, Hazell P, vanHeeringen K. Psychosocial interventions for self-harm in adults. CochraneDatabase Sys Rev. 2016;14:CD012189.
11. Hawton K, Witt KG, Salisbury TLT, Arensman E, Gunnell D, Hazell P,Townsend E, van Heeringen K. Psychosocial interventions following self-harm in adults: a systematic review and meta-analysis. Lancet Psychiatry.2016;3(8):740–50.
12. Tondo L, Albert M, Baldessarini R. Suicide in relation to health care access inthe United States: an ecological study. J Clin Psychiatry. 2006;67(4):517–23.
13. Kapusta N, Poschm N, Niederkrotenthaler T, Fischer-Kern M, Etzersdorfer M,Sonneck G. Availability of mental health service providers and suicide ratesin Austria: a nationwide survey. Psychiar Serv. 2010;61(12):1198–203.
14. Kawaguchi H, Koike S. Association between the density of physicians andsuicide rates in Japan: Nationwide ecological study using a spatial Bayesianmodel. PLoS One. 2016;11(2):e0148288.
15. Bruffaerts R, Demytenaere K, Hwang I, Chiu W, Sampson N, Kessler R, Alonso J,Borges G, de Girolamo G, de Graaf R, et al. Treatment of suicidal peoplearound the world. Br J Psychiatry. 2011;199(1):64–70.
16. Czyz E, Horwitz A, Eisenberg D, Kramer A, King C. Self-reported barriers toprofessional help seeking among college students at elevated risk forsuicide. J Am College Health. 2013;61(7):398–406.
17. Aggarwal N. Applying mobile technologies to mental health service deliveryin South Asia. Asian J Psychiatr. 2012;5(3):225–30.
18. de Beurs D, Kirtley O, Kerkhof A, Portzky G, O'Connor R. The role of mobilephone technology in understanding and preventing suicidal behavior.Crisis. 2015;36(2):79–82.
19. Kerkhof A, van Spijker B, Mokkenstorm J. Suicide Prevention and NewTechnologies: Evidence-Based Practice. In: Mishara B, Kerkhof A, editors.Reducing the burden of suicidal thoughts through online cognitivebehavioural therapy self-help. New York, NY: Palgrave Macmillian; 2013.
20. Hayes J, Maughan D, Grant-Peterkin H. Interconnected or disconnected?Promotion of mental health and prevention of mental disorder in thedigital age. Br J Psychiatry. 2016;208(3):205–7.
21. Moher D, Liberati A, Tetzlaff J, Altman D. Preferred reporting items forsystematic reviews and meta-analyses: the PRISMA statement. BMJ.2009;339(7):b2535.
22. Christensen H, Batterham P, O'Dea B. E-health interventions for suicideprevention. Int J Environ Res Public Health. 2014;11(8):8193–212.
23. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M-R, Christensen H.Smartphones for smarter delivery of mental health programs: a systematicreview. J Med Internet Res. 2013;15(11):e247.
24. Lai M, Maniam T, Chan L, Ravindran A. Caught in the web: a review ofweb-based suicide prevention. J Med Internet Res. 2014;16(1):e30.
25. Jacob N, Scourfield J, Evans R. Suicide prevention via the internet: adescriptive review. Crisis. 2014;35(4):261–7.
26. Perry Y, Werner-Seidler A, Calear A, Christensen H. Web-based and mobilesuicide prevention interventions for young people: a systematic review. JCan Acad Child Adolesc Psychiatry. 2016;25(2):73–9.
27. Deeks J, Higgins J, Altman D. Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S, (editors). Cochrane Handbook forSystematic Reviews of Interventions Version 5.1.0. The CochraneCollaboration; 2011. Available from http://handbook.cochrane.org.
28. Beck A, Kovacs M, Weissman A. Assessment of suicidal intention: the scalefor suicidal ideation. J Consult Clin Psychol. 1979;47(2):343–52.
29. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials.1986;7(3):177–88.
30. Higgins J, Thompson S, Deeks J, Altman D. Measuring inconsistency inmeta-analysis. BMJ. 2003;327(7414):557–60.
31. Meerwijk E, Parekh A, Oquendo M, Allen I, Franck L, Lee K. Direct versusindirect psychosocial and behavioural interventions to prevent suicide andsuicide attempts: a systematic review and meta-analysis. Lancet Psychiatry.2016;3:544–54.
32. Faber T, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Meta-analysesincluding non-randomized studies of therapeutic interventions: amethodological review. BMC Med Res Methol. 2016;16:35.
Witt et al. BMC Psychiatry (2017) 17:297 Page 17 of 18
33. Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literaturereview on the representativeness of randomized controlled trial samplesand implications for the external validity of trial results. Trials. 2015;16:495.
34. Parker L, Saez N, Porta M, Hernández-Aguado I, Lumbreras B. The impact ofincluding different study designs in meta-analyses of diagnostic accuracystudies. Eur J Epidemiol. 2013;28:713–20.
35. Higgins J, Altman D, Sterne J. Assessing risk of bias in included studies. In:Higgins JPT, Green S, (editors). Cochrane Handbook for Systematic Reviewsof Interventions Version 5.1.0. The Cochrane Collaboration; 2011. Availablefrom http://handbook.cochrane.org.
36. A tool for assessing Risk Of Bias In Non-randomized Studies of Interventions(ROBINS-I), version 7.0. http://www.riskofbias.info. Accessed 7 Mar 2016.
37. Christensen H, Farrer L, Batterham P, Mackinnon A, Griffiths K, Donker T. Theeffect of a web-based depression intervention on suicidal ideation:secondary outcome from a randomised controlled trial in a helpline. BMJOpen. 2013;3(6):e002886.
38. Mewton L, Andrews G. Cognitive behaviour therapy via the internet fordepression: a useful strategy to reduce suicidal ideation. J Affect Disord.2015;170(1):78–84.
39. Robinson J, Hetrick S, Cox G, Bendall S, Yuen H, Yung A, Pirkis J. Can aninternet-based intervention reduce suicidal ideation, depression andhopelessness among secondary school students: results from a pilot study.Early Interv Psychiatry. 2016;10(1):28–35.
40. Tighe J, Shand F, Ridani R, Mackinnon A, de la Mata N, Christensen H. iBobblymobile health intervention for suicide prevention in Australian indigenousyouth: a pilot randomised controlled trial. BMJ Open. 2017;7:e013518.
41. Guille C, Zhao Z, Krystal J, Nichols B, Brady K, Sen S. Web-based cognitivebehavioral therapy intervention for the prevention of suicidal behaviour inmedical interns: a randomized clinical trial. JAMA Psychiatry. 2015;72(12):1192–8.
42. van Voorhees B, Fogel J, Reinecke M, Gladstone T, Stuart S, Gollan J,Bradford N, Domanico R, Fagan B, Ross R, et al. Randomized clinical trial ofan internet-based depression prevention program for adolescents (projectCATCH-IT) in primary care: 12-week outcomes. J Dev Behav Pediatr.2009;30(1):23–37.
43. Whiteside U, Richards J, Steinfeld B, Simon G, Caka S, Tachibana C, StuckeyS, Ludman E. Online cognitive behavioral therapy for depressed primarycare patients: a pilot feasibility project. Perm J. 2014;18(2):21–7.
44. Hill RM, Pettit JW. Pilot Randomized Controlled Trial of LEAP: A SelectivePreventive Intervention to Reduce Adolescents’ Perceived Burdensomeness.J Clin Child Adolesc Psychol. 2016. ePub ahead of print. doi:10.1080/15374416.15372016.11188705.
45. Moritz S, Schilling L, Hauschildt M, Schröder J, Treszel A. A randomizedcontrolled trial of internet-based therapy in depression. Behav Res Ther.2012;50(7–8):513–21.
46. Kordy H, Wolf M, Aulich K, Bürgy M, Hegerl U, Hüsing J, Puschner B,Rummel-Kluge C, Vegger H, Backenstrass M. Internet-delivered diseasemanagement for recurrent depression: a multicentre randomized controlledtrial. Psychother Psychosom. 2016;85(2):91–8.
47. van Spijker B, van Straten A, Kerkhof A. Effectiveness of online self-help forsuicidal thoughts: results of a randomised controlled trial. PLoS One. 2014;9(2):e90118.
48. Hedman E, Ljótsson B, Kaldo V, Hesser H, Alaoui S, Kraepelien M, AnderssonE, Rück C, Svanborg C, Andersson G, et al. Effectiveness of internet-basedcognitive behaviour therapy for depression in routine psychiatric care. JAffective Disord. 2014;155(2):49–58.
49. Wagner B, Horn A, Maercker A. Internet-based versus face-to-face cognitivebehavioral intervention for depression: a randomized controlled non-inferoritytrial. J Affective Disord. 2014;152(7):113–21.
50. Franklin J, Fox K, Franklin C, Kleiman E, Ribeiro J, Jaroszewski A, Hooley J,Nock M. A brief mobile app reduces nonsuicidal and suicidal self-injury:evidence from three randomized controlled trials. J Consult Clin Psychol.2016;84(6):544–57.
51. Elbourne D, Altman D, Higgins J, Curtin F, Worthington H, Vail A. Meta-analyses involving cross-over trials: methodological issues. Int J Epidemiol.2002;31:140–9.
52. Boele F, Verdonck-de-Leeuw I, Cuijpers P, Reijneveld J, Heimans J, Klein M.Internet-based guided self-help for glioma patients with depressivesymptoms: design of a randomized controlled trial. BMC Neurol. 2014;14:81.
53. Eylem O, van Straten A, Bhui K, Kerkhof AJ. Reducing suicidal ideationamong Turkish migrants in the Netherlands and in the UK: effectiveness ofan online intervention. Int Rev Psychiatry. 2015;27(1):72–81.
54. Mühlmann C, Madsen T, Hjorthøj C, Kerkhof A, Nordentoft M, Erlangsen A.The self-help online against suicidal thoguhts (SOS) trial: study protocol fora randomized controlled trial. Trials. 2017;18:45.
55. Perry Y, Calear A, Mackinnon A, Batterham P, Licinio J, King C, Thomsen N,Scott J, Donker T, Merry S, et al. Trial for the prevention of depression(TriPoD) in final-year secondary students: study protocol for a clusterrandomised controlled trial. Trials. 2015;16(6):451.
56. Robinson J, Hetrick S, Cox G, Bendall S, Yung A, Yuen H, Templer K, Pirkis J.The development of a randomised controlled trial testing the effects of anonline intervention among school students at risk of suicide. BMC Psychiatry.2014;14(5):155.
57. Simon G, Beck A, Rossom R, Richards J, Kirlin B, King D, Shulman L, LudmanE, Penfold R, Shortreed S, et al. Population-based outreach versus care asusual to prevent suicide attempt: study protocol for a randomized controlledtrial. Trials. 2016;17:452.
58. Stallard P, Porter J, Grist R. Safety, acceptability, and use of a smartphoneapp, BlueIce, for young people who self-harm: protocol for an open phase Itrial. JMIR Res Protoc. 2016;5:e217.
59. van Spijker B, Calear A, Batterham P, Mackinnon A, Gosling J, Kerkhof A,Solomon D, Christensen H. Reducing suicidal thoughts in the Australiangeneral population through web-based self-help: study protocol for arandomized controlled trial. Trials. 2015;16(8):62.
60. Joiner T, Pfaff J, Acres J. A brief screening tool for suicidal symptoms inadolescents and young adults in general health settings: reliability andvalidity data from the Australian national general practice youth suicideprevention project. Behav Res Ther. 2002;40:471–81.
61. Osman A, Bagge C, Gutierrez P, Konick L, Kopper B, Barrios F. The suicidalbehaviors Questionniare-revised (SBQ-R): validation with clinical andnonclinical samples. Assessment. 2001;8(4):443–54.
62. Reynolds W. Suicidal Ideation Questionnaire Junior. Odessa, FL:Psychological Assessment Resources; 1987.
63. Goldberg D, Hillier V. A scaled version of the general health questionnaire.Psychol Med. 1979;9(1):139–45.
64. Derogatis L, Lipman R, Rickels K, Uhlenhuth E, Cori L. The Hopkins symptomchecklist (HSCL): a self-report symptom inventory. Behav Schi. 1974;19(1):1–15.
65. van Spijker B, van Straten A, Kerkhof A. Online self-help for suicidalthoughts: 3-month follow-up results and participant evaluation. InternetInterventions. 2015;2:283–8.
66. Watts S, Turnell A, Kladnitski N, Newby J, Andrews G. Treatment-as-usual(TAU) is anything but usual: a meta-analysis of CBT versus TAU for anxietyand depression. J Affect Disord. 2015;175:152–67.
67. Wampold B, Budge S, Laska K, Del Re A, Baardseth T, Flükiger C, Minami T,Kivlighanm D, Gunn W. Evidence-based treatments for depression andanxiety versus treatment-as-usual: a meta-analysis of direct comparisons.Clin Psychol Rev. 2011;31:1304–12.
68. Mars B, Cornish R, Heron J, Boyd A, Crane C, Hawton K, Lewis G, Tilling K,Macleod J, Gunnell D. Using data linkage to investigate inconsistentreporting of self-harm and questionnaire non-response. Arch Suicide Res.2016;20:113–41.
69. NICE. Self-harm: the short-term physical and psychological managementand secondary prevention of self-harm in primary and secondary care.Leicester, UK: British Psychological Society; 2004.
70. Witt K. The use of emergency department-based psychological interventionsto reduce repetition of self-harm behaviour. Lancet Psychiatry. 2017;4:428–9.
71. Milner A, Spittal M, Kapur N, Witt K, Pirkis J, Carter G. Mechanisms of briefcontact interventions in clinical populations: a systematic review. BMCPsychiatry. 2016;16:194.
72. Daine K, Hawton K, Singaravelu V, Stewart A, Simkin S, Montgomery P. Thepower of the web: a systematic review of studies of the influence of theinternet on self-harm and suicide in young people. PLoS One. 2013;8:e77555.
73. Swinkles A, Giuliano T. The measurement and conceptualization of moodawareness: attention directed towards one's mood states. Personal SocPsychol Bull. 1995;21:934–49.
74. Hetrick S, Robinson J, Spittal M, Carter G. Effective psychological andpsychosocial approaches to reduce repetition of self-harm: a systematicreview, meta-analysis, and meta-regression. BMJ Open. 2016;6:e011024.
75. Larsen M, Nicholas J, Christensen H. A systematic assessment of smartphonetools for suicide prevention. PLoS One. 2016;11:e0152285.
Witt et al. BMC Psychiatry (2017) 17:297 Page 18 of 18