Full Terms & Conditions of access and use can be found athttp://www.tandfonline.com/action/journalInformation?journalCode=umid20

Download by: [Petra Boström] Date: 04 August 2016, At: 00:32

Journal of Mental Health Research in IntellectualDisabilities

ISSN: 1931-5864 (Print) 1931-5872 (Online) Journal homepage: http://www.tandfonline.com/loi/umid20

Subjective Mental Health, Peer Relations, Family,and School Environment in Adolescents withIntellectual Developmental Disorder: A FirstReport of a New Questionnaire Administered onTablets PCs

Petra Boström, Jakob Åsberg Johnels, Maria Thorson & Malin Broberg

To cite this article: Petra Boström, Jakob Åsberg Johnels, Maria Thorson & Malin Broberg(2016): Subjective Mental Health, Peer Relations, Family, and School Environment inAdolescents with Intellectual Developmental Disorder: A First Report of a New QuestionnaireAdministered on Tablets PCs, Journal of Mental Health Research in Intellectual Disabilities, DOI:10.1080/19315864.2016.1186254

To link to this article: http://dx.doi.org/10.1080/19315864.2016.1186254

Published online: 13 Jun 2016.

Submit your article to this journal

Article views: 8

View related articles

View Crossmark data

Subjective Mental Health, Peer Relations, Family, andSchool Environment in Adolescents with IntellectualDevelopmental Disorder: A First Report of a NewQuestionnaire Administered on Tablets PCsPetra Boström a, Jakob Åsberg Johnelsb, Maria Thorsona, and Malin Broberga

aDepartment of Psychology, University of Gothenburg, Sweden; bThe Sahlgrenska Academy at theUniversity of Gothenburg, Institute of Neuroscience and Physiology, Division of Speech and LanguagePathology, Sweden

ABSTRACTFew studies have explored the subjective mental health of ado-lescents with intellectual disabilities, while proxy ratings indicatean overrepresentation of mental health problems. The presentstudy reports on the design and an initial empirical evaluation ofthe Well-being in Special Education Questionnaire (WellSEQ).Questions, response scales, and an application for tablet PCswere developed in cooperation with students and teachers inspecial education schools. One hundred and thirteen students(age 12–16) and their parents and teachers participated. Positiveresults in terms of test-retest reliability, internal consistency ofscales, and response rates were obtained. Level of readingappeared to affect the students’ understanding of items.Teachers’ and parents’ ratings on the WellSEQ correlated wellwith established measures. Correlations between proxy ratingsand students’ reports varied. The questionnaire and technologyof WellSEQmay enable students with IDD to participate indepen-dently in research with good completion rates and reliableresponding.

KEYWORDSChild report; family;Intellectual disability; mentalhealth; mental ill-health;peer relations; proxy; school;self-report measures

As many as 30–50% of children and adolescents diagnosed with intellectualdevelopmental disorder (IDD) suffer from additional psychiatric conditions(Einfeld, Ellis, & Emerson, 2011; Emerson, 2003). Reports based on ratingsfrom parents, teachers, and health professionals indicate that children andadolescents with IDD suffer disproportionally from mental health problemsin general when compared with same-age peers (Einfeld et al., 2011).Externalizing as well as internalizing problems are more common amongchildren with IDD than without IDD (de Ruiter, Dekker, Verhulst, & Koot,2007). The most frequent comorbidities with IDD are autism spectrumdisorder (ASD), attention-deficit/hyperactivity disorder (ADHD), conductdisorders, anxiety disorder, and hyperkinesis (Emerson, 2003; Matson &

CONTACT Petra Boström petra.bostrom@psy.gu.se Department of Psychology, University of Gothenburg,Box 500, SE-405 30 Gothenburg, Sweden.Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/UMID.

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIEShttp://dx.doi.org/10.1080/19315864.2016.1186254

© 2016 Taylor & Francis

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Shoemaker, 2009). Despite these concerns, young people with IDD are seldomasked about their subjective experiences of mental health and the factors affect-ing health, as they tend to be excluded from large-scale surveys (Linehan,Walsh,Van Schrojenstein Lantman de Valk, Kerr, & Dawson, 2009). The present studyreports on the design and evaluation of the Well-being in Special EducationQuestionnaire (WellSEQ)—a 42-item questionnaire developed in coopera-tion with students and teachers in special schools for children with cognitivedisabilities. It is specifically tailored to measure self-rated mental health andill-health, peer relations, as well as school and family environment in ado-lescents with mild or moderate IDD, and is administered via an interactivetouch-based application for tablet PCs.

Challenges in Assessing Subjective Mental Health in Children withIDD

Mental health in children and adolescents with IDD has been assessedpreviously (Einfeld et al., 2011), but mostly through proxy ratings by parentsor teachers. The few attempts that have been made to explore the subjectiveexperiences of mental health problems and behavioral difficulties haveemployed methods requiring the presence and support of teachers orresearchers. Structured interviewing has turned out to be a reliable techniquefor collecting data from this group (Douma, Dekker, Verhulst, & Koot, 2006;Emerson, 2005), but may not be the preferred method of choice whencollecting data in large-scale surveys as it is highly time consuming.

Large-scale surveys on youth mental health are commonly administeredvia schools as paper questionnaires or web surveys. If at all included, childrenwith IDD struggle to complete questionnaires designed for same-age peers inregular education in terms of format and complexity of questions andresponse scales. Completion rates tend to be low, sometimes as low as 50%(Scott, Wishart, & Currie, 2011). One successful way to deal with thisproblem is for teachers to assist students with explanations and clarifications(Scott et al., 2011). However, support from school staff may jeopardize thestudents’ right to anonymity and affect their responses to sensitive questions.

Consequently, there is a need for survey formats enabling students withIDD to complete questionnaires independently and reliably. The complex-ity of questions and response scales needs to match the levels of readingability and word comprehension in order for children to fully comprehendquestions and be able to respond intentionally. Pitfalls to avoid whendesigning questionnaires for any population, such as difficult words, longstatements/questions, ambiguity of questions, too many items, and the useof complex response scales (Trost, 2012), are even more crucial to take intoaccount for students with IDD (Hartley & MacLean, 2006). When consid-ering various response scales, Hartley and MacLean (2006) reported that

2 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

various Likert-type scales have been used successfully for people (mostly adults)with borderline to mild IDD, preferably with 3–5 response alternatives. As theattention span may vary widely across individuals, the questionnaire should bekept as short as possible to avoid mistakes or early dropout due to attentiondeficits.

Requests to estimate general life quality over extended time periods, suchas the past 6 months, may require more cognitive processing than for shorterintervals, and appear to result in overly positive responses from persons withIDD, i.e., responses that do not correspond with objective assessments of thelife conditions and mental health of this group (Hartley & MacLean, 2006).For instance, when asked about their life satisfaction, 80% of students withIDD responded that they had the “best possible” life (Scott et al., 2011,p. 444), while at the same time these children are known to be overrepre-sented as victims of neglect and abuse (Sullivan & Knutson, 2000) andcommonly suffer from mental health problems. This positive response bias iscommon among individuals with lower cognitive ability (Hartley & MacLean,2006). The inconsistency between subjective ratings and proxy ratings mayreflect actual differences in perspective between a child with IDD and his/herparent or other proxy rater, or it may be due to errors of measurement. Theextent of agreement or disagreement between self-reports by children withIDD and proxy-ratings have, to our knowledge, not yet been explored.Recently, Mileviciute and Hartley (2015) examined indicators of depressiveproblems in adults with mild IDD according to self-reports and proxy ratings.Results showed that while there was some agreement in correlation analyses ondifferent subscales (r ranging between –.06 and .45), the adults with IDDreported a higher frequency of depressive thoughts and feelings than thecarer reported on the proxy questionnaire. The authors highlighted the impor-tance of including self-reports when examining mental health in persons withIDD as it provides important additional aspects which carers may not beaware of.

In quality of life (QoL) studies of children and adolescents without IDD,agreement between child and parent reports has varied. In Robitail et al.(2007, 2006), agreement increased with child age and was lower for boysthan girls. Parents and children agreed to a higher extent about factorssuch as the child’s physical well-being and school environment, but dis-agreed about moods and emotions, sense of autonomy, and experiences offamily relations. Yet in other studies of health-related QoL reviewed byUpton, Lawford, and Eiser (2008), the sources of variation in agreementbetween child and proxy ratings were somewhat inconclusive. In somestudies, agreement was higher for observable domains (e.g., physicalhealth) than for nonobservable ones such as emotional states, whileother studies found the opposite. A diagnosed physical illness appearedto increase agreement.

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 3

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

The aspects mentioned above constituted the general guidelines for theformat and presentation mode of the WellSEQ questionnaire for adolescentswith IDD. The content was guided by a conceptual framework (of mentalhealth and related factors; see below). To create and make decisions aboutspecific questions, response scales, and supportive features, participantsneeded to be involved as design partners. Previous studies have shown theadvantages of involving children with ID and autism in the process ofdesigning low-level technology (Frauenberger, Good, & Keay-Bright, 2011;Guha, Druin, & Fails, 2008). In the present project, students and teachingstaff were involved in a process of cooperative inquiry as co-designers of notonly the technological aspects (Boström & Eriksson, 2015) but also thequestions asked.

The Constructs of Mental Health and Ill-Health

Psychological health research may be construed as consisting of three mainareas: mental health/well-being, mental health problems, and psychiatricconditions (Bremberg & Dalman, 2015). The distinctions between thesedomains are not clear-cut and may vary across theories and have changedover time. Researchers and practitioners have traditionally focused mainly onmental health problems, while positive mental health/well-being has oftenbeen considered as the absence of mental health problems (Westerhof &Keyes, 2010). Within this framework, mental health and ill-health have beenconsidered as two end points of an axis that describes the severity andfrequency of emotional and behavioral problems (Keyes, 2007), while psy-chiatric conditions are categorically defined (Bremberg & Dalman, 2015).Recent empirical findings regarding the developmental trajectories of mentalhealth and mental health problems across the lifespan suggest that these areseparate but related constructs (Keyes, 2007; Westerhof & Keyes, 2010). Ayoung person may be in the clinical range for behavioral problems and at thesame time experience positive mental health—although it is of course morecommon that persons with severe mental health problems also report lowpositive mental health. In adult life, positive mental health appears to be animportant predictor of the ability to function in society in the generalpopulation. Positive mental health distinguishes between persons withsame-level mental health problems in terms of better functioning, which isan important part of general well-being (Westerhof & Keyes, 2010).

What is to be included in the concepts of mental health and ill-health hasalso been debated. Mental health and ill-health may be considered to includea wide range of aspects from within the individual (e.g., emotional states andself-concept) to relational and social factors (e.g., experiences of peer rela-tions, family functioning, and the school environment), as is often the case inQoL measures (e.g., Ravens-Sieberer, 2005). Alternatively, the concepts of

4 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

mental health and ill-health may be narrowly defined and include onlyemotional and behavioral factors within the individual (Eriksson, Cater,Andershed, & Andershed, 2011).

Several instruments designed to assess mental health in young people, suchas the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001) andthe Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997), focusmainly on emotional and behavioral problems of the young person. Thereare also measures designed to assess positive and negative aspects within QoLresearch, including Kidscreen (Ravens-Sieberer et al., 2005). Both the CBCLand the SDQ have been used to assess mental health problems in childrenwith IDD. They seem to work well when administered as proxy ratings (DeRuiter et al., 2007) or as child interviews (Emerson, 2005), while the self-report paper versions may be difficult for many children with IDD tocomplete due to a complex layout or a large number of questions included.

Although it would have been desirable to use an established questionnaire inthe present study, and make necessary adjustments for children with IDD, thiswas not possible due to copyright regulations. Consequently, a new question-naire, tailored specifically to students in special education, was developed. It wasbased on the view that mental health and mental ill-health are separate butrelated constructs. The definitions of mental health and ill-health were restrictedto include only aspects within the individual such as emotional states, self-acceptance/appreciation, and general sense of well-being. Relational factorslocated between the individual and his/her environment, or environmentalfactors, were considered as separate from, and possibly affecting, mental healthand ill-health.

Factors Affecting the Mental Health and Ill-Health of Young People withIDD

In a model describing universal risk factors affecting child and adolescentmental health at various stages of development, Kieling et al. (2011) empha-sized family, peer, and school problems as particularly important duringadolescence. In addition to measuring mental health and ill-health, thequestionnaire developed in the present study aimed to measure proximalrelational and environmental factors that have been described as importantby adolescents with IDD. Peer relations affect well-being (Foley et al., 2012),and adolescents with IDD are known to be less likely to develop and sustainclose friendships with same-age peers (Guralnick, 2006; Matheson, Olsen, &Weisner, 2007) and are at heightened risk of being victims of bullying(Christensen, Fraynt, Neece, & Baker, 2012).

While typically developing adolescents rapidly extend their range of free-dom during the teenage years, adolescents with IDD tend to remain depen-dent on the family to a higher degree (Cuckle & Wilson, 2002). For

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 5

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

adolescents with IDD, the family appears to be important both for sharingpositive experiences and for providing support (Foley et al., 2012). School isanother area of importance. Adolescents spend a lot of time in school, and ifschool is experienced as a positive environment that provides meaningfultasks and activities with challenges adjusted for the individual student, it maycontribute to positive mental health.

The Current Study

As argued above, there is a need to develop questionnaires that young peoplewith IDD can complete independently and reliably. The present study aimedto design a questionnaire in cooperation with students and teachers in specialeducation that measures students’ mental health, mental ill-health, peerrelations/bullying, family environment, and school environment in an acces-sible and reliable way.

To enhance response rates and quality of responses, the study aimed toincorporate knowledge from previous studies in designing an applicationfor tablet PCs, which constituted the format of presentation. This enabledfunctions such as a standardized introduction and clarifications of items,audio and visual support for students with reading difficulties, an adjus-table pace for the flow of questions, and the presentation of one questionat a time.

The present study reports on the work to: (a) design a questionnaire withitems regarding mental health, mental ill-health, peer relations/bullying,family, and school environment to which adolescents in special educationclasses can respond independently with high completion rates; (b) initiallyexplore the reliability of five scales (questions and response scales) in termsof internal consistency, test-retest reliability, and response bias; and (c)explore the level of agreement between students’ ratings and proxy ratingsfrom teachers and parents.

Methods

Participants

Eighteen schools in urban areas (mid- or small-sized cities/towns) in VästraGötaland County, Sweden participated in the project. A total of 190 studentsin special education classes (age 12–16 years, grades 6–9) were informedabout the research project, and 113 (59%; range 23–78%) of them acceptedthe invitation to participate (see Table 1 for demographic data). A majority ofthe participating students were boys (71%)—a gender distribution represen-tative of the population (e.g. Emerson, 2003). The only criterion for exclusionwas a response rate below 80%.

6 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Sweden has 9 years of compulsory schooling. Most children begin school inAugust of the year they turn 7 years old. In case a child has IDD, so-called specialneeds comprehensive school (grundsärskolan in Swedish) is an option. Thisseparate form of schooling has its own curriculum and aims to provide educa-tion adapted to each student’s specific needs, and should, in addition to dis-seminating knowledge, also contribute to the students’ personal development,independence, and sense of belonging in the community (Skolverket, 2015).

Table 1. Demographic Data: The Grades the Participants Are in, Their Gender, and Their Parent orGuardian’s Educational Level.Pupils (n = 113) %

Grade 1

6 23.37 25.28 25.29 26.2Mean age (years) 14.3% males 70.8

Additional diagnosis2

Only IDD 13.3ASD 27.4ADHD 24.8Down syndrome 7.1Dyslexia, language disorder 5.3Other syndrome 8.8Illness, injury 11.5Physical disability 12.4

Reading3

Below grade 1 24.8Grade 1 11.9Grade 2 30.3Grade 3 16.5Grade 4 and above 16.4

Living situation4

With both parents 68.7With one single parent 14.3Alternating between parents 11.9Foster care 4.5

Birth country4

Sweden 87.8EU country 6.7Non-EU country 5.6

Language spoken at home4

Swedish 80.4Swedish and other 6.5Other 13.0

Parent or guardian participant (n = 67) 4

Mother 55.2Father 13.4Both parents 26.9

Parental education4

No university studies 48.5University studies 51.5

Note. 1 Grade as reported by teachers. 2 Reported by the parents in the consent form (n = 92). 3Score on LäSttransformed into school grade. 4Reported by the parents in the parent questionnaire (n = 67).

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 7

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

An IDD diagnosis is required for special needs comprehensive schooling.Previously, an autism spectrum diagnosis was also sufficient, but as of 2011this is not the case unless the child has co-occurring IDD. The data for thepresent study was collected after 2011, but some schools may still have beenin the process of transferring some students with autism but no IDD to theordinary school system. Collection of data on the WISC-IV Vocabularysubtest (Wechsler, 2003) confirmed that the sample was performing at alow level in verbal ability, with a mean score of 1.77 (range 1 to 9) on theage-converted scale score (M = 10, SD = 3). Unfortunately, time constraintsat the schools made it impossible to administer all subtests of the WISC-IV.

Comorbidity was common in the sample as parents reported that 87% ofthe students had between one and five additional diagnoses, as would beexpected in a group of students in special education. In addition to IDD, 27%were reported to have an autism spectrum disorder, 25% attention-deficit/hyperactivity disorder, 7% Down syndrome, 5% a language disorder, and12% a physical disability (see Table 1).

A random selection of 47 students also participated in the retest datacollection. The retest sample did not differ significantly from the others interms of age, WISC vocabulary, or reading ability (all t < .9, all p > .40), orgender χ2 (1) = 0.59, p = .809.

The parents and teachers of all participant students were asked to com-plete a web-administered questionnaire. Out of the 67 participating parents,about half had university-level education. A majority of these parents (69%)lived together with the child’s other birth parent (Table 1). Parents reportedthat 88% of the children were born in Sweden, but the parents of thesechildren were not necessarily Swedish natives. It is likely that the representa-tion of children born in other countries was higher in the group of childrenwhose parents did not respond as all information was in Swedish.

Of the 27 school staff from the 18 participating schools, 12 were specialeducation teachers, 14 were teachers, and 1 was an assistant teacher. Theycompleted a web-administered questionnaire, rating a total of 97 students.Teachers choose to complete the rating of each student individually or as ateam. This meant that one or several teachers rated multiple students, whichwas not controlled for in the study.

Procedure

Development of the WellSEQThe WellSEQ questionnaire was developed through literature reviews, expertconsultations, and classroom workshops with students in grades 6–9 in specialeducation through a process of cooperative inquiry (Frauenberger et al., 2011).The aimwas to tailor items, response alternatives, and the format of presentationto students in special education. The development of the questionnaire was an

8 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

iterative process of close cooperation between the research team, a master’sstudent in interaction design, and students and teachers in a special educationschool. A draft of fifty items with response scales was created. During fiveworkshops, a total of 10 students aged 12–16 participated. One or two studentswere absent at each workshop due to illness or compulsory activities. Thestudents were presented with different parts of the draft. Statements such as “Iam often happy,” “I am grumpy and irritable,” and “In my family we have funtogether” were presented to the students, who were then asked to draw picturesand select photographs, drawings, or cartoons from a collection that illustratedthe statements. The students were also asked to reflect on whether there wereparticular colors (such as green-yellow-red) or symbols (e. g. smiley faces orthumbs up or down) that were representative of the response scale. Items andthe response scale were adjusted according to student and teacher reflections. Asopinions were not always univocal, the research team interpreted and weightedthe different aspects presented and made adjustments to present a new versionin the following workshop.

Data CollectionPrincipals from 18 of 33 contacted schools chose to participate, six declined,and nine never responded. Teachers and students were informed about theresearch process through letters and a visit to the school. Teachers signed aconsent form of participation stating that participation was voluntary, that allinformation would be treated confidentially, that participants could withdrawfrom the project at any time, and that teachers who participated wouldreceive a symbolic gift (€10 value). Two letters were distributed to eachfamily, one specifically designed for the student. The letters contained infor-mation about participation and a consent form with the same information asfor the teachers. Students and parents were informed they would each receivea gift voucher for a movie ticket following participation.

A researcher collected data from one student at a time at the school via aniPad. The student was asked to wear headphones and listen to the instructionsbefore answering the questions. The researcher was present during data collec-tion to answer any questions but refrained from interfering with the student’sresponding. Completion of the WellSEQ was followed by a short reading test(LäSt) and a word comprehension test (the WISC Vocabulary subtest). Thetotal testing time was 18–45 minutes.

Retest data was collected three weeks after the first data collection follow-ing the same procedure, but only using the WellSEQ. The procedure wasapproved by the ethical review board at the University of Gothenburg.

Proxy ratings from parents and teachers were collected via a web survey ata time and place of their own choice around the time of data collection in theschool. Parents and teachers who had agreed to participate were remindedtwice (if needed) via e-mails.

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 9

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Measures

WellSEQ—Well-being in Special Education QuestionnaireThe final version and characteristics of the WellSEQ are presented in the resultssection (Table 2). The child version ofWellSEQ included 42 items regarding thechild’s mental health/ill-health, peer relations/bullying, home, and school envir-onment. Students were asked to consider their experiences during the past weekand answer on a 3-point scale (“yes,” “sometimes,” and “no”). The neutral

Table 2. Content of the WellSEQ Scales (Including all Items).WellSEQ Scale Items

Mental health Are you usually happy?Do you feel happy with life?Do you like how you look?Are you happy to be you?Do you usually have a lot of energy?Do you feel healthy and fit?

Mental ill-health Are you usually worried?Do you often break things?Are you usually grumpy and annoyed?Do you get angry easily?Do you feel like everything you do goes wrong?Do you think most things are boring?Do you often feel sad?Do you feel stressed?Do you often have a stomach ache?Do you often get headaches?Do you find it hard to sleep?Do you easily forget what you learn?Is it hard for you to sit still and concentrate at school?

Family relations Do you feel safe with mum?Do you feel safe with dad?Are there a lot of arguments at home?Do you have fun together at home?Do you prefer to be away from home?Does your family have too little money?

School environment Do you often stay home from school?Do you like being at school?Is school going well for you?Are you happy with your teachers?Do your teachers support you?Do your teachers listen to you?

Peer relations and conflict Do you have a really good friend?*Is it hard for you to make friends?Do people seem to like you?Do you feel lonely?Do your friends support you when you need it?Are you afraid someone will be mean to you at school?Do people bully you?Is it easy for you to make friends?Do you often end up in fights?Are you a good friend?Do you tease people?

Note. The response scale consisted of “yes,” ”sometimes,” and ”no.” * Response scale for this item consistedof only “yes” and “no.”

10 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

format of the response scale without symbols or colors was a result of thecooperative inquiry process and was considered the most suitable alternative,since it could be used meaningfully with both positively and negatively wordeditems. The questionnaire was administered via an application for tablet PCs(iPads) and opened with verbal instructions (Boström and Eriksson, 2015). Eachquestion and the response alternatives were presented in text and read aloud inthe voice of an adolescent girl. The questions were also supplemented with aphotograph to further clarify the question (Figure 1).

Two versions of WellSEQ for proxy ratings were developed as part of thepresent research for the sole purpose of directly exploring agreement betweenself-reports and proxy ratings on the same items. The questionnaire wasdistributed online. The parent version of WellSEQ contained 34 items, andthe teacher version of WellSEQ 31 items regarding the child’s mental health/ill-health, peer relations/bullying, and school environment. For example, theitem “Do you feel lonely?” was reworded “Does your child/student feellonely?” for the parent and teacher versions. Some items were excludedfrom the proxy versions that directly involved the relation with the proxyrater himself/herself (e.g., “Do you feel safe with dad/mom?”) or items thatthe proxy rater most likely had limited information about, such as teachersrating the child’s perception of his/her family climate. The items excludedfrom the proxy versions were also excluded from the child version ofWellSEQ when analyzing agreement.

Figure 1. Screenshot of the application for tablet PCs. Note. Translation of item and responsescale: Do you feel lonely? No, Sometimes, Yes.

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 11

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Means were calculated for each scale after negative items were reversed.Higher numbers (approaching 3) indicated good mental health and positiveexperiences while a low rating (toward 1) indicated low mental health andnegative experiences.

Reading SkillsThe word decoding part of the LäSt test (Elwér, Fridolfsson, Samuelsson, &Wiklund, 2009) with a total of 200 out of context words was used as ameasure of reading ability. During two 45-second periods, the student wasinstructed to read aloud as many words as possible from a list starting witheasy two-letter words and leading to more advanced multi-syllable wordsfurther down the list. The number of correctly decoded words in each roundwas calculated. For descriptive and analytical purposes, raw total scores werethen converted to school grade equivalents. A score of 40 words or belowindicates preschool reading ability, a score of 41–118 corresponds to a read-ing ability of grades 1–3, and a score of 119 or above is considered a sign offluent reading skills (grade 4 or above).

Strengths and Difficulties QuestionnaireAgreements between the parent and teacher versions of the Strengths andDifficulties Questionnaire (SDQ; Goodman, 1997) and the WellSEQ wereassessed. These instruments could be expected to overlap partly as theyexplore similar constructs, and could consequently be correlated with the purposeof getting an initial view of the construct validity. Therefore, we expectedmoderatecorrelations between supposedly related scales. The SDQ contains 25 itemsmeasuring five dimensions: emotional symptoms, conduct problems, hyperactiv-ity, peer problems, and pro-social behavior. A total problem scale including thefour problem scales (emotional symptoms, conduct problems, hyperactivity, peerproblems) was used as an overall measure of proxy-rated child problems. Parentsand teachers were instructed to consider their child’s behavior in the past 6months when answering the questions on a 3-point Likert-type scale rangingfrom zero (not true) to 2 (certainly true). Cronbach’s alpha for the parent-ratedSDQ scales ranged from 0.72 to 0.85, and from 0.61 to 0.88 for teacher ratings.

Analysis

Scale DevelopmentAt this stage of scale development the process was primarily based on theoreticalgrounds. An iterative process oscillating between a theoretically based construc-tion of scales and a test of the fit in terms of internal consistency (Cronbach’salpha) of each scale was carried out. This process commenced from the theore-tical standpoints that mental health and ill-health are related but separateconstructs, and that peer relations/conflict, family environment, and school

12 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

environment may be considered independent factors that may affect mentalhealth and ill-health. The factors are interrelated and may overlap to someextent, but most questions fit into and construct the core element of each scale(e.g., “Do your teachers support you?” belonged to the school-environment scaleand “Do you have fun together at home?” was part of family environment).A few questions related to more than one scale (e.g., “Do you often stay homefrom school?” which could be connected to both mental ill-health or beingunhappy with school). In cases where theoretical guidance was ambiguous, thechoice was based on the best fit in terms of internal consistency. The decision toinclude questions about both peer relations and elements of conflict and bullyingin one scale instead of two, and to have separate scales for mental health andill-health instead of one health/ill-health continuum, was also supported byimproved internal consistency.

Response BiasA positive response bias was defined as students responding with the mostpositive response alternative to all questions. Another way to detect responsebias was by analyzing the consistency of individual students’ responses to thereverse-worded questions “Is it hard for you to make friends?” and “Is it easyfor you to make friends?” Answering “Yes” or “No” to both questions wasconsidered inconsistent. We explored the frequency of such biased respond-ing and whether it was related to level of reading skills.

Test-Retest Reliability and Student-Proxy AgreementTest-retest correlations were explored using the non-parametric Spearman’scorrelation coefficient (rs) as a measure of reliability (since the data had anon-normal distribution). Correlations between students’ and parent/teacherWellSEQ ratings, and between the parent versions of the WellSEQ/SDQ andthe teacher version of the SDQ, were also explored with Spearman’s rho.

Due to the positively skewed distribution of data, alternative methods fordescribing interrater agreement were employed. The proportion of itemagreement for each student-parent and each student-teacher couple wascalculated. The results are presented as mean percentages of absolute agree-ment, ±1 step disagreement, and ±2 steps disagreement for the group on eachscale. The expected distribution of agreement resulting from random ratingusing the 3-point scale would be 33% agreement, 44% ± 1 step disagreement,and 22% ± 2 steps disagreement, which can be considered as a point ofreference for understanding level of agreement, but not as a numericalindicator of good agreement.

Further analyses of interrater agreement were made using weighted Kappastatistics (Fleiss & Cohen, 1973). Squared differences were used as disagree-ment weights as the response scale was an ordinal scale. Agreement in each

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 13

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

student-teacher and each student-parent couple was explored analyzing rawdata for the items within each scale.

To explore differences in ratings between the students and the parents andteachers, respectively, paired sample t-tests were performed. The significancelevel was set at p < .05.

Results

WellSEQ—A Questionnaire Designed for Adolescents in Special EducationClasses

Out of the 113 participating students, 109 completed the WellSEQ with nomissing responses and 111 completed more than 80% of the questions oneach scale. The two students who had a response rate below 80% wereexcluded from further analyses. All students completed the questionnaireindependently except for two, who needed assistance from the researcher topress the screen due to motor impairments. The time it took for students tocomplete the 43 questions varied from 4 minutes and 18 seconds to 17minutes and 49 seconds, with a median of 7 minutes and 40 seconds.

A high completion rate and a relatively short response time could beconsidered as indicators of a well-adapted questionnaire and presentationformat for the target group. However, the quality of responses also needed tobe explored. The pattern of fingertip touches on the screen was recordedautomatically in the application and evaluated qualitatively in terms ofintentionality of responses (e.g., whether the student had pressed theresponse buttons intentionally and not just pressed the screen randomly)and use of the full response scale (as opposed to repeated pressing ofone response alternative). Visual inspection of this data showed that afew students with motor disabilities had occasionally touched nonfunc-tional areas on the screen, but not to an extent that indicated randomtouching. No participant was excluded due to unintentional or one-sidedresponding.

Reliability of the WellSEQ Scales

Internal Consistency and Test-Retest ReliabilityForty-two questions were finally assigned to the five scales: Mental Ill-Health,Mental Health, Peer Relations and Conflict, School Environment, and FamilyEnvironment (Table 2) with levels of Cronbach’s alpha ranging from 0.56 to0.85 in the first data collection (Table 2). In the retest data collection,consisting of a random selection of 47% of the original sample, internalconsistency was generally improved and ranged from 0.60 to 0.88. Thisimprovement could be an effect of students learning to use the WellSEQ

14 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

application. The most dramatic improvement in terms of internal consis-tency (from 0.56 at test to 0.78 at retest) was found for the scale PeerRelations and Conflict, which consisted of the questions presented at thebeginning of the questionnaire. One question was excluded as several stu-dents asked what it meant during data collection.

There appeared to be good correspondence between the test and retestcollections as correlations (Spearman’s rs) were moderate to strong(Table 3) and no significant differences between test and retest meanswere found for any of the scales (Table 2).

Response BiasData were analyzed in terms of tendency to respond with the most positiveresponse alternative. Among the 111 participants, 2.7% had chosen the mostpositive alternative to all questions.

Table 3. Means, Standard Deviations and Internal Consistency of the WellSEQ completed byPupils, Parents, and Teachers along with Paired t-Tests between Parents-Pupils and Teacher-Pupils.Scale, (n), test/retest (T1/T2) Mean (SD) α T1/T2 t Parent-Pupil t Teacher-Pupil t

Mental healthTotal sample (111), T1 2.76 (0.33) .69Retest sample (47), T1 2.79 (0.26)T2 2.79 (0.32) .71 −0.019Parent (65–67) 2.69 (0.39) .829Teacher (81–93) 2.60 (0.41) 4.149**

Mental ill-healthTotal sample (111), T1 2.35 (0.42) .85Retest sample (47), T1 2.30 (0.39)T2 2.27 (0.50) .88 0.615Parent (65–67) 2.38 (0.36) −1.189Teacher (81–93) 2.52 (0.29) −2.884*

Peer relations and conflictTotal sample (111), T1 2.64 (0.30) .56Retest sample (47), T1 2.65 (0.25)T2 2.58 (0.37) .78 1.836Parent (65–67) 2.30 (0.38) 6.389**Teacher (81–93) 2.52 (0.41) 4.249**

School environmentTotal sample (111), T1 2.68 (0.42) .74Retest sample (47), T1 2.71 (0.33)T2 2.76 (0.35) .67 −1.031Parent (65–67) 2.69 (0.42) .028Teacher (81–93) 2.85 (0.26) 1.123

Family environmentTotal sample (111), T1 2.68 (0.34) .56Retest sample (47), T1 2.64 (0.35)T2 2.66 (0.35) .60 −0.596

Note. Higher mean scores indicate more positive responses.*p ≤ .01. **p ≤ .001 (2-tailed).

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 15

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Most participants responded consistently to the pair of reverse-wordedquestions “Is it hard for you to make friends?” and “Is it easy for you to makefriends?” except for 18 of the participants who responded in a contradictoryway with “Yes” or “No” to both questions. When examining these responsesin relation to the respondents’ reading ability, a connection between imma-ture reading and responding inconsistently was found. None of the partici-pants with a fourth-grade reading ability or higher (measured with LäSt)responded in a contradictory manner to the reverse-worded questions. Tenpercent of the students with a third-grade reading ability, 15% with a second-grade reading ability, 23% with a first-grade reading ability, and 30% rated aspre-reading made the same mistake. In the retest data collection, there was amarked reduction in inconsistency and only 50% of the ones who hadresponded inconsistently repeated the mistake. Reading ability was notassessed at retest but it is considered unlikely that skill levels had improvedin the short period that had passed.

Interrater Agreement

To investigate the agreement between students and proxy measures, severalmeasures were used. As an overall index of agreement between students andproxy raters, all items for each student-proxy pair were correlated, separately forparents and teachers. The students’ and parents’ ratings correlated significantly(rs = 420, p < .001), as did the students-teacher ratings (rs = 547, p < .001).

On subscale level, teachers’ and students’ ratings were significantly corre-lated on the Mental Health scale (rs = .266, Table 3), although teachers (M =2.60, SD = 0.41) rated students (M = 2.79, SD = 0.29, t (92) = 4.14, p < .001, [allpairwise comparisons are reported in Table 2]) lower. Student-parent ratingsof mental health were not correlated, but there were no significant meandifference between the students’ and parents’ ratings. The average levels ofitem agreement were similar for students-parents and students-teachers (54%vs. 56%, Table 3). The expected distribution of agreement resulting fromrandom rating using the 3-point scale would be 33% agreement, 44% ± 1step disagreement, and 22% ± 2 steps disagreement. (This is obviously a verylow threshold, and should only be considered as a point of reference).

Students’ ratings of Mental Ill-Health did not correlate significantly withthe teachers’ or the parents’ scores. Teachers reported more positive ratings(M = 2.51, SD = 0.29) than students (M = 2.38, SD = 0.36, t (92) = −2.88, p =005) on Mental Ill-Health, while the absolute agreement was 49%. On thisscale, the item agreement between students and parents was 47% and themeans did not differ significantly.

Further explorations of agreement using weighted Kappa (KW) resulted inlow levels of student-parent and student-teacher agreement for both MentalHealth and Mental Ill-Health (0.10–0.17, Table 3). The low levels were

16 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

related to the positive skew of results and the assignment of positive items toMental Health and negative items to Mental Ill-Health. When calculatingweighted Kappa for the combined Mental Health and Ill-Health includingboth positive and negative items, agreement was found to be moderate(student-parent KW = 0.52, student-teacher KW = 0.48).

Students rated their peer relations/conflict in significantly more positiveterms (M = 2.60, SD = 0.31) than their parents (M = 2.26, SD = 0.40, t (66) =6.39, p ≤ .001) and teachers (M = 2.48, SD = 0.44, t (93) = 4.25, p ≤ .001), andthe average proportion of absolute item agreement for student-parent andstudent-teacher dyads were 46% to 53%, respectively. Weighted Kappapointed in a similar direction with higher agreement for students-teachers(KW = 0.51) than for students-parents (KW = 0.35).

The highest agreement was found for School Environment. Students’perceptions of their school environment were significantly correlated withtheir parents’ (rs = .420, p ≤ .05) and teachers’ (rs = .212, p ≤ .05) ratings(Table 3). Absolute item agreement was also relatively high, as each student-parent and student-teacher on average agreed on 66% and 79% of the items,respectively, and weighted Kappa was 0.30 for students-parents and 0.73 forstudents-teachers. There were no significant mean differences between stu-dent and proxy ratings on School Environment.

Parent and Teacher Versions of the WellSEQ: Associations with the SDQ

There were significant correlations between parent- and teacher-ratedWellSEQ scales and the SDQ (Goodman, 1997) with various strengthsdepending on the subscale (Table 4). The WellSEQ Mental Ill-Health scaleinter-correlated with the Total Difficulties, Emotional Symptoms, andConduct Problems scales of the SDQ for both parents and teachers. There was

Table 4. Parent-Pupil and Teacher-Pupil Agreement of the WellSEQ Scales.Child T2 Parent proxy Teacher proxy

Child T1 rs rs % Agreement1 KW rs % Agreement1 KWMental health .690** .175 56/30/14 .099 .266* 54/33/13 .130Mental ill-health .696** .061 47/42/11 .167 .075 49/40/11 .115Peer relations and conflict .710** .056 46/41/13 .353 .129 53/38/9 .514School environment .487** .420** 66/25/9 .301 .212* 79/15/6 .730Family environment .683**Parent proxyMental health .260*Mental ill-health .401**Peer relations and conflict .329*School environment .466**

1Mean percentage of items with agreement/±1 step disagreement/±2 steps disagreement. A randomdistribution of 33%/44%/22% would be expected by chance.

*p ≤ .05 (2-tailed); **p ≤ .01 (2-tailed).

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 17

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

also some correspondence between the WellSEQ’s Mental Ill-Health and SDQ’sHyperactivity for parents. The WellSEQ’s Mental Health scale was correlatedwith SDQ’s Total Difficulties, Emotional Symptoms, and Conduct Problemsscales at a somewhat lower level than for Mental Ill-Health. The WellSEQ’s PeerRelations and Conflict and SDQ’s Peer Problems and Pro-Social Behavior scaleswere inter-correlated, indicating a high resemblance between the SDQ’s PeerProblems and the WellSEQ’s Peer Relations and Conflict.

Discussion

The aim of the present study was to design a questionnaire in cooperationwith students and teachers in special education schools with items regardingmental health/ill-health, peer relations/bullying, family environment, andschool environment in an accessible and reliable way. Specifically, an appli-cation for tablet PCs was developed which enabled good completion ratesfrom 111 students with IDD. The levels of test-retest correspondence andinternal consistency indicated that the WellSEQ is a reliable measure. Therewere meaningful associations between the students’ reading skills and theirability to respond to the questions in a coherent manner. Explorations ofagreement between students’ and teachers’/parents’ ratings varied betweenscales and modes of analyses. These initial findings are promising since thereis a lack of self-report measures of psychological and mental health problemsadapted for students with IDD. Still, there are issues that need furtherconsideration and exploration in future studies, including determination ofthe construct validity and the structure of subscales. Hence, the WellSEQshould be considered a work in progress. Implications, limitations and futuredirections are discussed below.

Table 5. Convergence between the WellSEQ and the SDQ.SDQ WellSEQ

Parent Mental health rs Mental ill-health rs Peer-relations and conflict rsTotal difficulties −.546** −.686**Emotional problems −.571** −.681**Conduct problems −.450** −.517**Hyperactivity −.251* −.473**Peer problems −.782**Pro-social .465**TeacherTotal difficulties −.438** −.581**Emotional P\problems −.214* −.505**Conduct problems −.487** −.480**Hyperactivity −.086 −.298**Peer problems −.789**Pro-social .438**

*p ≤ .05 (2-tailed); **p ≤ .01 (2-tailed).

18 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Children with IDD often find it difficult to respond to conventional paper-and-pencil questionnaires, which may result in low completion rates anduncertainties in terms of reliability (Finlay & Lyons, 2001; Hartley &MacLean, 2006). To resolve some of these problems in the current study,students were involved in the design of the questionnaire and the presenta-tion format on the iPad. The application commenced with a short introduc-tion of its different functions. One question at a time was presented, and apicture and a voice recording complemented the written question andresponse scale to enhance understanding. The completion rates for theWellSEQ were high—at levels comparable to questionnaires administeredas structured interviews (e.g., Douma et al., 2006; Emerson, 2005). Thestudents were able to complete the questionnaire with no or little supportfrom the research team, and all participants appeared to respond intention-ally. This means that, in future projects, data could be collected via theWellSEQ questionnaire and application from students in regular classroomsettings.

Confirmation of reliable responding is a common challenge in the devel-opment of new research tools. In the present study, internal consistency, test-retest agreement, and response bias were aspects of reliability that wereconsidered. Forty-two questions were assigned to five scales intended tomeasure subjective experiences of mental health and ill-health, as well asrelational factors concerning peers, the school environment, and family life.The indexing process relied mostly on theoretical constructs and the ques-tions aimed to capture main aspects of the constructs. This deductive processwas combined with inductive elements emphasizing sensitivity to data in anattempt to find the scales with the best fit for the participant group.

In the first data collection, three out of five scales (Mental Health,Mental Ill-Health, and School Environment) reached a level of internalconsistency considered to be good (α ≤ 0.7). In the retest collection,internal consistency was further improved, particularly for Peer Relationsand Conflict. The standard level of Cronbach’s alpha is a rule of thumbthat needs to be considered in context. The WellSEQ’s 3-point responsescale was designed based on what the students themselves consideredcomprehensible, and the aim was to capture five broad areas of mentalhealth and relational factors. These scale characteristics may have contrib-uted to lower Cronbach’s alpha.

Other sources of variance that need to be considered are the individualdifferences in how constructs are represented and experienced subjectively.Students in special education constitute a heterogeneous group of individualswho may vary extensively in terms of, e.g., developmental level and socialand cognitive skills. There may be differences in how these students perceivea construct within the group, as well as compared with adolescents withoutdiagnosis.

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 19

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Test-retest reliability appeared to be good. Relatively good stability wasfound despite the fact that the students were asked about their present statesand experiences as opposed to rating their health over the past 6 months. Theimprovement of internal consistency found for Peer Relations and Conflict atretest could possibly be a reflection of the fact that these items were pre-sented at the beginning of the questionnaire. Students may need time tograsp the questionnaire format, and they consequently gave relatively morebiased and contradictory responses in the beginning. Answering the reverse-worded questions inconsistently (items 2 and 8) was related to low readingability—a tendency that also decreased at retest. A case could be made forexcluding these participants from our analyses, arguing that they may be toolow functioning to partake in self-reports of mental health altogether.However, we chose not to exclude anyone at this stage, since there weresome indications of improvement with practice. This, rather, indicates thatstudents with mild to moderate IDD, and perhaps particularly weak readers,might benefit from additional practicing on sample questions. Future studiesusing tablet PCs could also enable interactive functions such as providingfeedback on students’ response patterns, which could further support learn-ing of the questionnaire format.

The phenomenon of positive response bias has been detected in a number ofstudies involving persons with IDD (Hartley &MacLean, 2006). In these studies,between 0–50% of participants choose the most positive response alternative toall items of a questionnaire. In the present study, only 2.7% gave exclusivelypositive responses. However, there was a positively skewed distribution on allscales, which may reflect a positive response bias also in our study group. It isimportant to note that the same positive distribution was found in the parents’and teachers’ reports, and that positively skewed distributions are commonwhen measuring mental health and relational factors in school-based samples(e.g. Robitail et al., 2006), probably because most students feel healthy.

Proxy ratings are often used as a substitute for children’s own reportsof subjective experiences and it is therefore important to evaluate theagreement between these. The level of agreement tends to vary acrossstudies (Upton et al., 2008). In the present study, correlations betweenstudents and their parents/teachers were generally low on subscale level,while analysis of agreement and Kappa indicated higher agreement onthe School Environment, Peer Relations and Conflict, and the combinedmental health/ill-health items. This discrepancy between results couldpotentially be explained by the fact that analyses of correlations areaffected by restriction of range, while calculations of Kappa and agree-ment are less sensitive to the distribution of data. The fact that studentsand proxy raters generally differ in their reports could be explained bytheir differences in perspective. As highlighted by Mileviciute andHartley (2015), self-reports of persons with IDD could add unique

20 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

information about the internalized experiences that carers may not beaware of. This line of reasoning may apply to the present study as well,with the argument that students can provide important additional infor-mation about their mental health. As of yet, however, we still do notknow whether the marginal associations between students’ ratings andproxy ratings were related to flaws of the instrument or differences inperspective and experiences.

Future research is needed to determine the validity of the self-reportWellSEQ, which has not been formally assessed here. Still, the currentstudy provides some preliminary data on the validity of the items used inthe proxy ratings. Parents’ and teachers’ ratings on relevant parts from theWellSEQ were compared with their ratings on the SDQ—a well-establishedinstrument used worldwide. Results showed that the Mental Health andMental Ill-Health scales of the WellSEQ were related to the problem scalesof the SDQ. The WellSEQ contains more positive items than the SDQ, whichmay account for some of the divergence. The WellSEQ’s Peer Relations andConflict scale was more strongly related to the SDQ’s Peer Problems than itsPro-Social Behavior scale. An explanation for this could be that the SDQ’sPro-Social Behavior scale mainly focuses on the child’s “friendly behavior,”while the WellSEQ’s Peer Relations and Conflict scale has its main focus onexperiences of friendship and peer problems.

Limitations, Future Directions, and Conclusions

The reliability of the WellSEQ was relatively well explored using severalmethods, while validity needs further exploration. Further methods ofexploring construct validity could, for instance, involve collecting additionaldata regarding the student’s well-being and relationships through healthservice records, in-depth interviews, observations, or friendship nominationsin the peer group.

We recognize that it is highly challenging to develop a questionnairefor this group of adolescents. The scales may need to be further adjustedin future studies when more extensive data is collected from studentswith IDD from different contexts, including cases with clinically recog-nized mental health problems. For example, the distinction betweenmental health and mental ill-health (that was motivated by theoreticalconsiderations described in the introduction) is in need of furtherempirical validation, especially in IDD populations. Moreover, the scaletentatively called Family Environment received a rather low Cronbach’salpha, suggesting that the items of scale may tap into more than oneconstruct. To further explore underlying factors of the scales and howitems relate to these, more data need to be collected and analyzed usingconfirmatory factor analysis. Recruitment of participants via special

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 21

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

needs comprehensive schools was vastly time-consuming and resulted invarying levels of participation. Taken together, further research is neededfor a full psychometric evaluation of the WellSEQ.

In the present sample, no information was obtained on the students andparents who chose not to participate. In particular, students with severeemotional and behavioral difficulties may have been underrepresented in thesample as some teachers expressed concern about letting them participate.

In the present study, the choice to use a 3-step response scale was based onthe difficulty of finding words for a more fine-graded ordinal scale that allstudents could comprehend. Unfortunately, using a limited number of scalesteps affects the accuracy of the measure and limits the distribution of data.However, the technology of tablet PCs enables the exploration of innovativesolutions to this problem.

In the present study, we aimed to develop a questionnaire for largerhealth surveys. The study showed that a lot can be gained from involvingstudents with IDD as active participants when designing questionnairesfor this group. We have also seen that students can respond indepen-dently if the questions, response scale, and presentation format areadapted to students with IDD in special education. Using practice ques-tions with feedback may further enhance reliability of responses, parti-cularly for students who are immature readers. Further research isneeded to evaluate the proposed scale structure and explore whetherthe WellSEQ can be used to identify individuals in need of support invarious populations of individuals with IDD. Indeed, the current attemptto assess mental health in children with IDD from a first-person per-spective raises many questions for future research.

ORCID

Petra Boström http://orcid.org/0000-0002-7952-8283

Acknowledgements

We would like to thank all the students, teachers, and parents involved in the WellSEQproject. This work was supported by the Swedish Research Council Formas, the SwedishResearch Council for Health, Working Life and Welfare, the Swedish Research Council, andSweden’s Innovation Agency VINNOVA. A special thanks is given for the contributionsmade by Robert Fohlin and Anders Boström.

References

Achenbach, T., & Rescorla, L. (2001). Manual for the ASEBA school-age forms & profiles: Anintegrated system of multi-informant assessment. Burlington, VT: University of Vermont,Research Center for Children, Youth, & Families.

22 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Boström, P., & Eriksson, E. (2015). Design for self-reporting psychological health in childrenwith intellectual disabilities. Presented at the Interaction design and children, Boston, MA,USA.

Bremberg, S., & Dalman, C. (2015, June). Begrepp, mätmetoder och förekomst av psykisk hälsa,psykisk ohälsa och psykiatriska tillstånd hos barn och unga - En kunskapsöversikt [Concepts,measurements and prevalence of mental health, mental ill-health and psychiatric condi-tions in children and youth - a review]. Stockholm, Sweden: Forte.

Christensen, L. L., Fraynt, R. J., Neece, C. L., & Baker, B. L. (2012). Bullying adolescents withintellectual disability. Journal of Mental Health Research in Intellectual Disabilities, 5(1),49–65. doi:10.1080/19315864.2011.637660

Cuckle, P., & Wilson, J. (2002). Social relationships and friendships among young people withDown’s syndrome in secondary schools. British Journal of Special Education, 29(2), 66–71.doi:10.1111/1467-8527.00242

De Ruiter, K. P., Dekker, M. C., Verhulst, F. C., & Koot, H. M. (2007). Developmental courseof psychopathology in youths with and without intellectual disabilities. Journal of ChildPsychology and Psychiatry, 48(5), 498–507. doi:10.1111/jcpp.2007.48.issue-5

Douma, J. C., Dekker, M. C., Verhulst, F. C., & Koot, H. M. (2006). Self-reports on mentalhealth problems of youth with moderate to borderline intellectual disabilities. Journal ofthe American Academy of Child & Adolescent Psychiatry, 45(10), 1224–1231. doi:10.1097/01.chi.0000233158.21925.95

Einfeld, S. L., Ellis, L. A., & Emerson, E. (2011). Comorbidity of intellectual disability andmental disorder in children and adolescents: A systematic review. Journal of Intellectualand Developmental Disability, 36(2), 137–143. doi:10.1080/13668250.2011.572548

Elwér, Å., Fridolfsson, I., Samuelsson, S., & Wiklund, C. (2009). LäSt Handledning [LäStManual]. Linköping, Sweden: VasaTryck AB.

Emerson, E. (2003). Prevalence of psychiatric disorders in children and adolescents with andwithout intellectual disability. Journal of Intellectual Disability Research, 47(1), 51–58.doi:10.1046/j.1365-2788.2003.00464.x

Emerson, E. (2005). Use of the strengths and difficulties questionnaire to assess the mentalhealth needs of children and adolescents with intellectual disabilities. Journal of Intellectualand Developmental Disability, 30(1), 14–23. doi:10.1080/13668250500033169

Eriksson, I., Cater, Å, Andershed, A.-K., & Andershed, H. (2011). What protects youths fromexternalising and internalising problems? A critical review of research findings and impli-cations for practice. Australian Journal of Guidance and Counselling, 21(02), 113–125.doi:10.1375/ajgc.21.2.113

Finlay, W. M., & Lyons, E. (2001). Methodological issues in interviewing and using self-report questionnaires with people with mental retardation. Psychological Assessment, 13(3),319–335. doi:10.1037/1040-3590.13.3.319

Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclasscorrelation coefficient as measures of reliability. Educational and PsychologicalMeasurement, 33, 613–619. doi:10.1177/001316447303300309

Foley, K.-R., Blackmore, A., Girdler, S., O’Donnell, M., Glauert, R., Llewellyn, G., & Leonard,H. (2012). To feel belonged: The voices of children and youth with disabilities on themeaning of wellbeing. Child Indicators Research, 5(2), 375–391. doi:10.1007/s12187-011-9134-2

Frauenberger, C., Good, J., & Keay-Bright, W. (2011). Designing technology for children withspecial needs: Bridging perspectives through participatory design. CoDesign, 7(1), 1–28.doi:10.1080/15710882.2011.587013

Goodman, R. (1997). The strengths and difficulties questionnaire: A research note. Journal ofChild Psychology and Psychiatry, 38(5), 581–586. doi:10.1111/jcpp.1997.38.issue-5

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 23

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Guha, M. L., Druin, A., & Fails, J. A. (2008, June). Designing with and for children with specialneeds: An inclusionary model. Presented at the Proceedings of the 7th InternationalConference on Interaction Design and Children, New York, New York, USA.

Guralnick, M. J. (2006). Peer relationships and the mental health of young children withintellectual delays. Journal of Policy and Practice in Intellectual Disabilities, 3(1), 49–56.doi:10.1111/ppi.2006.3.issue-1

Hartley, S. L., & MacLean, W. E. (2006). A review of the reliability and validity of Likert-typescales for people with intellectual disability. Journal of Intellectual Disability Research, 50(11), 813–827. doi:10.1111/jir.2006.50.issue-11

Keyes, C. L. (2007). Promoting and protecting mental health as flourishing: A complementarystrategy for improving national mental health. American Psychologist, 62(2), 95.doi:10.1037/0003-066X.62.2.95

Kieling, C., Baker-Henningham, H., Belfer, M., Conti, G., Ertem, I., Omigbodun, O., . . .Rahman, A. (2011). Child and adolescent mental health worldwide: Evidence for action.The Lancet, 378(9801), 1515–1525. doi:10.1016/S0140-6736(11)60827-1

Linehan, C., Walsh, P., Van Schrojenstein Lantman de Valk, H., Kerr, M., & Dawson, F.(2009). Are people with intellectual disabilities represented in European public healthsurveys?Journal of Applied Research in Intellectual Disabilities, 22(5), 409–420.doi:10.1111/jar.2009.22.issue-5

Matheson, C., Olsen, R. J., & Weisner, T. (2007). A good friend is hard to find: Friendshipamong adolescents with disabilities. Journal Information, 112(5), 319–329.

Matson, J. L., & Shoemaker, M. (2009). Intellectual disability and its relationship to autismspectrum disorders. Research in Developmental Disabilities, 30(6), 1107–1114. doi:10.1016/j.ridd.2009.06.003

Mileviciute, I., & Hartley, S. L. (2015). Self-reported versus informant reported depressivesymptoms in adults with mild intellectual disability. Journal of Intellectual DisabilityResearch, 59(2), 158–169. doi:10.1111/jir.12075

Ravens-Sieberer, U., Gosch, A., Rajmil, L., Erhart, M., Bruil, J., Duer, W., . . . Czemy, L.(2005). KIDSCREEN-52 quality-of-life measure for children and adolescents. ExpertReview of Pharmacoeconomics & Outcomes Research, 5(3), 353–364. doi:10.1586/14737167.5.3.353

Robitail, S., Ravens-Sieberer, U., Simeoni, M.-C., Rajmil, L., Bruil, J., Power, M., . . . Mazur, J.(2007). Testing the structural and cross-cultural validity of the KIDSCREEN-27 Quality ofLife questionnaire. Quality of Life Research, 16(8), 1335–1345. doi:10.1007/s11136-007-9241-1

Robitail, S., Simeoni, M.-C., Erhart, M., Ravens-Sieberer, U., Bruil, J., Auquier, P., & Group,E. K. (2006). Validation of the European proxy KIDSCREEN-52 pilot test health-relatedquality of life questionnaire: First results. Journal of Adolescent Health, 39(4), 596.e1–596.e10. doi:10.1016/j.jadohealth.2006.01.009

Scott, J., Wishart, J., & Currie, C. (2011). Including children with intellectual disabilities/special educational needs into national child health surveys: A pilot study. Journal ofApplied Research in Intellectual Disabilities, 24(5), 437–449. doi:10.1111/jar.2011.24.issue-5

Skolverket. (2015). For children and young people aged 7–20; Special needs comprehensive andupper secondary school. Stockholm, Sweden: Fritzes.

Sullivan, P. M., & Knutson, J. F. (2000). Maltreatment and disabilities: A population-basedepidemiological study. Child Abuse & Neglect, 24(10), 1257–1273. doi:10.1016/S0145-2134(00)00190-3

Trost, J. (2012). Enkätboken [The survey book]. (4th ed.). Lund, Sweden: Studentlitteratur.

24 P. BOSTRÖM ET AL.

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016

Upton, P., Lawford, J., & Eiser, C. (2008). Parent–child agreement across child health-relatedquality of life instruments: A review of the literature. Quality of Life Research, 17(6), 895–913.doi:10.1007/s11136-008-9350-5

Wechsler, D. L. (2003). Wechsler Intelligence Scale for Children (4th ed.). San Antonio, TX:The Psychological Corporation.

Westerhof, G., & Keyes, C. M. (2010). Mental illness and mental health: The two continuamodel across the lifespan. Journal of Adult Development, 17(2), 110–119. doi:10.1007/s10804-009-9082-y

JOURNAL OF MENTAL HEALTH RESEARCH IN INTELLECTUAL DISABILITIES 25

Dow

nloa

ded

by [

Petr

a B

ostr

öm]

at 0

0:32

04

Aug

ust 2

016