Metalinguistic and pragmatic abilities of participants in adult literacy programs

ELSEVIER

METALINGUISTIC AND PRAGMATIC ABILITIES OF PARTICIPANTS IN ADULT LITERACY PROGRAMS MARIE E. BYRNE THOM AS A. CROWE SUE T. HALE Department of Communicative Disorders, University of Mississippi, University, Mississippi

EDWIN E. MEEK Public Relations and Resource Development, University of Mississippi, University, Mississippi

DENISE EPPS Department of Communication Sciences and Disorders, Howard University, Washington, D.C.

The relationship between oral and written language skills was explored in this study involving 97 participants enrolled in an adult literacy program. Pragmatic language skills were assessed in videotaped conversational dyads which were later analyzed for level of conversational assertiveness and responsiveness and for the status of basic interactional skills. Those behaviors which were judged as being most likely to detract from communicative effectiveness were body movement, eye gaze, and facial expression. Metalinguistic/ semantic language skills were assessed with the Test of Word Knowledge (TOWK) (Wiig & Secord, 1991). A significant relationship was found between total scores on the TOWK and reading levels of subjects as they entered the program. Post-testing was conducted approximately 6 to 8 months after the initial testing. These results were available for 22 of the subjects and indicated that, even for adults, some aspects of oral language improve as reading levels increase.

INTRODUCTION

It has been estimated that 50°70 to 70070 of adults who enroll in literacy programs drop out of them before completion (Harman, 1987). Research in

Address correspondence to Marie E. Byrne, Ph.D., Department of Communicative Disorders, University of Mississippi, University, MS 38677.

J. COMMUN. DISORD. 29 (1996), 37-49 0021-9924/96/$15.00 © 1996 by Elsevier Science Inc. SSDI 0021-9924(95)00025-9

38 BYRNE ET AL.

the area of adult literacy has focused primarily on attempting to determine the factors that promote participation in literacy programs (Fitzsimmons, 1991; Ziegahn, 1992) or act as deterrents to successful completion of these programs (Hayes, 1988). The influence of factors such as personal motiva- tion, family support, and the ability to foresee attainment of goals has been recognized as being important to a person's success in achievement of literacy through adult literacy programs (Fitzsimmons, 1991). The influence of an adult learner's oral language skills as a factor in the development of reading skills has not been discussed in the recent literature on literacy. However, a positive relationship between reading and oral language, particularly focused on metalinguistic skills, has been demonstrated in emergent literacy (Menyuk, Chesnick, & Liebergott, 1991; Schuele & van Kleeck, 1987; Snow, 1991).

Research on early literacy development indicates an interdependence between oral language and reading skills. The attainment of adequate oral language and metalinguistic skills is necessary for development of early literacy (Miller & Gildea, 1989; Scholes & Willis, 1987; van Kleeck & Schuele, 1987). Reading requires readers to make use of their knowledge of phonology, syntax, semantics, and pragmatics and also depends on the development of metalinguistic skills, i.e. the ability to consciously reflect upon language. Deficits in oral language which affect vocabulary and word finding appear to interfere with reading acquisition (Schuele & van Kleeck, 1987). Further, just as poor semantic development will impede development of literacy, a reciprocal effect will be seen by the fourth grade as reading be- comes a major source of semantic development (Miller & Gildea, 1989). As Scholes and Willis (1987) demonstrated, oral language changes as reading skill is increased. Although these authors saw developments in metalinguistic skills involving phonology, morphology and syntax as children's reading skills improved, a control group of illiterate adult subjects in their study did not demonstrate an increased level of metalinguistic ability. Semantic development is also an important factor in the development of functional literacy skills. Level of acquired word knowledge is considered to be the "best single predictor of reading comprehension as well as lifetime earning potential" (Wiig & Secord, 1991, p. 2). The acquisition of mature reading skills requires an individual to abandon the principles of mutual exclusivity (only one word exists for expressing a given meaning) and contrast (more than one meaning cannot be expressed by any one word). These early oper- ating principles must be rejected if higher level metalinguistic skills necessary for reading literacy are to develop (Kuczaj, 1992; Wiig & Secord, 1991).

Although illiteracy is found in all segments of society, it appears to be concentrated among minorities with low income levels (Hunter & Harman, 1985). A 1991 report to the Mississippi Employment Commission and the Governor's Office for Literacy estimated that 35°7o of adults in the state

METALINGUISTIC AND PRAGMATIC ABILITIES 39

had low literacy skills (Cosby, Howell, Carr, & Miller, 1991). African- American subjects were at greater risk for low literacy skills even when educa- tional attainment levels were similar to white subjects. In addition, apprecia- bly lower than average levels of literacy were found for adults receiving public assistance. One conclusion of that report was that these adults would have better prospects for employment or advantage in their current jobs if they developed higher literacy skills.

Literacy alone may not be enough to attain and maintain employment. Social communication skills are important for success in job interviews as well as in daily workplace interactions with peers and authority figures (Auty, Goodman, & Foss, 1987; Siegelman & Davis, 1978; Trent, 1987). There has been little research on the relationship between social communication skills and employment opportunities for economically disadvantaged adults. This relationship has been demonstrated for adults who are learning disabled or mentally retarded. When problems in pragmatic behaviors such as appropriate listening and body language have been identified, training to improve performance in these areas has had positive results for vocational rehabilitation clients (Farley & Hinman, 1986).

Project LEAP (Learn, Earn, and Prosper) was implemented as a joint venture of the University of Mississippi, the Mississippi Department of Hu- man Services, and the Governor's Office on Literacy. This program was designed to provide adults receiving government assistance with literacy instruction and guidance in workplace communication by an on-site teacher and via satellite instruction for 20 hours per week. It was the first project of its kind in the nation to use satellite technology to enable participants to acquire those literacy skills and pragmatic communication skills needed to secure employment. At the time of this study there were 688 adults receiving Project LEAP literacy and communication skills training at 53 sites throughout the state of Mississippi.

This study examined the semantic/metalinguistic and pragmatic communication skills of adults enrolled in Project LEAP. The study was designed to answer two research questions: (1) What pragmatic skills essen- tial to workplace communication are not present in adults participating in Project LEAP?, (2) What is the relationship between reading and semantic/metalinguistic language skills for Project LEAP participants?

SUBJECTS

The subjects participating in this investigation were 92 females and 5 males between the ages of 16 and 52 years (mean age = 30 years). Subjects were predominately African-American (88.3°7o) and all were receiving Aid for Dependent Children, a federal program administered through the Missis-

40 BYRNE ET AL.

sippi Department of Human Services. Project participants were required by the state of Mississippi to attend Project LEAP classes as a condition for receiving welfare benefits. Exceptions to this requirement were made when circumstances resulted in a change in a participant's status such as illness or pregnancy.

All subjects voluntarily agreed to participate in the study and were selected from 10 of 53 Project LEAP training sites throughout the state. All subjects were tested for hearing acuity and middle ear function and were found to have hearing within normal limits. Reading grade levels of participants ranged from 0.1 to 10.9 with a 5.3 mean reading grade level as measured by the Test of Adult Basic Education, or the Wide Range Achievement Test.

PROCEDURES

Assessment of Pragmatic Communication Skills To assess pragmatic skills that might be important to workplace communication, subjects were interviewed regarding their backgrounds and interests. Each interview was videorecorded to allow for later analysis of nonlinguis- tic aspects of communication in addition to linguistic and paralinguistic behaviors. Using interview questions, the examiners encouraged conversational interaction with each subject. (See Appendix A.) The videotaped samples were then analyzed for conversational assertiveness and responsiveness as well as for basic interactional skills such as eye contact and facial expression.

A 10-item pragmatic checklist based on the Pragmatic Protocol developed by Prutting and Kirchner (1987) was used to code each subject's conversational skills. (See Appendix B.) Three graduate students in the Depart- ment of Communicative Disorders at the University of Mississippi who were familiar with the purposes of the study and trained in using the checklist served as raters. They separately viewed videotapes of each participant en- gaging in conversation with an examiner and judged the appropriateness of each of the 10 pragmatic checklist items for each subject. Each video- tape contained conversational samples for each participant from a given LEAP training site. The order of tape presentation was not controlled. Ap- proximately eight months elapsed between the students' rating of pretest and posttest samples. Raters could judge a behavior to be appropriate, inappropriate or indicate that there was no opportunity to observe that behavior. A consensus of agreement was reached when at least two of the three judges agreed upon a rating. A consensus was reached for 99.2070 of behaviors assessed.


Assessment of Metalinguistic/Semantic Language Skills All subjects were individually administered the four Level B core subtests from the Test of Word Knowledge (TOWK) (Wiig & Secord, 1991). This test was administered to determine the participants' abilities to define words, identify similar meanings, understand figurative language, and use words in multiple contexts.

The TOWK was scored according to instructions provided in the manual. Raw scores were calculated for each subtest and a total raw score was calculated for the four subtests administered. Although normative data for adults are not provided for this test, the authors stated in the manual that the test "may be used as a criterion reference measure, with the criterion being the level of maturation of word knowledge attained at age 17" (p. 3). For this investigation, standard scores were derived by comparing the subjects' raw scores with the normative data for individuals aged 14.0 through 17.11 years of age in order to determine areas of relative strengths and weak- nesses. Multiple correlational analysis involving the reading levels of participants, subtest scores and total TOWK scores was performed.

Reliability To establish reliability of the pragmatic checklist, 25070 of the videotaped samples were randomly selected and rescored by the first author using this instrument. Point by point agreement of 9407o with the consensus ratings was established for scoring of items included on the pragmatic checklist.

R E S U L T S

Table I lists the results from the assessment of subjects' pragmatic language skills. A mean of 907o of observed behaviors on the pragmatic checklist were judged to be inappropriate.

Behaviors most often found to be inappropriate included body movement (33.807o), eye gaze (19.5070) and vocal intensity (16.9070). These nonlin- guistic and paralinguistic behaviors were judged to have a negative effect on subjects' communicative effectiveness and were judged to be detrimen- tal to the communicative interaction. Turn-taking responsiveness was judged to be generally appropriate for all subjects, but turn-taking initiation was rated as inappropriate for nearly 10070 of the subjects. To determine if an examiner's race was a possible influence on subjects' performance, results from subjects interviewed by an African-American examiner were compared to those obtained from dyads containing a white examiner. A total of 1307o of the samples included an African-American examiner and a mean of 807o

42 BYRNE ET AL.

Table 1. Behaviors Assessed on the Pragmatic Checklist

Inappropriateness Behavior (07o)

Variety of speech acts 3.9 Turn-taking initiation 9.1 Turn-taking response 0.0 Turn-taking pause time 0.0 Feedback to speaker 2.6 Lexical specificity 5.2 Vocal intensity 16.9 Body movement 33.8 Facial expression 9.1 Eye gaze 19.5

total behaviors were judged as being inappropriate for those samples. Vocal intensity (33070) and body movement (25070) were the behaviors most often judged to be inappropriate in those dyads.

On the TOWK, subjects scored higher on the receptive subtests (Syno- nyms and Figurative Usage) compared to the expressive subtests (Word Definitions and Multiple Contexts). An area of relative strength was the subjects' knowledge of figurative language whereas word definitions and multiple contexts were areas in which the subjects obtained lower scores. Subjects tended to be inflexible in their use of words, often demonstrating only one operable definition for a single lexical item. Mean scaled scores for each subtest are listed in Table 2.

A significant correlation (r = 0.56, p < .01) was obtained for the relationship between total score on the Test of Word Knowledge and reading level. The best predictor of reading level was performance on the subtest assessing knowledge of synonyms (r = 0.57, p < .01).

Post-Testing Of the 97 Project LEAP participants seen for initial language assessment, 22 remained in the program at the time of posttesting. These Project LEAP participants ranged in age from 19 to 45 years (mean age = 31 years) and were seen for reassessment after completing between 6 and 8 months of literacy training. All subjects were female and the majority (91°70) were African-American. At the time of the initial language assessment, these subjects' reading grade levels, as measured by the Test of Basic Education or the Wide Range Achievement Test, ranged from 2.1 to 6.0 with a mean reading grade level of 3.0. At the time of oral language posttesting, reading grade levels on these measures ranged from 2.5 to 8.9 with a mean reading


Table 2. Mean Subtest Standard Scores on the TOWK

Subtest Mean

Word definitions 4.63 Multiple contexts 4.74 Synonyms 5.46 Figurative usage 6.38

grade level of 4.8. The change in reading grade level was significant (t = -3.68, p < .01).

The TOWK (Wiig & Secord, 1991) was again used to measure semantic language skills, specifically the subjects' abilities to define words, identify similar meanings, understand figurative language, and use words in multi- pie contexts. Pretest and posttest comparisons demonstrated significant in- creases in subjects' abilities to recognize appropriate synonyms for words (t = - 3.79, p < .001). As reading level increased, so too did word knowledge. An increase in performance on all other subtests assessing semantic and metalinguistic knowledge was also demonstrated but these changes did not reach statistical significance (See Table 3).

Pragmatic language skills were also reassessed in a manner similar to that used for pretesting. The same conversational skills assessed during initial evaluation were rated from videotapes of subjects engaged in conversation. No significant change was seen in these ratings. For the 22 subjects in the posttesting sample, 11°70 of behaviors were rated as being inappropriate during posttesting compared to 10070 of these behaviors rated as inappropriate during pretesting. Specific behaviors most likely to be rated as being inappropriate during posttesting included body movement (2907o) and eye gaze (3307o).

D I S C U S S I O N

Subjects' pretest performance on the Test of Word Knowledge indicated that they retained the principles of exclusivity and contrast which limited semantic development. This was evidenced by their low performance on the TOWK subtests designed to assess use of words in multiple contexts and knowledge of synonyms. Approximately 5007o of English vocabulary consists of multiple meaning words (Durkin, Crowther, & Shire, 1986). Sub- jects' inability to appreciate that one word may have more then one meaning, and that more than one word can be used to express the same meaning, substantially impedes semantic and metalinguistic language development. The poor performance of subjects on subtests assessing knowledge of multiple meanings and synonyms would seem to support the assumption of

44 BYRNE ET AL.

Table 3. Mean and Standard Deviation for TOWK Raw Scores

Pretest Posttest

Synonyms Mean 19.15 23.27 SD 5.01 8.04

Figurative usage Mean 24.09 27.41 SD 9.81 8.93

Word definitions Mean 24.00 25.91 SD 9.07 7.22

Multiple meaning Mean 7.64 9.87 SD 5.26 6.70

Receptive composite Mean 66.23 73.18 SD 10.03 10.36

Expressive composite Mean 67.72 71.13 SD 9.09 9.14

a reciprocal relationship between reading achievement and semantic/metalinguistic development. Adults who do not become functional readers do not have access to an important means of learning about language. Just as lower levels of word knowledge impede reading development, low literacy skills hinder further development of word knowledge.

Some change in oral language skills is possible, however, even into adulthood with increased literacy skills. Results of this study indicated that as reading level improves, so too does a person's word knowledge. Adult literacy programs can effect changes not only in reading literacy but in oral language as well. This effect was primarily demonstrated by changes in oral language comprehension, specifically in knowledge of synonyms, for subjects in this investigation. It appeared that subjects began to abandon the principle of mutual exclusivity as reading level increased. This effect, which has been noted in the literature on emergent language (Kuszaj, 1992), has not previously been reported for adult learners.

Pragmatic behaviors judged to be inappropriate during pretesting and posttesting were primarily those associated with communicative stress. Ex- tralinguistic behaviors such as averted eye gaze and excessive body movement together with decreased turn-taking initiation may wrongly commu- nicate disinterest or apathy. These are behaviors that would penalize individuals in communicatively stressful settings such as job interviews.

Only one of the examiners participating in the videotaped interviews was African-American. Although subjects' performances were not substantially


Table 4. Comparisons of Initial Reading and Oral Language Levels

Status Reading level TOWK raw score

Still attending 3.95 75.27 Completed GED 7.89 106.17 Dropped out 4.99 67.45

different in the interviews conducted by the African-American examiner, future investigations which control for the race of the examiner may yield different results. In addition, the possibility that these behaviors may be related to cultural differences should not be overlooked and should be further explored. Even if these behaviors are associated with cultural differences, they are a potential source of miscommunication and should be directly addressed in the curriculum of adult training programs. The positive outcomes of communication training programs for vocational rehabilitation clients (Farley & Hinman, 1986) should encourage the use of such programs with any adults who wish to improve their prospects for employment.

A limitation of this study was the low subject retention rate from pretesting to posttesting. Some subjects were still attending Project Leap classes but were not on-site on the day the posttesting was completed. Most of the pretest subjects, however, were no longer attending classes at the time of posttesting. Information concerning reasons for withdrawal was available for 49 subjects who had left the program. The participants withdrew from the program for a variety of reasons. Seven students unavailable at the time of posttesting had successfully passed the GED. Other reasons for leaving the program included successful job placement (n = 3), transferred or left the area (n = 6), and lack of interest in the program (n = 9).

Post-hoc multiple range testing following analysis of variance revealed significant initial reading and oral language differences between subjects who successfully passed the GED and subjects who either were continuing in the LEAP program or who had dropped out. Those who successfully passed the GED had higher initial reading levels and oral language skills as measured by the TOWK compared to the other two groups of subjects. (See Table 4.)

The results of this study reaffirmed the relationship between oral language and literacy skills. The reciprocal effect of disabilities in oral and written language appears to continue to be operable into adulthood for those in impoverished environments. But change appears possible with literacy training. Further research is needed to explore the effects of literacy on oral language skills. Project LEAP is an ongoing literacy training program and modifications will be made to this program based on the results of these and other research findings. These types of research studies are

46 BYRNE ET AL.

i m p o r t a n t no t on ly to l i teracy t ra in ing p rog ram deve lopment bu t also to resolving issues concern ing p rog ram efficacy.

This s tudy was funded th rough a grant f rom the Miss iss ippi D e p a r t m e n t o f H u m a n Services Jobs Program. The authors grateful ly acknowledge the con t r ibu t ions o f Marga re t Haire, E l izabe th Young, Jenni fer Stewart, and Melan ie M o o r e for thei r pa r t i c i pa t i on in the compi l a t i on o f d a t a for this invest igat ion.

Appendix A. Project LEAP Interview Questions (Q) and Conversational Prompts (P)

1. What is your name? (Q) 2. I always thought my name was rather different. (P) 3. What is your address? (Q) 4. What is your phone number? (Q) 5. Boy, we almost had a wreck coming over here today. (P)

Scared me to death. 6. When were you born? (Q) 7. So that makes you how old? (Q) 8. Where were you born? (Q) 9. I was born very far from here. (P)

10. What was the last grade you completed in school? (Q) 11. Where did you mostly go to school? (Q) 12. Was that out in the country, in a town, or in a city? (Q) 13. Why didn't you finish school? (Q) 14. Did you ever go to any special classes? (Q) 15. Have you ever had a speech or hearing problem? (Q) 16. Do you have any children? (Q) 17. What are their ages? (Q) 18. Boy, my sister has a bunch of kids that keep her busy. (P) 19. What are your hobbies? (Q) 20. I guess my hobby would be considered as unusual. (P) 21. Do you have a favorite TV program? (Q) 22. I have a favorite TV program I watch all the time. (P)


Appendix B. Pragmatic Assessment Form

Subject: Site:

Pragmatic Assessment

Appropriate Inappropriate

Variety of speech acts

Turn-taking initiation

Turn-taking response

Turn-taking pause time

Turn-taking feedback to speaker

Lexical specification/ accuracy

Vocal intensity

Foot/leg hand/arm and body movements

Facial expression

Eye gaze

Insufficient opportunity to observe

REFERENCES

Auty, W., Goodman, J., and Foss, G. (1987). The relationship between in- terpersonal competence and work adjustment. Vocational Evaluation and Work Adjustment Bulletin 20, 49-52.

Cosby, A.G., Howell, E M., Cart, J. C., and Miller, L. A. (1991). The Mis- sissippi Literacy Assessment: A Report to the Mississippi Employment Commission and The Governor's Office for Literacy. State of Mississippi.

48 BYRNE ET AL.

Durkin, K., Crowther, R.D., and Shire, B. (1986). Children's processing of polysemous vocabulary in school. In K. Durkin (Ed). Language development in the school years. 77-94. London: Croom-Helm.

Farley, R.C. and Himan, S. (1986). Enchancing job interview and job retention behavior with relationship training. VocationalEvaluation and Work Adjustment Bulletin 19, 55-60.

Fitzsimmons, K.A. (1991). African American women who persist in literacy programs: An exploratory study. The Urban Review 23, 231-250.

Harman, D. (1987). Illiteracy: A national dilemma. New York: Cambridge Press.

Hayes, E.R. (1988). A typology of low literate adults based on perceptions of deterrents to participation in adult basic education. Adult Education Quarterly 39, 1-10.

Hunter, C. and Harman D. (1979). Adult illiteracy in the United States: A report to the Ford Foundation. New York: McGraw Hill.

Kuczaj, S.A. (1982). Language Development Vol 1: Syntax and Semantics. Hillside N J: Elbaum.

Menyuk, P., Chesnick, M. and Liebergott, J. (1991). Predicting reading problems of at risk children. Journal o f Speech and Hearing Research 34, 5-10.

Miller, G.A. and Gildea, P.M. (1987). How children learn words. Scientific American 257, 94-99.

Nelson, N.W. (1993). Childhood language disorders in context: Infancy through adolescence. New York: Merril.

Prutting, C.A. and Kirchner, D.M. (1987). A clinical appraisal of the pragmatic aspects of language. Journal o f Speech and Hearing Disorders 52, 105-119.

Scholes, R.J. and Willis, B.J. (1987). Age and education in oral language skills. Developmental Neuropsychology 3, 239-253.

Schuele, C.M. and van Kleeck, A. (1987). Precursors to literacy: Assess- ment and intervention. Topics in Language Disorders 7, 32-44.

Siegelman, C.K. and Davis, P.J. (1978). Habilitation. Education and Training o f the Mentally Retarded 2, 71-76.

Snow, C. (1991). The theoretical basis for relationships between language and literacy in development. Journal o f Research in Children Education 6, 5-1o.


Trent, S.D. (1987). The importance of social skills in the employment interview. Education o f the Visually Handicapped 19, 7-18.

van Kleeck, A. and Schuele, C.M. (1987). Precursors to literacy: Normal Development. Topics in Language Disorders 7, 13-31.

Wiig, E.H. and Secord, W. (1991). Test o f Word Knowledge. New York: The Psychological Corporation.

Ziegahn, L. (1992). Learning, literacy, and participation: Sorting out pri- orities. Adult Education Quarterly 43, 30-50.

Documents

Metalinguistic and pragmatic abilities of participants in adult literacy programs