‘It’s good things and bad things to both systems’:

AAE MORPHOSYNTACTIC FEATURES AND EDUCATIONAL EVALUATION

Danny E. Flynn, Jr.&

Dennis R. PrestonMichigan State University

Background

How occurrences of nonstandard English influence the evaluation of written work in schools is surprisingly understudied.

This study seeks to discover the relative weights assigned to two linguistic features associated with good writing and to occurrences of nonstandard English, in this case, morphosyntactic features of African American English (AAE).

The Essay

We selected an essay from a website, (http://www.emsc.nysed.gov/ciai/dbq/assmtexp.html),

prepared by the Social Studies Division of the New York State Education Department. It was given the highest possible ranking for a 12th grade essay. The topic was democratic versus totalitarian forms of government.

We then modified this “good essay” (GE) lexically and syntactically, resulting in a “bad essay” (BE) (as in Freedman 1978).

Lexical ModificationWe substituted less sophisticated vocabulary or paraphrases. Typical substitutions were as follows:

Original Revised

…many advantages and disadvantages…

…many good and bad things…

… totalitarian form of government…

…forms of government with only one person in charge…

Dictator… Leader… …he enforced a totalitarian state. He made all of the rules. …revolt… …fight… liberals some people Totalitarian rulers… Rulers that are like kings…

Syntactic Modification

Using average words per “T-unit” (roughly, independent clauses with associated subordinate clauses) as a measure of sophistication, based on Hunt (1965), we did away with many complex grammatical constructions, reducing words per T-unit from 14.6 to 9.3. (Hunt’s work showed that average words per T-unit for 12th graders is 14.4, 8th graders 11.5, and 4th graders 8.6).

Typical syntactic modificationsORIGINAL REVISED

Throughout history, societies have held their different viewpoints on governmental decision making and the roles of citizens in this decision making process.

Throughout history, societies have always held their different viewpoints on governmental decision making. Societies have also held their different viewpoints on the roles of citizens in this decision making.

In a totalitarian form of government, the citizens have no say in the decisions that are made.

There are forms of government with only one person in charge. The citizens have no say in the decisions that are made.

The dictator has complete and total control.

The leader has complete control, and he has total control.

Democratic nations, such as the U.S. and England, seem to have fewer problems than totalitarian governments, like communist Cuba and Russia.

The U.S. and England are democratic nations. They seem to have fewer problems than governments that are not democratic like Cuba and Russia.

For example, in the change from

“In a totalitarian form of government, the citizens have no say in the decisions that are made.”

to

“There are forms of government with only one person in charge. The citizens have no say in the decisions that are made.”

there is a replacement of the lexical item “totalitarian” with a paraphrase and a reduction of a complex sentence with a word per T-unit score of 17 to two with scores of 11.

AAE ModificationThe following 8 AAE morphosyntactic features were added to both the GE and the BE

AAE Modification (cont.)

Essays SubmittedThese same AAE constructions were then doubled in both texts (which we called “heavy” AAE use), resulting in six different essays:

1.GE-NoAAE (Good essay; no AAE)2.GE-LAAE (Good essay; light AAE)3.GE-HAAE (Good essay; heavy AAE)4.BE-NoAAE (Bad essay; no AAE)5.BE-LAAE (Bad essay; light AAE)6.BE-HAAE (Bad essay; heavy AAE)

Methodology 1These six essays were presented to undergraduate students at Michigan State University at the beginning of Fall Semester 2005 (N=173).

The majority (N=93) are English majors and have secondary English teaching as their goal. The remaining respondents are mostly English and History majors. Demographic information (age, sex, ethnicity, region) and career goals and relevant experience (e.g., tutoring) was sought for each respondent.

Each student who participated saw only one form of the essay and was asked to grade this essay as if it were the work of a high school senior, using the traditional A-B-C-D-F scale.

The participants were also encouraged to write comments on the essay itself or on the back of the op-scan sheet used to collect the data.

Methodology 2

The data were analyzed statistically after converting the A-F scale into a 0.0 - 4.0 scale, one more often associated with university-level work. The 0.0-4.0 scale is used in the reporting of the data that follows, but 4.0 may be equated with an “A” and so on down the scale.

Before we look at the results, let’s imagine some of the possible outcomes:

Methodology 3

Nothing counts for nothing. Everybody gets a “C,” or you can imagine an “A” or an “F” if you like.

0

0.5

1

1.5

2

2.5

3

3.5

4

GE/NoAAE GE/LAAE GE/HAAE BE/NoAAE BE/LAAE BE/HAAE

POSSIBLE OUTCOME #1

A good essay is rated high and a bad one low, regardless of AAE amount or presence.

0

0.5

1

1.5

2

2.5

3

3.5

4

GE/NoAAE GE/LAAE GE/HAAE BE/NoAAE BE/LAAE BE/HAAE

POSSIBLE OUTCOME #2

Any AAE presence is rated low, regardless of amount or of essay quality.

0

0.5

1

1.5

2

2.5

3

3.5

4

GE/NoAAE GE/LAAE GE/HAAE BE/NoAAE BE/LAAE BE/HAAE

POSSIBLE OUTCOME #3

Amount of AAE influences ratings, regardless of essay quality.

0

0.5

1

1.5

2

2.5

3

3.5

4

GE/NoAAE GE/LAAE GE/HAAE BE/NoAAE BE/LAAE BE/HAAE

POSSIBLE OUTCOME #4

Amount of AAE and essay quality influence ratings.

0

0.5

1

1.5

2

2.5

3

3.5

4

GE/NoAAE GE/LAAE GE/HAAE BE/NoAAE BE/LAAE BE/HAAE

POSSIBLE OUTCOME #5

This is most like #1 (“nothing”) and #5 (“interaction”); the red boxes show that 1 and 2 are significantly different from 6 (.007).

ACTUAL OUTCOME

0

0.5

1

1.5

2

2.5

3

3.5

4

GE/NoAAE GE/LAAE GE/HAAE BE/NoAAE BE/LAAE BE/HAAE

Although good essays were rated slightly better and presence and greater number of AAE structures resulted in slightly lower ratings, neither factor was robust.

AAE presence by itself was not particularly important, as shown by the fact that Essay 2 (“good essay” with “light” AAE) was highest rated.

Results 1

But good essays were not very highly-ranked in general. There was no statistical difference between the three “good essays (1, 2, and 3) and two of the “bad essays” (4 and 5).

Only heavy AAE and “bad essay” combined to produce a statistically significant lower evaluation (.007), and then only in contrast to essays 1 and 2.

Results 2

Respondent sex played no role in these ratings, and there were insufficient numbers of minority students to calculate. Student major was also not significant.

Region, however, showed an interestingly different distribution of grades (.05).

1) Detroit urban area2) Other MI urban areas (Flint, Lansing,

Grand Rapids, etc.)3) Rural Michigan4) Other

Results 3

0

10

20

30

40

50

60

Detroit N=22 Urban MI N=73 Rural MI N=42 Other N=35

ABCDF

Overall assignment of grades by region

Rural respondents are much harsher in their ratings overall (.05), but small numbers in each cell do not allow us to relate that to the quality of the essay or the presence or amount of AAE.

These students, many English teachers in training, are in fact very harsh overall. The “good essay” (with no AAE) got a mark just barely above a “C.” Experienced professionals obviously thought much better of it.

Results 4

Some Qualitative Results

We hope to shed some light on these quantitative results by surveying some of the qualitative data collected.

For the moment we limit ourselves to some examples of these comments. We plan a more thorough investigation based on content classification.

Here are 3 comments on the worst-rated BadEssay/Heavy AAE

This first respondent refused to assign a grade:“I stopped at this point [i.e., end of paragraph 2]. There is no thesis, organization, transition or grammar. Please re-write, starting w/ a clear thesis & outline. The thesis needs some serious work! It needs to be more detailed, clearer, arguable etc…”[The respondent liberally annotated the essay with comments and corrections up to that point.]

Grade: F“Your paper did seem to get better as I read on but it needs a lot of work. I would love to meet with you after school & discuss the corrections I have made. –Mrs. Teacher”[She corrected all AAE and offered several other suggestions which would have made the grammatical structure more complex.]

Grade: Grammar: D, Overall: C-“Overall, first para. is to [sic] wordy, needs better thesis. Overusing the word this & they. PS, Is this a real essay?”[This rater also wanted the essay double spaced and the first paragraph indented.]

And here is a comment made on a Good Essay/LightAAE

GRADE: B“If there weren’t so many grammatical errors the essay would have recieved [sic] an “A”, but the errors were sufficient enough to lower the grade by a whole letter.”

Finally, here are 3 comments on the GoodEssay/NoAAE

Grade: BThe respondent made the notation “first name” by “Mussolini.”

Grade: C“avoid ‘good’ and ‘bad’ use high school vocabulary”

Grade: D “What a steaming pile of crap! This is one of the most boring papers I have ever read.” [Other expurgated comments.]

Some comments found in several essays included

1) Misidentification of many short sentences as “sentence fragments” (e.g., “This causes them to be unhappy.”).

2) Complaints that the essay contained “incorrect information,” although the essay was written by a student who had been given a reading on these facts.

In conclusion, English teachers-to-be (at least at Michigan State University) are very fussy graders, but attention to AAAE grammatical features does not overwhelm their evaluation of these essays in general, although an interaction between heavy AAE use and a less sophisticated essay (with regard to vocabulary and structure) results in a statistically lower evaluation.

What next?

1) What would real teachers (not teachers in training) do? Would those in other areas (e.g., history, social studies) have rated these essays differently. What is the source of the rural-urban difference?

2) Would it have made any difference if we had told the respondents that this was “on-the-spot” work? We noted only that it was 12th grade written work; some comments suggested that extensive proof-reading and revision would have helped.

3) What other characteristics of good and bad essays or other nonstandard features might be studied in this format? How important would they prove?

4) Would further demographic differences of raters (whether trainees or teachers) make any difference? Our respondent group was almost uniquely European American.

We hope to have more to report to you along the lines of these

improvements and others which we hope to get from you.

Thank you for your attention.

You can see this PowerPoint and the complete text of all six essays

at www.msu.edu/~preston