OSPI Bilingual Education

OSPI BILINGUAL EDUCATION

Step by step instructions for completing the OSPI AMAO Data Analysis Worksheet

AMAOsHow do I find my district’s

results?

AMAO 2012-2013

The Detailed Excel AMAO files are posted on the OSPI website in the EDS application “Washington Assessment Management System

(WAMS)” https://eds.ospi.k12.wa.us/Login.aspx .

If you do not have access to WAMS, contact your District Assessment Coordinator (DAC) to provide you with the file.

The Detailed Excel AMAO file provides the same information as had been reported in previous years on the WABilingual website, without the conditional formatting.

This file includes all results regardless of the minimum cell size, so are only available by logging onto EDS. Please treat these data as confidential in compliance with FERPA requirements.

9/10/2013

https://eds.ospi.k12.wa.us/Login.aspx

1

2

AMAO 2012-2013

Detail Excel AMAO file

9/4/2013

Select the file indicated under “AMAO 2012_2013.xlsx” to download your file.

7

Your Detail Excel AMAO file should look like this:

Formatting your Detail Excel AMAO file to evaluate Title IIIThe file requires formatting to be readable. You can do this yourself, or

format your file like the example in this presentation by following the instructions found at:http://www.k12.wa.us/MigrantBilingual/pubdocs/AMAO2013formattable.pdf

http://www.k12.wa.us/MigrantBilingual/pubdocs/AMAO2013formattable.pdf

http://www.k12.wa.us/MigrantBilingual/pubdocs/AMAO2013formattable.pdf

9

Takes half an hour+ to complete

AMAOsHow do I analyze my district’s

results?One process as recommended by Paul McCold, OSPI

11

Download the AMAO Data Analysis Worksheet

You need this to follow the steps

AMAO Data Analysis Worksheet

Analyze data using the AMAO District data for SY 2012-2013 to determine factors that prevented achievement of the annual measurable achievement objectives.

a.Download AMAO Detailed Excel File from OSPI in the EDS application WAMS https://eds.ospi.k12.wa.us/Login.aspx . If you do not have access to WAMS, contact your District Assessment Coordinator (DAC) to provide you with the file.

b.Complete this “AMAO Data Analysis Worksheet” to identify areas of concern.

Use additional copies of the sheets below as needed.

Office of Superintendent of Public InstructionMigrant and Bilingual Education

9/10/2013

https://eds.ospi.k12.wa.us/Login.aspx

Part I AMAOs. Use the Detailed Excel File for your district to complete the following questions.

•Which AMAO Targets were not met?

•Scan across the district total values (1st district row) to locate which AMAOs fail to meet targets. Use the percentage column for grades reporting “NR” to see if they have less than the targeted values (67.5% & 7.4%). Recommend highlighting cells not meeting targets.

•Indicate below which AMAOs the district did not meet, and for which grades at the district level. For AMAO-3 list subject and grades that did not meet.

(First highlighted row is an example).

9/10/2013

1

2

Select the “Data” tab and then click the “Filter” icon.

Your Detailed Excel AMAO file is now ready to analyze.

Scroll down so the District Level rows and the AMAO summary columns are showing.




•Indicate below which AMAOs the district did not meet, and for which grades at the district level. For AMAO-3 list subject and grades that did not meet. (First highlighted row is an example).

18

For AMAO-3 list subject and grades that did not meet.For unformatted sheets, these are in Columns AA and AG.




•Indicate below which AMAOs the district did not meet, and for which grades at the district level. For AMAO-3 list subject and grades that did not meet. (First highlighted row is an example).

2. AMAO 1: Grades and SchoolsEven if the district met the 67.5% AMAO-1 improvement target, identify specific schools or grades needing improvement.

a. Scan down the AMAO1 Pct Gain column to locate the percentages that have less than 67.5% of students making targeted improvement. Do not rely on Met AMAO1 column for this determination.

b. Indicate below any schools and grades within schools falling below 67.5% students showing improvement. Do not include schools with less than 10 matched students.

3. AMAO 2: Grades and SchoolsEven if the district met the 7.4% AMAO-2 transition target, identify specific schools or grades needing improvement.

c. Scan down the AMAO2 Pct Transitioned column to locate the percentages that have less than 7.4% of students transitioning from the TBIP program. Do not rely on Met AMAO2 column for this determination.

d. Indicate below the schools and grades within schools falling below 7.4% of students transitioning. Do not include schools with less than 10 tested students.

4. AMAO 3: Grades and SchoolsThe percent meeting standard in MSP/HSPE reading and math are shown in the last set of AMAO columns (Reading Met AMO and Math Met AMO). Cells that prevented the district or school from meeting the targets for AMAO3 are indicated by a “No” in columns these columns (or the Met Participation cells if <95%).

e. Scan down the Met AMAO3 column to locate schools and grades not meeting AMO. Highlight.

f. Indicate below schools not meeting AMAO-3 (AMO) by subject (math or reading). “NR” indicates the entity met due to N<20. Treat “NR” as meeting AMAO-3.

it is recommended to list elementary schools first, middle schools next, and high schools last

AMAO Targets Not Met by school and grade.

Filter on Gradeuncheck “Select

ALL”scroll down

then check “Total”

You can copy the school names and paste these into the worksheet, or print on paper.

Once filtered, you will only be

selecting cells that are unfiltered.

You can copy the results (one result at a time) and paste these into the worksheet, or print the worksheet and write these into the cells.

“NA” means there were no Title III students testing in those schools.

For the grades within schools, you will need to unfilter the spreadsheet and scroll downthrough each school’s results noting the grades not meeting for each AMAO.

For cells with N<R (number less than required < 20) if there are 10 or more students, use the actual percent gaining/transitioning to determine if they met. Ignore if <10 students.

Use Pct Gain

column

Use Pct Transitioned column

Copy extra pages or insert extra rows as needed to include all schools with Title III/TBIP students.

AMAO Targets Not Met by school and grade.

How well are your ELL students performing by the Language Domains?

Scroll over to the Pct Advanced Transitional columns (AJ to AM on formatted sheets).

These columns indicate the percent of all ELLs in your district that have moderate-high English comprehension (i.e., percent scoring at Level 3 or Level 4).

Highlight any District Level values more than 10% points below the state Level for each respective grade.

note: you can copy the “ü” in the worksheet and paste it into the cells.

Part II Language Domains. Use the Detailed Excel File for your district to complete the following questions.

Filter for each grade and compare school results to statewide results for each grade.

Look for cells 10% or more below the statewide percent for that grade. Ignore schools with less than 10 Title III students overall.

You can highlight the cells you want to focus on. Note that the Writing column had less than 15% at L3 or L4, so include those with less than half of the state total (12% / 2 = 6% or less).

Enter for each grade filtered, the schools whose grades are 10% or more lower than the state percentage of advanced English students in each language domain.

Complete the table by consecutively filtering by grade and highlighting cells with low percent of advanced English students. These are the places where the English language development needs are the greatest.

Notice that when the filter is turned off, you can readily see the schools and specific grades that have a lower than average percent of students at L3 or L4.These are your schools and grades with the greatest need for English Language support services.

9/10/2013 38

Where there are very few schools involved in a grade level, you can filter to include more than one grade at a time. Remember to compare the school specific grade to the comparable state level grade. Note: none of these middle school grades are 10% or more below the state grade equivalent totals.Among the High School grades, there are three cells 10% points or more below their state grade equivalents.

Part III. MSP/HSPE Performance: Use the Detailed Excel File for your district to complete the following questions.

How did ELL students perform on the MSP/HSPE in reading, math, writing and science?

Scan down the %Meeting MSP/HSPE Standards to determine which, if any, of the academic subjects are below the state averages for your district, schools, or grades.

Indicate (by placing a ✓) below for any grades at the district level, school totals, and then the grades within schools falling more than 5% points below the state total average, or the state average for each grade. Ignore cells with fewer than 10 students.

It is recommended to list elementary schools first, middle schools next, and high schools last.

Copy and paste the school names.

Highlight cells >5% points below the

state total

By highlighting the cells, you will be easily able to identify these later. Enter check marks (or x) into the cells indicated into the worksheet.

Repeat the filter for each grade consecutively until the worksheet is complete and all cells have been highlighted.

Notice that only

schools with 3 rd

grade are showing

Repeat for e

ach

grade. Ignore

results for 9

th grade

math.

Once all grades have been compared to the state results and the filter is turned off, you can easily see highlighted where your district is below the state average in the academic subjects.

Notice that once you filter for the upper grades, the number of schools you need to enter the results for is far fewer.

Because school differ in the English proficiency levels of their LEP students, the percent meeting state academic standards will be determined in part by these differences.

For example, if your 5th graders are well below the state average, it may be because your district has a greater concentration of Level 2 students in the 5th grade than the state average. Thus, your lower performance is due to English proficiency and not the quality of instruction for these students.

One way to control for these differences is to compare the MSP/HSPE results for each English language level separately (i.e., by WELPA level).

Filter for Grade “Total”

The columns starting “WELPA Level 1 Pct Met Reading” provide the Level-specific results.

Notice that there are no cells with 10 or more students at Level 1 English level proficiency in this district. Move the spreadsheet to the right to view Level 2 results.

All of the Level 2 cells at the district level have 10 or more students. Highlight or circle those subjects that the districts is more than 5% points below the state average. Also highlight or circle schools more than 5% points below the state average.

Notice that the results for the Writing and Math have the same number and same percentages. A view of the unfiltered file reveals that the Writing scores were repeated into the Math columns for all of the columns of WELPA Level by MSP results. For this year, you will need to live without the math scores on this part of the analysis.

Since none of the State Level results show more than 5% of Level 2 students meeting standard, do not indicate anything for Level 2 students.


All of the “Level 3” cells at the district level have 10 or more students. Highlight or circle those subjects that the districts is more than 5% points below the state average. Also highlight or circle schools more than 5% points below the state average.

Highlight or circle schools more than 5% points below the state average.


Notice that the results for the Writing and Math have the same number and same percentages. A view of the unfiltered file reveals that the Writing scores were repeated into the Math columns for all of the columns of WELPA Level by MSP results. For this year, we will need to live without the math scores on this part of our analysis.

Highlight the District Totals more than 5% points below the state average for each WELPA Level.

Enter the names of schools more than 5% points below the state average for each WELPA Level.

Sorry, math results not available by WELPA Level

Highlight the district-wide grade results that are more than 5% points below the state-grade level average for each WELPA Level. This section is above the previous table of school names.

Filter for the District Level and State Level only. Then also filter for grades (3 – 4 – 5, then 6 – 7 – 8, then 9 – 10 – 11 – 12 or one grade at a time if you have many schools).

None of the Level 1 students in these grades had 10 or more students, so ignore these.

At Level 2, only 4th grade had 10 or more students, but the state percentage for these grades is 5% or less, so ignore these.

Among the Level 3 students some grades had 10 or more students. The 4th graders 29% is within 5% points of the statewide 32%, but the 3rd grade and 5th grade reading are both more than 5% points below the state values.

Among the Level 4 students, only reading in 3rd and 5th grades and science in 5th grade had 10 or more students. None of these were within 5% points of the state-wide grade equivalents.

None of the Level 1 students in these grades had 10 or more students, so ignore these.

At Level 2, only 7th grade had 10 or more students, but the state percentage for these grades is 5% or less, so ignore these.

At Level 3 all grades are within 5% of their respective state-level grades in all subjects.

Among Level 4 students, all grades are below in reading, 7th grade in writing, and 8th grade in science.

Recall that the math scores here are a repeat of the writing results, so ignore the math columns.

None of the Level 1 or Level 2 students in high school had 10 or more students, so ignore these.

Only Level 3 students in high school grades had 10 or more students, and these percentages are within 5% points of the state-level grade equivalents.

Transfer these results to the Data Analysis Worksheet.

Your AMAO 2012-2013 district spreadsheet is now fully highlighted where your results are below the statewide averages for your Limited English Proficient students.

AMAOsHow do I summarize my district’s

results?

Identifying Areas of Concern

Summarizing your results for the District Improvement Plan.

1. Which AMAO Targets were not met?

Part I AMAO targets

SUMMARY - Grades

AMAO – 1: MET but low in grades 2, 4, 7, and 9

AMAO – 2: MET but low in grades K and 9

AMAO – 3: Not met in reading and mathReason for not meeting AMAO

targets

Summarizing your results for the District Improvement Plan.

2 – 4. Which Grades and Schools did not meet AMAO Targets?

Part I AMAO targets

SUMMARY – Schools and Grades

AMAO – 1: Scootney (2 3 4) Othello HS (9)

AMAO – 2: Lutacaga (1 2 3 5)

AMAO – 3: all except Hiawatha (math) Scootney (reading)

Summarizing your results for the District Improvement Plan.Part I AMAO targets

SUMMARY – Schools & Grades




SUMMARY - Grades



AMAO – 3: Not met in reading and math

Poor ELD growth

Poor ELD growth

These have unusually slow acquisition of English language.






SUMMARY - Grades



AMAO – 3: Not met in reading and math

Low English proficiency

Low English proficiency

These have unusually low levels of English proficiency.






SUMMARY - Grades



AMAO – 3: Not met in reading and math Low Academics

Low academics

All schools except these are below average in academic proficiency of LEP students.

Part II Language Domains.

5. Which language domain needs improvement?


Kindergarteners are lower than state averages in all four domains.

1st graders are below in reading, listening and writing English.

2nd and 10th graders are below average on reading English.

Part II Language Domains.

5. Which language domain needs improvement?


Wahitis students are lower than state averages in all four domains.

Lutacaga students are lower in reading and writing English .

Othello High students are lower than state average in reading and listening in 10th grade, and writing English in 9th grade.

74


Kindergarteners are lower than state averages in all four domains.

1st graders are below in reading, listening and writing English.

2nd and 10th graders are below average on reading English.


Wahitis students are lower than state averages in all four domains.

Lutacaga students are lower in reading and writing English .

Othello High students are lower than state average in reading and listening in 10th grade, and writing English in 9th grade.

Domains of English Language Acquisition

District’s entering students are below average in English proficiency across most language domains. The lowest ELD students are in Wahitis and Lutacage elementary. High school freshmen are below the state-wide average in ELD.

Part III MSP/HSPE Academic Performance.

6. How did ELL student perform on the MSP/HSPE in reading, math, writing and science?

SUMMARY – District and Schools

The district is below state average for LEP students in all four subject areas.

All schools except two elementary schools are below average in reading.

All elementary schools are below average in writing.

Two elementary and the middle school are below average in math.

Three elementary and the middle school are below average in science.




Three of four elementary schools are below average in 3rd grade reading and two in math.

Three of four elementary schools are below average in 4th grade in reading and writing, and two in math.

All elementary schools are below average in 5th grade in reading, and three in math and science.




The 6th graders in the middle school are below state average for LEP students in math.




High school students are behind state average for LEP students in math in all grades and in reading in the 10th grade.

79

SUMMARY – District and Schools

The district is below state average for LEP students in all four subject areas.

All schools except two elementary schools are below average in reading.

All elementary schools are below average in writing.

Two elementary and the middle school are below average in math.

Three elementary and the middle school are below average in science.


Three of four elementary schools are below average in 3rd grade reading and two in math.

Three of four elementary schools are below average in 4th grade in reading and writing, and two in math.

All elementary schools are below average in 5th grade in reading, and three in math and science.SUMMARY – Schools & Grades

The 6th graders in the middle school are below state average for LEP students in math.


High school students are behind state average for LEP students in math in all grades and in reading in the 10th grade.


Recall that the English proficiency of incoming K & 1st grade students are

below state average.

7. How did ELL students perform academically by WELPA level?

District totals


SUMMARY – Academics by ELD Level

WELPA Level 3 students are below the state average for reading.

WELPA Level 4 students are below state average for reading, writing and science.

Because the ELD level of the district’s incoming LEP students is below average, the expectation is that these students will necessarily perform below average on the academic assessments due to that lower level of English comprehension.

So it is important to “control” for this difference by looking at each WELPA level separately!

7. How did ELL students perform academically by WELPA level?District-wide grades


SUMMARY – Academics by ELD Level & grade

WELPA Level 3 students are below the state average for LEP students in reading in 3rd and 5th grades and below in science in 5th and 8th grades.

WELPA Level 4 students are below state average for reading in 3rd and 5th grades and all middle school grades, writing in 7th grade and science in 5th grade.

School results.7. How did ELL students perform academically by WELPA level?


SUMMARY – Academics by ELD Level & school

Hiawatha is below state average for LEP students in reading and science among Level 3 & 4

Lutacaga is below in reading and writing among Level 3

Scootney is below in writing among Level 3

McFarland is below in reading and science among Level 3 & 4 and writing among Level 4

Othello HS is below in reading for Level 3 & 4 students.

83

SUMMARY – Academics by ELD Level

Level 3 students are below the state average for reading.

Level 4 students are below state average for reading, writing and science.

SUMMARY – Academics by ELD Level & grade

Level 3 students are below the state average for LEP students in reading in 3rd and 5th grades and below in science in 5th and 8th grades.

Level 4 students are below state average for reading in 3rd and 5th grades and all middle school grades, writing in 7th grade and science in 5th grade.

SUMMARY – Academics by ELD Level & school

Hiawatha is below state average for LEP students in reading and science among Level 3 & 4

Lutacaga is below in reading and writing among Level 3

Scootney is below in writing among Level 3

McFarland is below in reading and science among Level 3 & 4 and writing among Level 4

Othello HS is below in reading for Level 3 & 4 students.


The effect of lower ELD level of incoming students persists into the upper elementary grades as is revealed by the below average academic performance of these grades.The results for reading are particular telling (absent math results) because it is tested in every grade. Hiawatha and Lutacaga require special consideration as do the high school’s LEP students. This is true even for students newly deemed English proficient.

84

Othello’s entering LEP students are lower than the state average in English proficiency across most language domains. The lowest ELD students are in Wahitis and Lutacaga elementary schools, and among high school freshmen.

The Othello School District did not meet the AMAO targets this year because of below average academic performance of the LEP students in Reading and Math.

The pace of English acquisition among the district’s LEP students is slowest in Lutacaga Elementary and among Othello High School freshmen.The below average academic performance of the district’s LEP students is at least partly due to the lower ELD level of incoming students. However, more effective English acquisition support is needed for Hiawatha and Lutacaga Elementary and Othello High School’s freshmen. Additional academic support is indicated even for students newly deemed English proficient.

IMPROVEMENT PLANIGRANTS FORM PACKAGE 671

85

WHAT CAN WE LEARN FROM THE AMAOS?AMAO-2 provides a map of where the need for English language support services are the greatest in your district.

AMAO-1 reveals where the progress of LEP students in acquiring English is the slowest in your district.

AMAO-3 (when ELD is controlled) reveals where student require additional academic content support in your district.

[email protected] questions please contact OSPI Bilingual Education

Documents

OSPI Bilingual Education