State of Arizona Office of the Auditor General



Baseline Study of Arizona's English Language Learner Programs and Data (April 2008)




The Office of the Auditor General has conducted a baseline study of the current state of English Language Learner (ELL) programs and available data. The report describes the ELL programs and participants in a sample of 18 school districts and charter schools, chosen to provide a cross-section of districts and charter schools across the State. This report also provides a point of comparison for future biennial audits, which are mandated by A.R.S. §15-756.12 to report on the effectiveness of the State’s ELL programs in achieving English proficiency. In addition, this report discusses the reliability of data necessary to manage and audit the programs and recommends steps to improve future data before the biennial audits begin. To place Arizona’s ELL programs in a national context, recent ELL data and trends from across the nation are presented.

ELL students and programs in Arizona
and in the U.S. (see pages 1 through 9)

Nation-wide, ELL students have composed 10 percent of the total student enrollment for kindergarten through 12th grade. In Arizona—one of five states with the highest concentration of ELL students—the 138,449 ELL students composed approximately 14 percent of the State’s total enrollment in fiscal year 2007. Although ELL students speak more than 400 different languages, Spanish is spoken by 80 percent of all ELL students nationally and 81.2 percent in Arizona. In Arizona, as in the nation as a whole, most of the ELL students are in elementary grades. The percentage of Arizona’s ELL students diminishes consistently from kindergarten through 12th grade. ELL students constitute over 15 percent of Arizona kindergartners and less than 2 percent of its 12th graders.

Arizona’s approach to ELL standards and assessment differs from many other states. Since 2004, Arizona’s districts and charter schools have relied on state-defined proficiency standards. In contrast, many districts in other states also use standards at the district, school, and even classroom level. For program entry, exit, and monitoring, Arizona’s districts are required to use the Arizona English Language Learner Assessment (AZELLA), a composite of speaking, listening, reading, and writing scores. The assessments used in other states are not as comprehensive. Ninety percent of districts surveyed in other states based entry decisions solely on oral proficiency, and nearly 82 percent used oral proficiency as the sole basis for exiting the program. Under the No Child Left Behind Act of 2001, the federal government requires that all public school districts annually assess all students with limited English proficiency, but neither defines proficiency nor specifies which tests that states must use to measure it.

Arizona’s most recent ELL legislation, HB2064, was an outgrowth of more than a decade of litigation and legislation, and further specifies the State’s ELL program structure. The bill requires the ELL Task Force to develop state-wide program models, ADE to monitor and report on the program’s success, and the Auditor General to audit the models’ effectiveness on performance-based outcomes and English proficiency. This study focused on current ELL programs operating at a sample of Arizona school districts and charter schools and the data these schools reported to ADE. The sample of 13 districts and 5 charter schools was designed to reflect different types of programs based on district or charter school size, location, number, and proportion of ELL population and percentage of ELL students reclassified as English proficient.

Characteristics of sample ELL programs
and participants (see pages 11 through 19)

ELL programs in the 18 sample districts and charter schools differ greatly from each other and from the structure they will have to employ in the future under the ELL Task Force’s new Structured English Immersion (SEI) models. Adopted in September 2007, the new models specify state-wide ELL policy regarding instructional approach and classroom practices. Implementation of these SEI models is expected beginning in fiscal year 2009. In fiscal years 2007 and 2008, the sample districts and charter schools operated ELL programs that varied by grade and proficiency levels and across individual schools. Although programs at the sample districts and charter schools were aligned with the new models’ requirements regarding assessment and English-only classroom materials, their instructional approaches were significantly different from the models’ future SEI requirement regarding English language development (ELD).

The Task Force’s models require all new ELL students to receive daily 4-hour ELD classes. In fiscal year 2008, only 3 of the 18 sample districts and charter schools were operating classes that met this upcoming requirement, but only for a portion of their ELL students. In 2007, the most recent year for which comprehensive data was available, more than half of all ELL students in the sample districts and charter schools attended programs that mainstreamed all ELL students, providing no hours of ELD instruction in an SEI setting. Forty-two percent were in programs that provided up to 2 hours of daily ELD instruction. The remaining 6 percent provided more than 2 and up to 4 hours of daily ELD. Across sample districts and charters, program officials identified the schedule and resource challenges to meeting the 4-hour daily ELD requirement. For example, program officials at three rural districts stated that they currently have difficulty filling regular teaching slots, and they do not know where they would find four to six more teachers with the qualifications to teach SEI classes.

The new models are designed to bring ELL students to full proficiency in 1 year. For the sampled districts and charter schools, about 7 percent of the approximately 8,700 ELL students became fully proficient in fiscal year 2007, and most of them had been in the program for at least 2 years. Between fiscal years 2006 and 2007, nearly two-thirds of the students remained at the same proficiency level or regressed, while about one-third moved to a higher proficiency level. ELL students who were at the lowest proficiency levels tended to make the most progress.

ELL data and data systems need
attention (see pages 21 through 28)

A.R.S. §15-756.10 requires ADE to collect and maintain ELL data related to program funding and monitoring. A.R.S. §15-756.12 requires the Auditor General to review compliance with program requirements in district performance audits and biennially report on the overall effectiveness of the State’s ELL program. Based on reviews of the data submitted by sample districts and charter schools to ADE and of source records, auditors found several errors affecting both the accuracy of funding ADE provides to districts and charter schools and the measurement of ELL students’ progress.

To adequately fund, manage, and audit the effectiveness of Arizona’s ELL programs, three main types of information are needed: the number of ELL students, achievement outcomes, and time in the program. First, since state funding is awarded on a per-student basis, ADE needs an accurate determination of the number of ELL students in each district or charter school. However, auditors’ review of sample files found that basic student identification information was not necessarily present, or if present, was not necessarily correct. Further, 2 percent of the sample ELL students who were funded had no assessment scores or had an assessment score indicating that, prior to entering the program, the ELL student was already proficient. Second, although accurate proficiency information is needed to monitor and evaluate districts’ and charter schools’ success, for 6.5 percent of the students, the classification of overall proficiency levels did not match their corresponding AZELLA scores. Additionally, 4 percent who were reclassified as proficient actually had scores that were below proficient or lacked valid assessments. Third, since funding is limited to 2 years, ADE needs to know how long each ELL student has been participating in the program. However, 6 of the 18 sample districts had ELL students with questionable entry and exit dates. To prevent such errors, ADE needs to work with district and charter schools to develop better ELL data submission and review processes.

ADE does not have adequate procedures in place to identify these errors and either correct them or minimize their impact. When data submitted by districts and charter schools is inaccurate or incomplete, ADE’s own internal processes should be able to identify the problems. Auditors found that ADE’s integrity checks and controls for doing so are insufficient. Consequently, the errors and limitations in the data are carried into the system. For example, based on schools’ erroneous data submissions, ADE included English-proficient students in program-funding counts. Further, an agency’s data system should include process controls to ensure that no data is added, lost, or altered during processing. In August 2006, the Auditor General’s performance audit of ADE’s information management recommended that ADE add such controls. The presence of such controls could help users and ADE prevent potential problems with student achievement and funding data, such as the detection and correction of a massive state-wide recalculation and override of ELL counts that occurred in July 2007. In this case, because of a processing error, over 20,000 ELL students were excluded from end-of-year funding counts. ADE eventually corrected the $8 million error, but analysts struggled to identify its cause. Further, ADE did not investigate prior years’ funded counts for similar processing exclusions.

Also, in order to better monitor implementation of the new SEI models and assess program success, ADE could collect additional data that is currently available from districts. Information about the number and qualifications of ELL teachers is not currently collected and maintained by ADE, although ADE requires districts to maintain ELL teacher certification and endorsement documentation for review during monitoring visits. Information about the number of hours of ELD instruction could also assist with monitoring efforts. Sample districts and charter schools provided auditors with information on students’ hours of ELD instruction based on their proficiency and grade levels.

Read full report in Acrobat PDF format




 Home | About UsPublications | Careers | Links | Contact Us | Privacy Statement | Webmaster

Copyright 2014 State of Arizona Office of the Auditor General, All Rights Reserved.