Early Assessment Program testing in English language arts

Posted by guest blogger Matthew Rosin, senior research associate, EdSource (Mountain View, CA).

This is my second post on California’s Early Assessment Program (EAP). The previous post provided a brief overview of the key components of the EAP, developed by California State University (CSU) and California’s K–12 leaders. The components include expanded versions of certain state assessments taken by 11th graders in English language arts and mathematics. These provide students with early signals about their college readiness.

This post focuses on how EAP testing is offered in English language arts, and what this means for student participation in the program.

To understand how EAP testing is conducted in English language arts, it is important to understand how the state of California assesses 11th graders in the subject. As part of California’s Standardized Testing and Reporting (STAR) program, the state administers California Standards Tests (CSTs). These tests assess student achievement relative to California’s academic content standards, across grades 2 through 11. There is a single English language arts CST for each of these grades.

In 2009, nearly all of California’s high school juniors (96%) took the Grade 11 English Language Arts CST, according to the California Department of Education. This means virtually all high school juniors in the state, if they wished, could participate in the EAP in English by taking an expanded version of the grade 11 test. By doing so, they could potentially achieve an exemption from placement testing in English at CSU.

The items that augment the Grade 11 English Language Arts CST include 15 additional questions and an essay. For the purpose of assessing students’ readiness for college, CSU considers students’ success on these additional items, and on about 40 questions from the original test.

A substantial majority of California’s high school juniors do participate in the EAP in English. CSU reports that 82% of 11th graders who took the English language arts CST in 2009 also participated in the EAP in the subject.

This means the vast majority of last year’s high school juniors in California public schools received early feedback on their readiness for college-level English during the summer before their senior years. Of these students, 16% were assessed as “ready for college” and exempted from CSU placement testing, according to CSU.

In the next post, we will take a closer look at how EAP testing is offered to 11th graders in mathematics. Because mathematics testing in California’s high school grades is organized differently than in English language arts, so too is the EAP in mathematics organized differently.

This post is adapted from EdSource’s High School to Community College report (Nov. 2008), updated with 2009 testing data.

First post on California’s Early Assessment Program

Posted by guest blogger Matthew Rosin, senior research associate, EdSource (Mountain View, CA).

California’s Early Assessment Program (EAP) has been cited as a potential model for other states hoping to align their high school standards and assessments with the placement expectations of their postsecondary institutions, and for sending students early signals about their preparedness to enter college without remediation in English and mathematics. Begun through partnership between the California State University (CSU), the California Department of Education and the California State Board of Education, California lawmakers also recently provided for the state’s community colleges to voluntarily participate in the EAP beginning this academic year.

The next several blog posts will explore how EAP tests are offered in grade 11 in English language arts and mathematics, what this means for student participation, and how this relates to the missions of different postsecondary institutions in California—all important issues with implications for other states that might wish to adapt the EAP to their own contexts. These posts will draw from EdSource’s November 2008 report, High School to Community College: New Efforts to Build Shared Expectations.

First, some background on the EAP is important. Offered for the first time in spring 2004, the EAP enabled CSU to provide California high school students with early feedback—during the summer before their senior years—about their preparedness for college-level classes in English and math. By giving high school students one year to become better prepared if needed, EAP developers hoped to reduce the proportion of incoming CSU students who need remediation in these subjects.

The developers of the EAP found that CSU’s placement expectations and the state’s K-12 standards for English and mathematics were aligned, but that CSU’s placement tests and the state’s high school assessments—the California Standards Tests (CSTs)—did not always emphasize the same things. The solution: give 11th graders the option to take expanded versions of CSTs in English and math. This decision avoided the need to develop yet another set of tests and standards to which students and teachers would need to respond.

The EAP has three components:

• Augmented versions of certain CSTs in grade 11. Students who do well on an EAP test are considered on track to be ready for college and are exempted from placement testing in English and/or math at CSU. In mathematics, students can also receive a “conditional” exemption from placement testing that is contingent on an additional year of mathematics during their senior year of high school.
• Assistance for students in grade 12 who need additional preparation, including additional coursework in English language arts and online services in mathematics. For example, the Expository Reading and Writing Course (ERWC) is designed to help high school students develop as readers and writers of exposition, analysis, and argument—skills that will be expected frequently in college courses.
• Professional development for high school teachers to build their capacity to improve students’ college readiness, including preparation to teach the ERWC.

In the next post, we will take a closer look at how the EAP test is offered in English language arts. Subsequent posts will examine how EAP tests are offered in mathematics, and what this means for California’s recent expansion of the EAP to include voluntary participation by the state’s open-access community colleges.

New study on community colleges and economic mobility

Posted by guest blogger Matthew Rosin, senior research associate, EdSource (Mountain View, CA).

The Pew Charitable Trusts’ Economic Mobility Project has released a new study, Strengthening Community Colleges’ Influence on Economic Mobility, based on analysis of attainment and earnings among students in Florida and including recommendations about career counseling. Among the study’s more interesting findings, the authors report that “lower-performing high school students who concentrate in high-return fields in community college”–among which the authors include computer science and engineering–“earn $48,000 annually, $4,000 more than A/B+ high school students who concentrate in low-return fields.” InsideHigherEd describes some of the early discussion that has emerged in response to the study.

Guest blogging, and a new piece by Michael W. Kirst

Posted by guest blogger Matthew Rosin, senior research associate, EdSource (Mountain View, CA).

It’s a privilege to be a guest blogger on The College Puzzle for the next two weeks during Michael W. Kirst’s absence. In keeping with Michael’s practice, I will post regarding interesting new developments in the secondary-to-postsecondary area.

This will include a series of posts about California’s Early Assessment Program (EAP), which has drawn attention as a potential model for other states to better align their K-12 and college academic standards, and to connect their existing high school assessments with the placement expectations of public postsecondary institutions. In part, these posts will draw on a report EdSource published in November 2008, High School to Community College: New Efforts to Build Shared Expectations.

In the meantime, I draw your attention to a short new piece by Michael W. Kirst, newly published by Education Commission of the States (ECS): “Progress and Gaps in College Preparation Policy.” He highlights four levers for improving college preparation policy: standards alignment, financial and other support for students, longitudinal data systems, and accountability.

A PDF of Michael’s new piece is available here.

Introducing Guest Blogger Matt Rosin

I will be away for awhile so Matt Rosin will be guest blogger. Matt is a senior research associate at Edsource in Mountain View, Ca. He specializes in transitions from secondary school to broad access postsecondary education. Matt has a Phd from Stanford and his latest book is New Agenda For Higher Education (Jossey Bass, 2008)

Washington State Gets Funds For New Community College Completion Initative

SEATTLE –The Washington State Board for Community and Technical Colleges (SBCTC) is launching the Washington State Student Completion Initiative aimed at dramatically increasing community college completion rates. The initiative, one of the most comprehensive completion efforts in the country, will launch new programs and expand successful pilot programs aimed at addressing key barriers to student success.

The initiative is supported by the state Legislature, a $5.3 million investment by the Bill & Melinda Gates Foundation, and $800,000 from the Ford Foundation.

“As Washington competes in the global economy, it is critical that we have the most highly skilled workforce available,” Gov. Chris Gregoire said. “That starts with our community and technical colleges. That’s why I have supported their innovative student completion initiatives in past state budgets and am so pleased to hear about this partnership with the Gates and Ford foundations, which will catapult their efforts even further.”

In today’s global economy, a college degree or postsecondary certificate is required to obtain a family-wage job. According to the federal Bureau of Labor Statistics, jobs for community college graduates will grow at a rate nearly twice as fast as the national average between 2006 and 2016. Nationally, only about 28 percent of first-time, full-time students at two-year institutions earn an associate degree within three years of enrolling. Washington is one of only a handful of states that is working aggressively to increase these percentages.

“Low-income young adults rely on community and technical colleges to get the skills they need in today’s economy, but many are struggling to succeed despite their best efforts,” said Jan Yoshiwara, SBCTC’s deputy executive director for education. “This partnership is funding some of the most innovative approaches to teaching and learning that will help us move more students further and faster to educational and economic success.”

The Washington State Student Completion Initiative will focus state and local attention on several key “achievement points” that research shows students must pass on the road to graduation, among them successfully completing pre-college or other remedial courses and completing a college-level math course.

“Getting more students into college means little if we’re not also making the effort to help them graduate,” said Hilary Pennington, director of Education, Postsecondary Success and Special Initiatives at the Bill & Melinda Gates Foundation. “Washington state has an inspiring track record of developing innovative and effective ways to do just that.”

Several Washington state programs already serve as national models for boosting student completion. These programs will be expanded over the four years of the Washington State Student Completion Initiative:

• The state’s I-BEST program combines basic academic courses and career skills classes to ensure that the least-prepared students not only complete college, but are competitive in the workforce upon graduation. Program evaluations suggest that I-BEST students are almost four times more likely to earn a credential or degree than similar students who were not enrolled in I-BEST. Under this initiative, Washington state will expand I-BEST to new pre-college and college-level degree programs. (I-BEST stands for Integrated Basic Education and Skills Training.)

• Washington’s Student Achievement Initiative provides financial incentives for institutions based on increases in student milestones that have demonstrated key linkages to college completion. It represents a significant shift from traditional funding, which typically is based solely on a college’s enrollment. During the pilot phase of the initiative, colleges served the same number of students but increased student achievement by nearly 5 percent. The greatest gains were a 10 percent boost in basic skills and nearly a 7 percent jump in college readiness. This initiative will expand the financial incentives to those colleges that see higher numbers of students who reach key graduation milestones.

The state will launch two new programs under the Washington State Student Completion Initiative:

• Strengthen and expand access to the colleges’ gatekeeper courses. There are 80 high-enrollment, gatekeeper and pre-college courses that most students must take to successfully earn a degree. Too often, students don’t complete these courses, can’t find an open section, or can’t afford the textbook. SBCTC and the 34 colleges will research, redesign, teach, and assess the 80 courses to improve completion rates through best practices in instructional design and active learning. This online initiative will reduce total student costs (an estimated $6.5 million a year) with open textbooks, course packs, existing library resources, and other open educational resources. All 80 courses will be digital, so faculty can select and continually improve courses, colleges can offer additional online or blended sections, and the courses can be shared throughout the college system and with the rest of the world. The state aims to achieve 95 percent completion rates in all redesigned high-enrollment gatekeeper courses.

• Take steps to improve students’ success in pre-college and college math. Math continues to be the major hurdle for students to complete certificates and degrees. More than 56,000 community college students in Washington are required to take remedial-level math (a 9 percent increase in the last year alone). Under this initiative, a coalition of seven colleges will focus on improving student math achievement by making substantive changes in curriculum, instructional practices and teacher support, and assessment. The program aims to increase successful completion of remedial math courses by 15 percent.

“Too many young people enroll in community colleges expecting educational advancement and broader opportunity—only to find a revolving door,” said Alison Bernstein, vice president of education, creativity and free expression at the Ford Foundation. “We are pleased to support initiatives such as Student Achievement that offer large numbers of students the chance to reach their ultimate goals of obtaining AA and BA degrees. We want students to complete their higher education, not just sample it.”

The grants announced today advance efforts by the Gates Foundation and the Ford Foundation to help increase the number of people in the United States who successfully earn a degree or certificate beyond high school.

Only 2% of K-12 Teachers Are African American Men

Only about 2% of teachers nationwide are African-American men. But experts say that needs to change if educators expect to reduce minority achievement gaps and dropout rates.

Macklin, now an 18-year-old college student, said he understands the circle that keeps many of his peers out of the classroom professionally.

“A lot of males, they don’t like being in school because they can’t relate to their teacher,” said Macklin, of Pittsburgh. “So why would you want to work there?”

American teachers are overwhelmingly white (87%) and female (77%), despite minority student populations of about 44%, according to the Bureau of Labor Statistics.

Why Are High School NAEP Scores Not Increasing?

Mark Schneider  former USA Commissioner of  Statistics has written an incisive analysis (below) of the static trend in NAEP test scores despite dramatic increases in academic course taking in recent decades. It used to be a research view that students scores will increase if they take more ELA and math courses, but no more!  However, be sure and read  Schnieder’s section on mitigating circumstances near the end. This post was on the American Enterprise website.

Math in American High Schools: The Delusion of Rigor

By Mark Schneider

The evidence on the failure of American high schools to educate and graduate their students is widespread. The release of the latest National Assessment of Educational Progress (NAEP) long-term trends (LTT) assessment data in April adds another data point to this sad compendium.[1] In this Outlook, I focus on trends in high school math, an area of critical national need, and one that has been a focus of national policy for decades. I present data that show a disconnect between the rigor of the math education that high schools claim to be delivering and the quality of the math education that students are actually receiving as measured by assessment data.

The 2008 NAEP’s LTT data reinforce other studies showing that U.S. high schools are failing in teaching math skills to their students. Administered every few years since the 1970s, NAEP’s LTT is one of the longest and most consistent data series we have on what American students know and what they can do in math. LTT is age-based and assesses the skills and knowledge of nine-, thirteen-, and seventeen-year-old students.

I focus here on the data for seventeen-year-olds, since these students are close to the end of their high school careers. Figure 1 shows that despite thirty years of effort to improve math instruction, our high schools have failed to improve their students’ math skills. In 1978, the average seventeen-year-old scored 300 on a 500-point scale. About fifteen years later, in 1992, the score was 306. And in 2008? Once again, 306.

In contrast to most of NAEP’s assessments, LTT does not have labels like basic, proficient, and advanced. Rather, NAEP describes the types of mathematical tasks a student can perform at five different cut points, starting at a score of 150 and increasing by fifty-point intervals to 350. At 300, students have demonstrated “moderately complex procedures and reasoning,” including computation with decimals, simple fractions, and commonly encountered percents. In addition, they can measure lengths and find averages–not exactly rocket science.

So what percentage of American seventeen-year-olds shows these moderately complex skills? In 1978, 52 percent; in 1992, 59 percent. And in 2008? Again, 59 percent. As a reminder, NAEP stands for the National Assessment of Educational Progress–not much progress to report in high school level math despite decades of effort.

Moreover, the LTT data show that the students high schools were failing to teach were coming to them with much better skills. Since 1978, thirteen-year-olds improved by seventeen points on the LTT math assessment and nine-year-olds by twenty-four points. And since 1994, scores of thirteen-year-olds increased by seven points and nine-year-olds by twelve points.

The depressing LTT data are matched by equally gloomy evidence from international assessments. Consider how our fifteen-year-old students (mostly tenth graders) stack up against their peers in other advanced industrial countries, as captured by the Organisation for Economic Co-operation and Development’s (OECD) Programme for International Student Assessment (PISA).[2]

As figure 2 shows, in 2003 the U.S. average was below the OECD average, and by 2006, the gap was even larger. Moreover, in 2006, across the OECD, the top 10 percent of students who took the assessment scored 615 or above. In the United States, the cut point for our top 10 percent was only 593–our best students do not match the performance of the best across the OECD. If, as President Barack Obama said in his speech to a joint session of Congress in February 2009, “countries that out-teach us today will out-compete us tomorrow,” our high schools are failing to build the foundation for tomorrow’s economy.

The Delusion of Rigor

Data about the failure of American high schools have been discussed for some time. But a number of indicators show that high schools can claim to be providing more rigorous instruction to a growing number of their students.

Here I rely on data from two different NAEP studies to show this disconnect: LTT and the series of NAEP high school transcript studies going back to 1990. Let us begin with the transcript studies.[3]

Figure 3, based on the most recent transcript study, shows that America’s high school students are taking more math courses. Since 1990, as documented in the top line of that figure, the average number of math Carnegie credits[4] a high school graduate completes has increased from 3.2 credits to 3.8–close to a 20 percent increase.

The LTT data give more detail about this increase in course credits. Figure 4 shows just how radically the composition of the math courses high school students take has shifted since the 1970s. In 1978, fully 20 percent of American high school students stopped with general math or prealgebra, and 17 percent stopped at Algebra I. Fewer than 10 percent reached precalculus or calculus as their final course.

The situation has changed dramatically. In 2008, only 3 percent of American high school students stopped at general math, and 7 percent stopped at Algebra I. In contrast, close to 20 percent now get through precalculus or calculus. While in 1978, only about one-third of American high school students completed Algebra II, now over half do.

This is a remarkable change in the course-taking patterns of American high school students. Combining the transcript data and the LTT data, we see that our high school students are completing a greater number of more difficult math courses.

Returning to figure 3, the bottom line shows that not only are high school students taking harder math courses, but high schools are also judging that they are doing better–since 1990, their average math GPA has increased by 18 percent from less than 2.2 to over 2.6.

Yet, increasing rigor in course taking and increasing grades do not square with the data from LTT or PISA showing that high school students have not improved their math knowledge or skills. The final piece of evidence showing the disconnect between the apparent increase in rigor and the lack of learning again comes from the LTT data.[5] Figure 5 shows the average NAEP score of students at each level of math-course completion. Students who stopped at Algebra I, geometry, and Algebra II all scored lower on NAEP in 2008 than the students enrolled in the same courses in 1978. The only bright spot is that students completing calculus now do about as well as their peers from thirty years ago.

Mitigating Circumstances

There are at least four mitigating factors that should be considered as we look at this disconnect. First, there is some debate about the validity of NAEP assessments for older students revolving around their motivation to take these exams seriously. While nine- and thirteen-year-old students take most exams seriously, jaded seventeen-year-old students may be bored with tests and, the argument goes, will not take a low-stakes test such as NAEP seriously. The National Center for Education Statistics (NCES) has conducted studies on this point and, while there is evidence that older students may be less motivated to take the exams compared to younger students, there is no consistent evidence on the impact of motivation on scores. There is also no evidence whatsoever that motivation has become a more serious issue in the last few years, during which time scores have remained flat.

Second, there is evidence of what is known as Simpson’s paradox. LTT reports scores for three major racial and ethnic groups–whites, blacks, and Hispanics–going back to 1973. In each group, we find increases in scores, but there has been no overall increase. Here is the paradox: how can all three of the major student groups increase their performance while the overall scores remain flat? The answer is in the declining proportion of higher-performing white students in the overall student population coupled with the rapid increase in the lower-performing Hispanic population. (The proportion of the K-12 population that is white declined from 78 percent in 1973 to about 57 percent in 2006 while the Hispanic proportion increased from 6 percent to around 20 percent. The black proportion has remained flat at around 15 percent throughout this period.) Thus, one could argue that had the American population not changed, the overall scores of our seventeen-year-old students would have increased. But the American population did change, and it changed throughout the K-12 system. Despite these changes, which affected schools at all levels, the LTT scores for nine- and thirteen-year-old students did increase, but our high schools failed to meet the challenge of America’s changing demography.

Third, there is what I will call the “tsunami” excuse–the forces pushing high schools to engage in an illusion of rigor were overwhelming. If policymakers decide that a mark of a successful high school career is completion of Algebra II, then schools enroll more students into a course called Algebra II. But not all math courses are equal–and it is easier to rebrand courses and still teach low-level math than it is to increase the rigor of math instruction.

Finally, we must note the role of the states. States have the authority to determine curriculum and set standards to which individual schools should conform. Both NCES and the Thomas B. Fordham Foundation have demonstrated how states set “proficiency standards” that are often below NAEP’s basic cut point.[6] Secretary of Education Arne Duncan has called this “lying to children and their parents because states have dumbed down their standards.”[7] In short, even as a wide swath of policymakers have demanded more rigor from high schools, most states have not stepped up to keep their high schools on track in delivering a more rigorous math curriculum that works.

More Math Courses, More Rigor, and No More Success

Here then is the evidence: high school students are taking more math courses with titles that imply more rigor. More of our high school students are getting through Algebra II and calculus, while fewer and fewer of them are stopping at general math and Algebra I. And transcript data show that even as they take more difficult courses, they are earning higher grades. All of these indicators reflect what schools claim to be delivering.

Objective indicators, however, show that the learning being delivered is faulty: LTT data show that, overall, while the math skills of elementary and middle school students entering high schools have improved, what American high school students know and what they can do in math have barely changed over the course of thirty years and not at all over the last fifteen. And when we step outside the United States to compare our high school students to students in other advanced industrial countries that are our peers and our competitors, the picture is also grim.

A delusion of rigor? As attorneys say: res ipsa loquitur–the thing speaks for itself.

Mark Schneider (mark.schneider@aei.org) is a visiting scholar at AEI and a vice president at American Institutes for Research.

Notes

1. U.S. Department of Education, National Center for Education Statistics (NCES), The Nation’s Report Card: Long-term Trend 2008 (Washington, DC: Department of Education, April 2009), available at http://nces.ed.gov/pubsearch/pubsinfo.asp?

pubid=2009479 (accessed September 23, 2009).

2. For more information, see U.S. Department of Education, NCES, “Program for International Student Assessment (PISA),” available at http://nces.ed.gov/Surveys/PISA (accessed September 23, 2009).

3. These data come from the 2005 report. See U.S. Department of Education, NCES, “NAEP High School Transcript Study,” available at http://nces.ed.gov/nationsreportcard/hsts (accessed September 23, 2009).

4. Each Carnegie credit represents 120 hours of classroom instruction.

5. These data were generated using the National Assessment of Educational Progress (NAEP) Data Explorer. See U.S. Department of Education, NCES, “NAEP Data Explorer,” available at http://nces.ed.gov/nationsreportcard/naepdata (accessed September 23, 2009).

6. U.S. Department of Education, NCES, Mapping 2005 State Proficiency Standards onto the NAEP Scales (Washington, DC: Department of Education, June 2007), available at http://nces.ed.gov/nationsreportcard/pubs/studies/2007482.asp (accessed September 23, 2009); and John Cronin, Michael Dahlin, Deborah Adkins, and G. Gage Kingsbury, The Proficiency Illusion (Washington, DC: Thomas B. Fordham Institute, October 2007), available at www.edexcellence.net/detail/news.

cfm?news_id=376 (accessed September 23, 2009).

7. Maria Glod, “46 States, D.C. Plan to Draft Common Education Standards,” Washington Post, June 1, 2009.

Community College Completion Will Pay Off

  According to the Department of Labor jobs for community college graduates will grow at double the national average over the next decade. This is faster job growth than for 4 year graduates. Some fields like computer support, nursing, and green jobs are examples of this trend for community colleges. But community college completion rates are low- around 25 to 35% , so it is unclear whether the supply of graduates will materalize. Their are waitng lists for courses in many 2 year colleges ,along with student aid cutbacks. Student services are being slashed, and financial incentives encourage colleges to enroll students but not completion.

How To Improve State Policy For Community College Developmental Education

Achieving the Dream, a national initiative to improve student success in community colleges, recently released a policy brief entitled Setting Up Success in Developmental Education: How State Policy Can Help Community Colleges to Improve Student Outcomes.  Remediation is well in excess of 60% of students in community colleges, so the need for new state policy is urgent.

In the report, Michael Collins describes how fifteen Achieving the Dream states are focusing on four key policies to improve outcomes for developmental students:

·         Prevention Strategies

·         Assessment and Placement

·         Implementation and Evaluation of Program Innovation, and

·         Performance Measurement and Incentives

The full report is available at:   http://www.jff.org/publications/education/setting-success-developmental-education-/839