image3.png

image4.png

image1.png

image2.png

V98 N6 kappanmagazine.org 47 Thinkstock

loRRie a. ShepaRd ([email protected]) is distinguished professor of education, WilliaM R. penuel (@bpenuel) is professor of learning sciences and human development, and kRiSTen l. daVidSon is a postdoctoral research associate, all at the University of Colorado Boulder, School of Education.

by lorrie a. Shepard, William R. penuel, and kristen l. davidson

When Congress passed the Every Student Succeeds Act (ESSA) in December 2015, it carried forward many of the same testing re- quirements that existed under No Child Left Behind (NCLB). But at the same time, it softened the consequences, taking away the federal government’s power to determine what will happen to schools that fail to meet specifi c testing goals. From now on, states and districts can decide for themselves what achievement targets to set, and they can choose to focus on needed supports instead of sanctions for their lowest-performing schools.

Free from strict Adequate Yearly Progress accounting, many state and district education leaders are exploring new ways to collect in- formation about student learning as well as new ways to use that information. Where NCLB mandated testing strictly for summa- tive purposes (i.e., to judge how well students, teachers, and schools have performed), ESSA permits and even provides some funding to encourage states to develop balanced assessment systems. Such sys- tems encompass not just summative tests but also local, formative assessments, which could include curriculum-embedded assessments designed to provide teachers with insights about instructional sup- ports that are needed.

Rather than being led by national testing mandates, state and local leaders should design balanced assessment systems guided by coherence, research on learning, and attention to equity.

R&D

R&D appears in each issue of Kappan with the assistance

of the Deans Alliance, which is composed of the deans of

the education schools/colleges at the following universities:

George Washington University, Harvard University, Michigan

State University, Northwestern University, Stanford University,

Teachers College Columbia University, University of California,

Berkeley, University of California, Los Angeles, University of

Colorado, University of Michigan, University of Pennsylvania,

and University of Wisconsin.

Design principles for new systems of assessment

48 Kappan March 2017

texts. And let’s say that school system leaders intro- duce assessments that are designed to help teachers gauge the kinds of support their students need in order to reach this standard. We would judge this system to be horizontally coherent if students are also given classroom assignments that call upon them to make well-reasoned arguments and if teachers use practices (such as asking complex questions as prompts for classroom discussions) that have been found to help students strengthen their own logic and use of evidence.

If, in turn, classroom assignments, course grades, and external accountability tests routinely ask stu- dents to demonstrate such reasoning, then the as- sessment system would be vertically coherent as well. In short, all parts of the educational system, including curriculum, instruction, and assess- ments of all kinds, ought to be working toward the same goals, helping move students toward shared definitions of what they ought to know and be able to do.

Unfortunately, the testing mandates of the past two decades have only made things less coherent (or coherent but not meaningful, insofar as they have fostered a teaching-to-the-test approach, aiming toward narrow curricular goals). Even in the face of those mandates, many teachers have experimented with various kinds of formative assessment, from exit slips to self-assessments, multidraft writing projects, and others. But in most cases, they have done so with- out much support from their districts or guidance about which approaches are most strongly grounded in research. And while these teachers’ uses of forma- tive assessment may have delivered a strong mes- sage about the kinds of learning they value, their students have likely received very different messages from the high-stakes standardized achievement tests they have had to take.

Given new flexibility under ESSA, districts and schools now have an opportunity to design and implement coherent systems, with formative as- sessment having a more prominent role. To do so effectively, however, educators will need some back- ground knowledge about the research that supports the main approaches to formative assessment. Thus, in the next section, we offer a condensed summary of four key assessment models (Penuel & Shepard, 2016), with a focus on their underlying theories of learning.

grounding assessments in a model of learning

In order to be coherent, an assessment system should be based on a shared model of learning. It won’t be effective, though, unless that underlying model of learning has a valid basis. And it won’t be equitable unless it includes curricular supports for students and

The question is: What guiding principles would help ensure the quality of these new, balanced as- sessment systems? Drawing on lessons learned over three decades of research and reform, we argue that state and local leaders should take the lead in design- ing new assessments guided by two core principles: First, make assessments coherent, integrating them with rich curriculum and effective instruction; sec- ond, ground this integration of curriculum, instruc- tion, and embedded assessments in equity-focused research on learning.

building coherent assessment systems

The idea of building a coherent system of assess- ments “from classroom to state” was first advanced in a National Research Council committee report, Knowing What Students Know (Pellegrino, Chu- dowsky, & Glaser, 2001, p. 9), which synthesized findings from contemporary research on both learning and educational measurement. Whether an assessment is meant to be used in a classroom or for state accountability, it should assess what is truly valuable for students to learn, such as core ideas and key skills from the various content areas (p. 248). By contrast, many classroom worksheets and multiple-choice tests that mimic state exams have reflected a negative kind of coherence, re- quiring students to answer superficial questions or recall simple facts.

Further, the report explained that assessments should be coherent not only vertically (i.e., the same standards and learning goals drive assessments at both the classroom and state accountability levels) but also horizontally. Horizontal coherence, at each level of the system, refers to the conceptual integra- tion of assessments with a shared model of learn- ing. At the state level, this means that accountabil- ity assessments must fully embody learning goals envisioned by standards. At the district level, as- sessments must be coherent with standards, cur- ricula, and professional development. And at the classroom level, horizontal coherence requires that assessments be so thoroughly integrated with cur- riculum and instruction that the insights they pro- vide can immediately be put to use. Thus, class- room formative assessments must be built on much more fine-grained models of learning than state- level tests. In order to provide the kinds of specific feedback and instructional supports that students need at intermediate stages of development, teach- ers need research-based tools that are attuned to the very specific ways in which student understanding develops in each academic domain.

For example, let’s say that a state or district stan- dard specifies that students must learn how to make and defend reasoned arguments about informational

V98 N6 kappanonline.org 49

Join the conversation

facebook.com/pdkintl @pdkintl

#1. Data-driven decision making

Data-driven decision making is most accurately portrayed as a policy theory of action. It relies on no specifi c model of learning but, rather, draws its in- spiration from theories of organizational change (Deming, 1986; Senge, 1990). The idea is that edu- cators should set specifi c learning goals, use interim or benchmark assessments (sometimes marketed as “formative assessments”) to check student progress toward reaching them, fi nd new teaching strategies to address areas of weakness, and continue to moni- tor student progress over time.

Data-driven decision making assumes that teach- ers will know how to help students — or will seek training that shows them what to do — if the interim tests reveal that students are struggling. But this as- sumption has never been supported by empirical re- search fi ndings. And researchers have found this ap- proach to be especially ineffective in low-performing schools that tend to lack the capacity to adapt in this way (Elmore, 2003).

To date, most of the research on data-driven deci- sion making has focused on the work of data teams (groups of educators tasked with analyzing test re- sults). Findings show that, at best, such teams are able to identify which students are the most in need of help and which objectives are most in need of reteaching (Shepard, Davidson, & Bowman, 2011). However, because interim assessments offer little to no insight into the reasons why students are underperforming or how to help them, their use hasn’t been found to lead to improvements in teaching or learning.

Further, data-driven decision making sometimes goes hand-in-hand with the use of extrinsic rewards and punishments to pressure students to improve. For example, teachers or administrators might post the results of interim tests in the hallway, letting ev- erybody know who’s on track and who still needs to get, say, three more items right to reach profi ciency. Yet, research has largely discredited this approach to motivating young people to learn. When students are struggling, being told how far behind they are doesn’t help them move ahead. Moreover, the re- sponse to identifying students in need of more sup- port has often been to create “pull-out” programs for such students, rather than promoting more equitable teaching in the regular classroom. Instead, students need meaningful opportunities to engage with the material, ask questions, try ideas, and receive useful guidance and feedback from teachers and peers.

#2. Strategy-focused formative assessment

Strategy-focused practices include various tools and techniques for engaging students in analyzing and improving their own work, such as ways to pose questions that invite classroom discussion about on-

adequate preparation of teachers to help students meet learning goals. Its goals must be compelling, but those goals must also be reachable, and they must feature teaching practices that are consistent with what is known about student motivation, identity formation, and cognitive development.

In an early and widely infl uential review of the research on formative assessment, Black and Wil- iam (1998) identifi ed a number of distinct lines of scholarship in this area. But while they noted that differing approaches to formative assessment relied on very different learning models, they did not of- fer a way to integrate those perspectives. Nor did they show how disparate ideas about motivation, self-assessment, mastery, the giving of feedback to students, and other issues could be integrated into a coherent whole.

Since that time, many researchers have described formative assessment as though it were a single, co- herent practice, without recognizing that the label refers to varied learning goals and theories that are not necessarily compatible with each other because they draw upon very different conceptual models. Here, we call out those differences, describing four distinct approaches that have been promoted as “for- mative assessment.” We argue that the latter two perspectives hold the greatest promise for support- ing more ambitious and equitable, next-generation visions of teaching and learning, but we also point out that each of these approaches has limitations.

All parts of the educational system ought to be working toward the same goals, helping move students toward shared defi nitions of what they ought to know and be able to do.

50 Kappan March 2017

content area and help them refl ect on and understand what it means to become truly profi cient in that area.

#3. Sociocognitive formative assessment

Sociocognitive approaches are meant to assess stu- dents’ understandings and skills as they participate in increasingly sophisticated practices common to disciplinary experts. Further, because thinking and learning are presumed to be fundamentally social activities, assessment is grounded in “local instruc- tional theories” of learning, whereby a sequence of instructional activities is devised to support the par- ticular group of students in developing profi ciency (Gravemeijer, 2004).

Instructional sequences are typically based on either a “learning progressions” (or “trajectories”) approach, which aims to help students move toward specifi c dis- ciplinary goals (Simon, 1995; Smith et al., 2006), or a “knowledge-in-pieces” (or “facets”) view, in which learning is seen as a less orderly process, and more attention is paid to the specifi c problems students are trying to solve that require use of disciplinary knowl- edge (diSessa, 1988). In both cases, assessment materi- als are designed for the particular content area, with attention to the challenges that students typically face when studying the given material, as well as common approaches to helping them move forward.

In addition to gauging students’ progress in mas- tering content knowledge, sociocognitive strategies aim to help them take on the dispositions and iden- tities of the given fi eld. Thus, this approach tends to favor assessment practices (such as collaborative inquiry, expertly facilitated questioning and discus- sion, and qualitative feedback) that allow teachers to pay attention to how students are (and are not yet) acting, thinking, and reasoning in disciplinary ways.

One example that has been found to be effective is the Inquiry Project, a three-year sequence of instruc- tional units designed to progressively build upper- elementary students’ understandings about the na- ture of matter (Smith et al., 2006). Another example, designed for the middle grades, is the Contingent Pedagogies project (Penuel et al., 2017), which gives teachers specifi c questions with which to elicit stu- dents’ ideas about the physical world, as well as dis- cussion prompts meant to get students talking about ideas and methods that are central to the study of Earth science.

Two key strengths of the sociocognitive approach to assessment are its discipline-specifi c learning goals and its well-articulated learning theory (Penuel & Shepard, 2016). Rather than telling students how many correct and incorrect answers they got on a test, the point is to reveal how they think about and try to solve specifi c problems that have been chosen precisely because they relate to key concepts in the

going projects, guidelines for assigning students to revise papers, and rubrics for self- and peer-assess- ment.

Evidence suggests that when teachers have mean- ingful opportunities to learn and try such techniques, they can become more skilled at creating classroom environments in which students assume an active role in their own learning. One well-known example of this approach, the King’s-Medway-Oxfordshire Formative Assessment Project — in which students were taught to assess and build upon their ideas, identify their own sources of intrinsic motivation, and monitor and regulate their own learning — was found to have signifi cant and positive effects on stu- dents’ engagement and academic progress (Black et al., 2003).

It’s important to note, though, that strategy-fo- cused approaches are not grounded in any particular theory of learning. Rather, they amount to a loose collection of all-purpose strategies by which teachers and students can assess their ongoing work. While those strategies could be used to help young people acquire a deep understanding of sophisticated aca- demic content, they can just as easily be used to pro- mote rote mastery of a shallow curriculum. When it comes to the goals of learning, the approach is agnostic and, for that reason, quite limited.

A much more effective way to practice formative assessment, we would argue, is to choose tools and measures that are connected to the specifi c fi eld and its goals and purposes. As the next two strategies demonstrate, the most powerful assessment tasks en- gage students in the genuine practices of the given

What guiding principles would help ensure the quality of these new, balanced assessment systems?

V98 N6 kappanonline.org 51

Join the conversation

facebook.com/pdkintl @pdkintl

given subject area. In turn, this gives teachers use- ful insights into what students already know, spe- cifi c ideas that confuse them, concepts they’ll need to learn right away, and other things to consider as they decide how best to teach the material.

However, developing these kinds of fi ne-grained, subject-specifi c assessment tools requires a lot of ex- pertise and resources. As a result, they are not yet available to many schools and districts, or for certain topics and grade levels. Further, since these tools are meant only to assess students’ understanding of par- ticular subject matter — and not to provide insight into their differing values, experiences, and personal goals — it remains to be seen whether their use will benefi t all students equitably. It could very well be the case that such tools are indifferent to students’ racial, ethnic, or gender identities, emerging bilin- gualism, conditions of poverty, and other important dimensions of students’ lives.

#4. Sociocultural formative assessment

Sociocultural interventions share with the socio- cognitive approach many of the same research-based premises about the social nature of learning and de- velopment, as well as a focus on student participation in disciplinary ways of knowing and doing. The two theories of learning diverge, however, in terms of how they account for students’ diversity. In short, sociocultural approaches more explicitly allow for students to engage with academic content and prac- tices through differing entry points and to follow differing pathways to mastery.

Sociocultural theories of learning recognize that

when students arrive at school, they bring with them important knowledge and interests that should in- form curriculum and instruction. Rather than cur- ricula or instructional practices that ignore students’ experiences, teachers ought to help students refl ect on how the school’s ways of knowing, doing, and be- ing relate to the practices that are valued in their own families and communities (Bang & Medin, 2010). A key purpose for assessment, then, is to elicit infor- mation about students’ experiences and help them relate their own interests and goals for learning to becoming a part of a disciplinary community.

One promising example is the Bellevue-University of Washington Curriculum Redesign Partnership, which takes units of study from the district’s elemen- tary science curriculum and repurposes them, alter- ing them in ways that give students more agency in the classroom and that tap into their diverse interests in the given topics. For instance, some units have been redesigned to build on students’ knowledge about their communities, such as by using photog- raphy to document the everyday lives of the people who live there (Clark-Ibañez, 2004). At the begin- ning of a unit on microbes and health, for example, students take photos of activities they do in daily life to prevent disease and stay healthy. Then they share these photos in class as a way to bring personally rel- evant experiences into the classroom as they launch the unit. Their documentation also helps shape a student-led investigation focused on students’ own questions, which are refi ned as students encounter key ideas in microbiology.

Using students’ interests, experiences, and knowl- edge is an important strategy in equitable instruc- tion. In order to become an integral part of balanced systems of assessment, however, sociocultural inter- ventions like the one implemented in Bellevue will need to be embraced by a wide range of stakeholders. To this end, educators and researchers need to com- municate with families and other educational stake- holders about the value of this approach to teaching and how its outcomes can be mapped onto familiar disciplinary standards and learning goals.

conclusion: Start with local curriculum and instruction

Under ESSA, states and districts have the oppor- tunity to build coherent systems in which formative assessments are codeveloped and integrated with lo- cal curricula, instruction, and professional learning — all of which are grounded in the same research- based model of learning.

It will also be critical to design for coherence be- tween local and state-level models of learning when possible or to acknowledge when a shared model of learning is not possible because of limitations in the

many state and district education leaders are exploring new ways to collect information about student learning, as well as new ways to use that information.

52 Kappan March 2017

(2003). Assessment for learning: Putting it into practice.

Buckingham, UK: Open University Press.

Black, P. &am