Linda Adler-Kassner is professor of writing and director of the writing program at University of California, Santa Barbara, where she teachers undergraduate and graduate writing courses. Through 2011, she is also the President of the Council of Writing Program Administrators and served an additional five years on the Executive Board of that organization, including working as founding coordinator of the WPA Network for Media Action. Most recently, her research has focused both on analyzing the implications of public policy for writing instruction, and on developing strategies for writing instructors and program directors to affect policy. She is also author of The Activist WPA: Changing Stories about Writers and Writing (Utah State University Press, 2008), and co-author or co-editor of five additional books. She also has published many articles and book chapters in journals such as College Composition and Communication, WPA Journal, and Journal of Basic Writing.

Peggy O’Neill, associate professor, directs the composition program and teaches writing and rhetoric courses in the department of writing at Loyola University Maryland. Her scholarship focuses primarily on writing assessment and pedagogy. Her work appears in academic journals, such as the Journal of Writing Assessment and College Composition and Communication, as well as in several edited scholarly collections. Her most recent book is A Guide to College Writing Assessment (Utah State University Press, 2009), which she co-authored with Cindy Moore and Brian Huot.  O’Neill has also edited or co-edited four books: Assessing Writing: A Critical Sourcebook (Bedford St.Martin’s / NCTE 2009); Blurring Boundaries: Developing Writers, Researchers and Teachers (Hampton Press 2007); A Field of Dreams: Independent Writing Programs and the Future of Composition Studies (Utah State University Press, 2002);and Practice in Context: Situating the Work of Writing Teachers (NCTE, 2002).

REFRAMING WRITING ASSESSMENT TO IMPROVE TEACHING AND LEARNING

By LINDA ADLER-KASSNER PEGGY O’NEILL

UTAH STATE UNIVERSITY PRESS

Copyright © 2010 Utah State University Press
All right reserved.
ISBN: 978-0-87421-798-8

Contents

Acknowledgments………………………………………………………vii1 Higher Education, Framing, and Writing Assessment………………………12 Framing (and) American Education……………………………………..133 The Framing of Composition and Writing Assessment………………………404 Reframing Strategies and Techniques…………………………………..815 Reframing in Action…………………………………………………1106 Reframing Assessment: Why Does My Participation Matter?…………………1457 Reimagining Writing Assessment……………………………………….179References…………………………………………………………..192Index……………………………………………………………….205About the Authors…………………………………………………….208

Chapter One

HIGHER EDUCATION, FRAMING, AND WRITING ASSESSMENT

Consider the following scenario, discussed on the Writing Program Administration listserv (WPA-L). The scenario is based on the experiences of a writing program administrator at a large midwestern university:

The writing program director learns that “there is a movement afoot” at her university to administer the Collegiate Learning Assessment (CLA) to first-year students and seniors. This will mean that these students will take a ninety-minute essay exam designed to “test” their critical thinking skills. The tests results will be published so that her institution can be compared to others in its category and, if necessary, used to improve any weaknesses that are identified.

In listening to the conversations on campus, this program director feels there is an implicit message that the test would be a way of marketing the school as a “first-rate institution.” Although no one explicitly discusses the CLA as an assessment of writing (instead, they say, it is an indication of critical thinking skills), she feels strongly that it will reflect on the writing program.

In response to what she is learning as she listens to the discussions on campus, the program director turns to the national community of writing professionals on the WPA-L to get background information for her upcoming meeting with the university assessment committee. She learns that the test is just one indicator the school wants to use to demonstrate “critical thinking”—although the other indicators were never articulated, at least not to her. After the meeting, she writes a memo to the committee and administrators outlining her concerns based on her knowledge of writing pedagogy, assessment, and the curriculum at her institution. The memo outlines the disadvantages of the test and the possibilities of developing an in-house assessment. This type of assessment, she argues, would better serve the needs of the local institution and would be more likely to improve teaching and learning. By her own admission, she doesn’t know if this detailed memo (almost three pages long) will do any good. However, much later, she learns that the CLA was abandoned, but she doesn’t know why. She “heard that one of the administrators who was involved in this mentioned” that her “memo was ‘unhelpful.'” None of her suggestions for a locally designed assessment that would track students across their undergraduate careers was adopted—nor was it even discussed seriously at the time, she admits. In her words, the CLA “threat” just went away. In the end, she has ideas about the motivation behind the initial move, but she has no concrete evidence. The writing program does do ongoing program review, and it has always been praised by the administration for that review. As the program director, she wasn’t aware of the conversations about using writing as way to assess critical thinking at the university level, and she wasn’t brought into the deliberations. She did, however, react to the news and provide her perspective as the program director and a composition scholar.

This example, which has been repeated in various permutations in listserv discussions, in hallway conversations at conferences, and on campuses across the country, illustrates the dilemmas and questions that can emerge around assessment. While the example reflects the experiences of many writing instructors, program directors, department chairs, or other writing professionals, it represents just one type of assessment—institutionally based, university-wide assessment—about which we hear fairly often. As the example also illustrates, the extent to which we writing professionals are “included” (asked or invited to participate; directed to provide input) in these discussions varies, sometimes from moment to moment or day to day.

We know that all postsecondary educators live in a culture where assessment is increasingly important. From the nearly ubiquitous process of program review to assessments (also) linked to accreditation to institutional participation in initiatives like the Voluntary System of Accountability (VSA, which might involve administration of one of three standardized tests: the Collegiate Assessment of Academic Progress [CAAP], Measure of Academic Progress and Proficiency [MAPP], or the Collegiate Learning Assessment [CLA]; or might involve use of the AAC&U’s VALUE rubrics), the sheer number of efforts—and acronyms—surrounding assessment can be at best incredibly confusing, and at worst positively dizzying. Whether a classroom instructor, a course director, someone who works with a writing program, or someone who serves in some other teaching and/or administrative capacity on campus, it’s likely that all of us have dealt with some kind of assessment scenario in our courses, programs, or institutions. Often these writing assessments are locally designed and administered, but they can be part of the alphabet soup that marks the national standardized testing marketplace. Conversations also frequently include exams more directly connected to writing programs, such as the SAT Writing Test, ACCUPLACER, COMPASS , E-rater, or even AP or CLEP exams.

Regardless of the end point or process used, three very broad questions can be asked about assessments:

1. What is the purpose of an assessment?

2. Who is (or are) the primary audience(s) for an assessment and its results?

3. What decisions might, or will, result from an assessment?

Within these broad questions, a number of others are also present: What assessments are appropriate, and why? What will be the relationship between assessment and curriculum? Who will determine what assessment(s) will be used, when, and how? How will the assessment results be interpreted, communicated, and used? More importantly, how do faculty—whether classroom instructors, directors of writing programs or centers, department chairs, or any other interested faculty—get involved in these discussions earlier than we so often do, hearing about them only as they drift down through campus administrators or conversations?

With so many different processes, programs, tests, and options, these discussions can be overwhelming and confusing. However, we believe they are currently not just important, but the most important discussions happening on our campuses (and even beyond them) today. They affect everything about our courses and programs—who is admitted to them; how they are taught; how students, courses, and instructor(s) are evaluated; what counts as valuable in them—in essence, everything that motivates writing instructors to do the work we do. For this reason, it is also crucial that postsecondary writing instructors and program directors get involved in assessment conversations, especially those that are explicitly about writing assessments (or are being used to make judgments about writing programs and student writing abilities).

This book is intended to help readers with that involvement. But as the vignette opening this chapter illustrates, “getting involved” isn’t simply about administering a test or grading portfolios. It is about becoming immersed in discussions about teaching writing and learning to write, about who is doing the teaching and the learning, and ultimately, about how we define these activities. Increasingly, it is also about participating in an ever-widening series of conversations not just about composition but also about how people inside and outside the academy understand the purpose(s) of education, what those purposes have to do with visions of what it means to be a twenty-first-century citizen, and how all of this comes back to what students should learn—in postsecondary education generally, and in writing classes more specifically. Writing assessment, in other words, cannot be separated from these larger issues because these larger, contextual issues determine—whether directly or indirectly—much of what happens in postsecondary institutions and our programs and classrooms. Mike Rose makes this case with typical eloquence when he notes that “[i]t matters a great deal how we collectively talk about education, for that discussion both reflects and, in turn, affects policy decisions about what gets taught and tested, about funding, about what we expect schooling to contribute to our lives. It matters, as well, how we think about intelligence, how narrowly or broadly we define it” (2009, 5).

What we’re saying, then, is that those discussions that we as two- or four-year college writing teachers or program directors might have about assessment—with other instructors in our program, with colleagues in our department, with campus administrators, or with people outside of our institutions—aren’t just about writing assessment. Instead, they exist within an ever-expanding galaxy of questions (represented in Fig. 1) about what people need to know to “be successful” in the twenty-first century. These questions are inexorably linked to other items that populate the galaxy, as well—ideas about what it means to be a part of “America” as a country; ideas about how America develops as a nation and what is necessary for that to happen; ideas about how the nation’s youth become “productive” citizens and what “productive” means.

Many, many different voices in this galaxy are attempting to weigh in on these discussions. Together, they embody the intrinsic relationship—sometimes implied, sometimes explicitly stated—between education and the nation’s progress. Since the mid-nineteenth century (and even before), the idea that schooling should prepare students for participation in the broader culture has been a taken-for-granted purpose of education. Horace Mann, the “father” of the common school movement, reiterated this point frequently in his Lectures on Education, where he outlined four “powers and faculties” that education needed to take into account: a “moral” conscience shaped by allegiance to God; “social or sympathetic” sensibilities that came from ties to community; linkages to family and family traditions; and a self-focused set of behaviors (1891, 126–127). The role of education, in Mann’s estimation, was to simultaneously appeal to these sensibilities and enable the nation’s youth to shape them into behaviors that would advance what he saw as the nation’s divinely inspired and fueled sense of mission.

Of course, literacy—both reading and writing—has long been considered a critical component of education. While it might seem an act of hubris on our part to situate writing instruction at the center of these ongoing conversations—after all, we are writing teachers and researchers ourselves—in fact, that is where policy reports and discussions about education place it, too. As a subject that is considered central to students’ successes when they enter college, during postsecondary education, and into careers, writing is often a centerpiece of discussions about (and, sometimes, a prescription for) what should happen in secondary and postsecondary classrooms in order for students to be prepared for next steps. Influential reports like the American Diploma Project’s Ready or Not: Creating a High School Diploma that Counts, released electronically on the Achieve Web site in 2004, include dire proclamations about students’ writing, like the warning that “[m]ore than 60 percent of employers question whether a high school diploma means that a typical student has learned even the basics, and they rate graduates’ skills in grammar, spelling, [and] writing … as only ‘fair’ or ‘poor'” (3). A discussion paper outlining the parameters of the Voluntary System of Accountability (VSA) makes virtually the same claim: “American companies report that many college graduates were lacking in … basic communication skills,” including writing, and that 57 percent of “the public” said that “top-notch writing and speaking ability” is ” ‘absolutely essential’ for higher education” (McPherson and Shulenberger 2006). These reports illustrate a refrain we often repeat when we talk about writing assessment with colleagues: writing is everyone’s business. Everyone, however, doesn’t agree on how to define “writing,” let alone “good writing” or “writing ability.” Likewise, everyone doesn’t agree on what to teach or how to teach when it comes to writing. More importantly, at least from our perspective, not everyone has the same expertise and the same investment in writing instruction.

In other words, although we may all agree that writing “is not a frill for the few, but an essential skill for the many,” as the National Commission on Writing in America’s Schools and Colleges argues in its 2003 report, The Neglected R: A Need for a Writing Revolution, we don’t necessarily agree on much beyond that. Employers, policymakers, and educators bring different experiences, expertise, agendas and commitments to the conversation. Within these broad groups, there are different perspectives that can represent disparate, even contradictory, views. The debate over reading instruction that swirled around Reading First legislation (and that ultimately played into No Child Left Behind, or NCLB) in the late 1990s, for instance, illustrates the high-stakes nature of these debates. The definition of what counted as “research”—”scientific study backed by quantitative data”— privileged phonics-based reading instruction (and research validating that approach) while discounting whole language instruction and research. While less high stakes (for now), a similar tension can be found in writing instruction, where there are often disagreements about the role of explicit grammar instruction in teaching writing. While most composition scholars trained in composition and rhetoric may agree that teaching grammar in isolation is not effective in improving students’ written texts, there are many, many writing instructors who have not had the kind of graduate education that supports this approach. Even if official positions by professional organizations do not endorse grammar exercises as a best practice, many writing instructors do in practice and in public venues. In other words, within the population of those who identify as writing instructors, there is diversity of positions and practices.

So if “everyone” (and that’s a big word, we realize) thinks writing is their business, how can we as college-level educators and scholars interact with them around these issues of writing assessment—issues, as we have indicated, that are related to considerably larger concerns and purposes, and about which there is sometimes considerable tension? That is the main question we take up in this book. Our position is that writing instructors and program directors know a lot about writing instruction and need to be centrally involved in discussions about writing assessment. Although not all writing professionals may agree on the particulars, by engaging in conversations about what we value, the purpose of education, and teaching practices, we can clarify some of the issues at the core of the assessment debate. We also believe the process we use to enter into and carry out these conversations is absolutely crucial and incredibly valuable. This book, then, is not an argument for taking a particular position in these discussions. Instead, it is an argument for getting involved—for working strategically, on our own and with others in our programs, institutions, and communities—in discussions about writing and writing assessment.

In writing this book we have two main purposes. First, we aim to convince readers—as individual professionals and as members of a profession—that we need to engage in these discussions about writing pedagogy and assessment wherever they might occur. Second, we provide readers with the necessary knowledge for engaging in these discussions. Our intention is not to tell readers what postsecondary instructors or administrators should say about writing instruction and assessment but to provide useful background knowledge and strategies for productive engagement in these discussions. Because we believe writing assessment is a powerful form of communication, we illustrate the strategies with real-world examples of writing program administrators and scholars engaging in writing assessments. We also report on professionals involved in writing assessment projects that move beyond their own campuses and into the broader context of higher education.

(Continues…)

Excerpted from REFRAMING WRITING ASSESSMENT TO IMPROVE TEACHING AND LEARNINGby LINDA ADLER-KASSNER PEGGY O’NEILL Copyright © 2010 by Utah State University Press. Excerpted by permission of UTAH STATE UNIVERSITY PRESS. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.