My Work in Reading and Language Arts Assessment
For 25 years I was a curriculum coordinator in the Ontario, Canada, public school system. I worked first for a small rural school board in the early days of local curriculum development and implementation and later in one of the largest school boards in the province. The work I did in those years involved me in curriculum development, implementation, and assessment -- the classical cycle of curriculum work. As it happened, the assessment component turned out to be particularly important for me.
The focus of my master's thesis (Worsnop, 1980, 1996a) was the development of a technique later to be called retrospective miscue analysis, a procedure that teaches youngsters with reading problems to understand the reading process and then to analyze their “miscues” -- instances when they say or read something that is different (usually an omission, an insertion, or a change) from what is on the page. This understanding and critical ability leads to improved confidence in the reader, and consequent improvements in reading ability. Several years before I completed my master's degree, Goodman and Burke's (1972) Reading Miscue Inventory (RMI) had begun to lead those using it as an assessment tool to look upon reading as a process of unlocking meaning from print, rather than exclusively of pronouncing the words accurately and in order. The RMI offered teachers and researchers a procedure for analyzing the miscues students made when they read, and determining what sort of instruction would be helpful to improve their efforts.
My own work with the inventory revolutionized the way I looked upon reading and led me to speculate if students could not benefit from the insights the RMI offered to their teachers. I set up a series of experiments that eventually demonstrated students' ability to improve their reading significantly by coming to understand that
The big lesson I learned from all this was that assessment is not much good unless it leads to improvements in learning.
In my later career as a curriculum coordinator, part of my duties involved visiting schools and performing on-site assessment of programs in subjects for which I was responsible: English and language arts, drama, and media education. I would spend a week in a high school or group of schools and read all the written programs, comparing them both to the examinations and to provincial curriculum documents. I also visited classrooms and interviewed teachers and students at every grade from junior high to high school. Among the lessons I learned was the importance of making the test congruent with the course. I also learned that wording an examination question was a skill that many people take for granted. Poorly structured examinations can lead to weak assessment, with students as the major victims.
Years later, I got the chance to be involved in some provincial-level writing initiatives. In the late 1980s, Ontario decided to hold a provincial review of writing in Grade 12. I worked on that project as a marker, and had my eyes opened to a number of issues in writing assessment that today seem obvious. Shortly thereafter I was appointed to a team charged with designing a province-wide test in reading and writing for all Grade 9 students. It was during this project that I began to apply all that I had accumulated as a set of beliefs about assessment:
Our test-development team struggled with these issues as well others associated with holistic versus analytic assessment, scales that emerge from observation of work samples versus those that are developed externally, instrument design as a “quick and dirty” test or as an authentic unit of curriculum, reliability, validity, authenticity, and fairness.
I next became involved in a project that stretched my understanding of assessment even further. My school board decided to inaugurate a portfolio assessment project. Judith Fine, a psychometrist, and I were assigned responsibility for developing an instrument for assessing writing at the elementary level, beginning at Grade 3. The Peel Writing Scales (Peel District School Board, 1995), the major product of the project, became a respected assessment instrument adopted by other boards and districts because of the rigor of its development, its reliability, and its validity. The rigor came from the fact that we operated with the kinds of controls one would expect of a research project. The validity came from two sources: (1) the scales were developed by practicing teachers rather than by an academic (or political) team working at a distance; (2) the standards described in the scales emerged from minute observation of students' writing, rather than being arbitrarily decided on somewhere else. The reliability came from the fact that we worked with an analytical scale with enough detail to keep assessors close to the standard in their assessments. The package also contained anchor papers at each grade level, rather than at only one or two, and an Instructional Practices Handbook based on the performance indicators of the scale that helped make the materials friendly and useful to teachers as a way of organizing their writing instruction and their writing assessment. When the province of Ontario later decided to test reading and writing in Grades 3 and 6, the procedures and instruments used showed a remarkable resemblance to both the Grade 9 test of reading and writing and to the Peel Writing Scales.
In 1995, just as the writing scales were taking on their final form (in their first edition), I retired from my work in the school board. In retirement I have been spent much time training teachers to use the scales. I also worked on the development of the assessment model for writing in the first administration of Ontario's Grade 3 test of reading and writing.
Reading Online, www.readingonline.org
Posted November 2000
© 2000 International Reading Association, Inc. ISSN 1096-1232