Using Think-Alouds to Analyze Decision Making During Spelling Word Sorts

Mary Jo Fresch

Introduction

In many language arts classrooms, spelling instruction includes word-analysis activities. Word sorting is often suggested because it fits well with theory about children's development of spelling knowledge (Gillet & Kita, 1980; Henderson, 1980, 1990; Sulzby, 1980; Templeton, Ganske, Invernezzi, Bear, Abouzeid, & Zutell, 1994; Zutell, 1996), as summarized in Table 1. This hands-on activity, with its focus on “active, thoughtful problem solving” (Zutell, p. 107), helps students compare and constrast various aspects of the language. Young spellers may begin sorting by matching pictures with letters of the alphabet. For example, a student may be given two sheets of paper, one marked “B” and one marked “D,” and 8 to 12 pictures of objects beginning with those letters. The child is then asked to categorize each picture by placing it on one of the pieces of paper. More challenging sorts might have students at later developmental levels categorize groups of words according to long or short vowel sounds, beginning blends, endings, or prefixes.

There are two types of sorts. With “closed” sorts, the teacher designates the categories, using key words or pictures; students then fit the words into the pre-established categories, describing how the words are similar or different. The “open” sort allows the student to determine categories. All categories are acceptable, as long as the student can justify them and describe the criteria by which words are assigned to one category and not another. Often, the student-designated categories are not what the teacher had in mind when he or she selected a particular set of words (Fresch, 2000).

We often look at the finished categories of an open sort in an attempt to determine how the children analyzed the words. Some children, for example, may sort by the last sound of the word, or by the s sound in the middle of several words, or by the silent e at the end of a word. There are, however, few studies that look closely at how students make decisions during the problem-solving phase of creating their categories. The study described here used students' think-alouds to provide a window on the process of sorting, gathering data from the children while they engaged in word sorting. The information gained from these think-alouds can be beneficial in tailoring instruction to meet individual students' needs.

Think-alouds have been used to collect systematic observations about higher level processes (Ericsson & Simon, 1984), improve children's metacognitive abilities (Baumann, Jones, & Seifert-Kessell, 1993), model and assess thinking of students who speak English as a second language (Mangubhai, 1990), and to “reflect on one's own cognitive processes” (Baker & Brown, 1984, p. 353). Olson, Duffy, and Mack (1984) believe that a thinking-out-loud task “is best used to study higher level processes” (p. 255) and is an effective way to “collect systematic observations about the thinking that occurs” (p. 256).

The think-aloud can provide powerful indications of cognitive processes in action. Verbalization can be concurrent (during performance of the task) or retrospective (immediately after the entire task or a portion of it is completed) (Ericsson & Simon, 1984). When thinking aloud is concurrent with performance, information held for current use in the subject's short-term memory is vocalized, thus giving the researcher insight into the subject's decision-making process. This insight can facilitate effective student-centered planning. The retrospective approach relies on long-term memory and reflects a more developed, but less traceable, path of decision making.

Concurrent think-alouds were used in this study to analyze how children approach the task of sorting words. This data-gathering approach seemed appropriate to the higher level thinking and frequent changes of mind that occur while a student is comparing and contrasting words during sorting. This classroom research focused on discovering how children established their categories, with the purpose of informing the design of future word-sorting activities (Fresch, 2000).

Back to menu

Method

Setting and participants. The study took place in a fifth-grade classroom in a suburban school in the midwestern United States, where a literature-based curriculum was in effect. The classroom teacher, Aileen Wheaton, and I have been involved in collaborative work in spelling for approximately 6 years.

Twenty-four students (aged approximately 10 years) took part in the study. The classroom included students of a range of ability levels, including children experiencing both educational and physical challenges. Word sorts were a primary means for exploring spelling in the classroom.

Procedure. Students were administered the Qualitative Inventory of Word Knowledge (QIWK) (Schlagal, 1989) to establish their instructional levels. This spelling inventory consists of eight lists of progressively more difficult words, selected for their frequency in written and spoken work at particular grade levels. Student attempts at spelling the words are analyzed for error patterns and indications of developing understandings. The test was administered at the beginning (September) and end (May) of the school year to identify developmental levels of knowledge (see Table 1, above, for complete descriptors). The test administered in May indicated that of the 24 students involved in the study, 2 were at the letter-name stage, 7 at within word, 12 at syllable juncture, and 3 at derivational constancy.

Aileen Wheaton began the school year by modeling sorting. After a few weeks of demonstration, the students were able to complete the activity alone or with a partner. For 6 months, lessons were observed to establish the classroom context for how sorting was learned and how the children handled the task. At the end of this period, immediately before research with the individual students began, one lesson was videotaped.

In March, each of the 24 students was videotaped individually while thinking aloud during two word-sorting activities. These sessions took place in a small, private room. A research assistant provided an explanation and demonstration of thinking aloud as a warm-up exercise (Ericsson & Simon, 1984). The assistant described thinking aloud as “putting your brain in your throat and telling us what you're thinking.” This statement, along with the reminder to “keep talking,” encouraged the students to verbalize their thinking as they sorted. For each word sort, students first read through a set of word cards to be certain of familiarity (defined as the ability to read the word aloud) and then sorted them into categories; unfamiliar words were removed. Sort 1 consisted of single-syllable words that demonstrated various spellings of the long i; sort 2 consisted of multisyllabic words that contained the short o and short a sounds (see Table 2).

Several weeks later, I interviewed the students individually as they watched their two videotaped sorting sessions. The interviews were also taped.

Back to menu

Data Analyses

Each student's September and May QIWK results were entered into a spreadsheet. Instructional levels (50-89% accuracy) were noted and related to a developmental spelling level (see Table 1), as suggested by Schlagal (1989). At the end of each videotaping session in March, each student's word sort was transcribed, and the categories were entered into a database. Aileen or I reviewed the tapes, and then exchanged them with each other. During our independent viewings, we each made notes about how students approached the sorting. What information were they using to create their word groups? At the end of each sort the student was asked to describe her or his groupings, which provided additional information about the approach taken. In addition, Aileen and I both assigned each student's approach to the word sorting to one of four categories:

1. Auditory
2. Visual
3. Auditory with visual check
4. Visual with auditory check

After Aileen and I watched the videos individually, we met in April to share our notes. We viewed the tapes a second time, with the four categories in mind. Inter-rater reliability was 92 percent for sort 1 and 96 percent for sort 2. In May, I used the interview sessions with the children to crosscheck the assignment of approaches. Categories for approach to sorting were matched against developmental level, as ascertained from the QIWK administered that month, for all 24 children (Table 3).

Back to menu

Examples of Student Thinking During Word Sorts

Students demonstrated a range of thinking during the sorting, as shown in the following video clips and accompanying transcripts of four students' think-alouds during word sort 1. These examples also show the great variance in categorization, based on students' developmental stage.

Lorraine

Lorraine, student 24 in Table 3, was an advanced speller who had reached the stage of derivational constancy. She appeared to note the long i sound while reading through the cards before sorting, and she observed what letter patterns made the sound. She worked first auditorily, then sorted by visual patterns. Her sorting was efficient and accurate. She established the following categories during the sorting activity, assigning words as indicated:

Video clip (1.6 MB)

Interviewer: OK, sort them into categories, and talk out loud. Tell us what you are thinking as you go through those. OK? Lorraine: Um, well, I'm just going to look at all them and see if I can find some...strategy. And the strategy I see is a long i...and, uh, some long i's I can see are made with a y -- and those [places “try,” “fly,” and “style” together] are the ones. And then some are i by itself, like “wise,” “nine,” “prize,” “twice,” and “smile.” And then others are i-g-h-t -- like “night,” “bright,” “flight,” and “right” -- and they all have the same long i sound. Interviewer: OK. So, tell me one more time about your categories. How did you get each one?... Lorraine: [Pointing in turn to each group of word cards.] These are the i sound made by the y. These are the i by itself -- and also I just noticed that the e on the end of them probably makes the i long. And then these are the i-g-h-t, where the g-h-t makes the i long. Interviewer: OK. Thank you, Lorraine.

Mark

Mark, student 16 in Table 3, was at the syllable juncture developmental stage. Had we simply considered his finished categories, shown below, we might have assumed that he worked similarly to Lorraine, whose categories were much the same. Instead, his think-aloud indicates Mark's visual focus -- his attention to particular letters and their placement. It is interesting to note that he established a category based on a silent letter.

Video clip (1 MB)

Interviewer: OK, talk to us while you sort. Mark: That [indicating “fly”] has a y on the end, so we can call that a category. [Places “night” beside it.] And silent e's on the end [places “nine” beside “night”] and “smile” [placing it on top of “nine”]. [Begins dealing out cards and placing each on one of the three piles.] And “bright” has the -gh-, and “wise,” and “try” has y on the end. And “prize” has silent e, and “style” would probably go here because silent e. And “twice” would also go there, and “flight” would go there [places card in the -gh- category], and so would “right.” We have three categories. Interviewer: I understand when Mrs. Wheaton...she asks you to write down a generalization. Would you tell us out loud about your generalization? Mark: OK. I made three categories. [Begins pointing at each pile in turn.] This one I made because it ends in y, and this one because it had a g-h in the middle, and this one because there's a silent e on the end. Interviewer: Anything else you'd like to tell us? Mark: No. Interviewer: OK. Thank you, Mark. You did a good job....

Brian

Brian, student 5 in Table 3, established categories based on the sound of single consonants, as shown below. Although he was by this time a within-word speller, he showed remnants of the letter-name stage with his focus on consonant sounds. He mentions that two words “sound the same,” and it appears that rhyming quality guided his categorization to some extent. When faced with a word that did not fit any of his established categories, he established an “oddball” pile.

Video clip (2.6 MB)

Brian: I'm going to put this in “right,” because it has the sound of a t. I'm not sure about that [indicating “style”] yet. “Try.” I'm going to put that with “right.” Oh no, I'm not sure about that. “Prize” -- the z -- I'm going to put that in the s pile. I'm not sure about that [looking at another card]. “Flight” -- that has the same sound as “right.” “Smile” [which he places with “style”] -- they both start with an s and they both sound the same. That's “style” and “smile.” “Try” -- I'm going to put that in the y section. “Fly” and “try” [piling the two on top of each other]. “Twice” -- I'm not sure about that one.... “Wise” and “prize” -- I'll put that in the, uh, the...s category, 'cause the z sounds like an s, and the...s sounds like a z. “Nine”...hmmm.... I'll try that...[moving it and “twice” aside]. “Bright” -- I'll put that in with “right,” “flight,” and “bright.” “Night” -- I'll put that in “flight,” and “bright.” [Studies “twice” and “nine”.] “Twice” -- I'll put that with “prize” and “wise” because the c sounds like an s, 'cause “twice”.... “Nine”...hmmmm.... “Nine”...hmmmm.... I'm not real sure about that one. Interviewer: What does Mrs. Wheaton do...what does she tell you when you are not sure about something? Brian: Put it in the oddball section. Interviewer: OK. Want to leave that in the oddball section, you think? Brian: Yeah. Interviewer: You have your categories out here. Can you generalize and tell me again why you put the words together like you did? Brian: Well, I put all these words together by how they sound. Because “style” and “smile” sound the same, and “prize” and “wise” sound the same. But this one [“twice”] doesn't sound the same, but the c sounds like an s and a z. “Right,” “flight,” “bright,” and “night” sound the same, and that's a category. And “fly” and “try” sound the same, too. Interviewer: OK. Anything else you want to tell us about your sort? No? Well, OK. Thank you.

Josh

Josh, a syllable-juncture speller and student 12 in Table 3, created categories based on letter sounds. He sorted in a procedural, rather than conceptual way. He knew the phrase “sorting within a sort,” but he failed to demonstrate a conceptual understanding of how to compare and contrast the words before him.

Video clip (5.1 MB)

Interviewer: ...and remember to talk about it. Put your brain in your throat. OK, Josh. Josh: We're going to do a sort within a sort...and it's just basically you sort about by the sound and then also by the spelling. So...“twice,” let's see...“twice,” “style,” sss...“style”.... I'm putting this one here [with “twice,”] because it's probably a c category, and this one, they're probably the sss sound is the category type, like the name. This one here, “ni...i...night,” “st...i...style,” “n-i-ght,” “st-i....” I take that back; it's probably in the t category with the t sound, but it could probably also be the c sound, so I'm going to put that there and that there [moving “night” up and down on the table]. They are both going to be in the same category -- same sound and same category -- so they both go there, and I'm going to put this one here because it doesn't have the c sound, but it does have the t. So I'm going to put that one here. “N-ine,” “n-ine,” “n-ine”.... Umm, “n-ine”...let me think about this one. This one, I'm going to, um, I'm going to put this one here [with “night”] because it doesn't share the same as the t but it does have the n. OK, “right.” This is probably going to have to go here [with “night”]. It's got the same sound, the same spelling. It basically has the same, except this letter is an n and this is an r. “Wise” -- you know, I'm going to put this here [with “twice” and “style”] because it shares the s and has the sss sound. This one [“flight”] is going to have to go here because it's got the t -- got the t sound -- but it could probably also go here, so it's in between. “Smile,” “style,” “smile” -- I'm gonna put this...uh...yeah, I'm going to put this here because it's about the same as “style.” “Smile” and“style” are about the same. “Try,” “try,” “right,” “try,” “night,” “try”...this is probably in a category all its own. It's about the same as these [“night” and “right”] but not exactly, not spelling wise. “Prize,” “right,” “prize,” “style,” “prize,” “wise”...umm.... I'm going to put it [“try”] in this one, too, because they both have r and they both have the same sound, you know? “Bright,” “right,” “try,” “bright,” “right”.... I'm going to put this here because the same...because it's got the t sound with it and it's spelled g-h-t, so it's there. “Fly,” “smile,” “fly,” “try,” “fly,” “prize,” “fly,” “flight.” I'm going to put his here because it's the same as “try,” “prize,” and “fly”...about...because they all have the i sound. Interviewer: How does Mrs. Wheaton tell you to express when you are finished with your sort? Josh: Usually we'll write a generalization. Interviewer: Can you tell me your generalization? Josh: These are generalized...these words are generalized using...they can also be...they can be spelled within a sort by the i sound, the g-h-t, i-l, r, and, um, um, and some of the oddballs, which are like “try,” “prize,” and “fly.” So the generalization is probably i, i, g-h-t, r, g-h-t, nine. Interviewer: OK, thanks. Thank you, Josh....

Back to menu

Findings

Close analysis of students' sorting processes, together with their think-alouds and generalizations, provided important insights into how this group of fifth graders determined their categories. While all the students sorted the same words, each saw or heard different features of them. For these 24 students, the sorting was a demonstration of their current word knowledge.