Tuesday, August 6, 2013

PSA: Conducting a Performance Review of the Inner Critic

For a few days I've been considering what Freud identified as the super-ego, what has become popularly known as the inner critic, or if you're abundantly blessed, your personal busload or committee or parliament of inner critics.

Generally speaking, the role of the inner critic seems primarily protective, as wrongheaded as its tactics are for that purpose. If you're a writer, your inner critic might advise you to clean the house instead of writing, or may suggest there's no point in writing, because everything you write is horrible, bad, and no-good, and that one 7th grade English teacher was just being nice or had bad taste or that the one story you wrote that got published or that your writing group liked was an anomaly. No attempt means no risk means safety from rejection and criticism. Probably there are inner critics for every occupation.

There are as many strategies of dealing with inner critics as there are types of inner critics: we can embrace them, treat them with compassion, or murder them. I'd like to offer a new one: conduct a performance review to evaluate their effectiveness on the job.

Frankly, if my inner critics were my employees, I'd have fired them by now. I always sort of knew they were making a mess of it--how did they even make it past the interview?-- but I'd never taken the time to consider the nature of their incompetence. I made one of the biggest mistakes employers make with performance reviews, according to Forbes: I haven't been conducting them.

To prepare for the review, I must specify the responsibilities of the inner critic: what is the job description?

A critic approaches a work with curiosity, is open to experiencing a work and responding to the work, observes those responses, and investigates the work in order to consider the artist's purpose and message, and how the artist achieves that purpose and conveys that message. A critic pays close attention to the work. A critic is dedicated to a particular art (and should be an expert of that art or field), and in the raising up of that art, and so the critic is responsible for celebrating achievements as much as for observing attempts that may fall short.

These are questions a critic might ask in examining a work of art:
  • What place does this work have in the tradition or genre?
  • How has this work been influenced by earlier works?
  • How does this work reflect a social, historical, or cultural context?
  • How may different levels of interpretation be applied to this work?
  • What are the messages of this work?
  • Is there a universal message or is the message specific to a group, time, place, instance?
  • How does this work convey those messages?
  • Does this work have a unique voice and style that set it apart from others of its type?
  • What contributes to the uniqueness of this work?
  • What emotional responses does a person have to this work?
  • How does this work elicit those emotional responses?
What is not in the critic's job description:
  • insulting, bullying, ridiculing, mocking, belittling, name-calling, disparaging the artist or the work, listing past mistakes, predicting future failure
Critics address the work with specific questions and use the information to discuss the work and explore the meaning of the work. It's a lot easier to approach a work from opinion (I like it, I hate it) or judgment (rotten, all right, awesome), but so much less interesting and less effective.

What good does opinion do, really? Even nice opinions aren't much use, as much as it feels better when a reader says that he likes one's writing than when he says that he hates it.

If you heard the way my inner critics talk, you'd agree that they've fallen down on the job. Or maybe they got handed the wrong job description in the first place. I can't recall their ever having performed satisfactorily.

Writers need to write, and we need to figure out ways around and through and over and under the obstacles to writing, even while we're constructing the obstacles.

And there are other reasons:
  • We parents, teachers, coaches, aunts, uncles, grandparents--we all of us adults--create the voices for future generations of inner critics. I'd like my kids to go out into the world with inner critics who contribute to, rather than sabotage, their happiness and well-being.
  • Many of us supervise others. In so doing, we can't help but impose the messages of our inner critics on others. We have the choice to be constructive or destructive. We have the choice to be reasonable (or not) with ourselves and others.
  • No good writing ever came from performance anxiety.
I'm concerned that how we think about, talk about, teach, and measure performance in writing trains students to believe that not only are they terrible writers, but they are not capable of writing well. The reality is that everyone can tell a good story, and if you can tell a good story, you can write a good story, and if you can write a good story, you should, because we all need stories, as many as possible, from as many perspectives as possible.

Thursday, August 1, 2013

The Big Idea, or Focus, Cross-Referenced to Basic Rules of Item Writing

What comes before preparation is intention, which we previously discussed here. Still, the concept of the Big Idea bears further exploration.

Let's consider how we might approach this grade 4 standard from the CCSS, RL.4.2:
Determine a theme of a story, drama, or poem from details in the text; summarize the text.
This standard is passage-dependent; students read a story, poem, or play (or excerpts of the same) and then answer questions about what they read.

This standard requires two distinct subskills: determining a theme and summarizing text. 

Either may be assessed with multiple-choice, constructed-response, or technology-enhanced items, although I note that in an ideal world, we wouldn't use multiple-choice for summarizing, but would instead ask students to create the summary. Again in that ideal world, it's best if we provide the student with opportunities to demonstrate mastery of a particular skill by allowing the student to perform the skill; however, we often operate under constraints that exclude the ideal. That's okay.

After we've read all of our ancillary support materials and have thoroughly acquainted ourselves with the story, poem, or play (for less experienced item writers and for all item writers without a strong background in literary analysis, I suggest making an outline of and annotating the passage in order to avoid the trap of writing superficial and repetitive items), we determine the theme(s). There may be more than one. Out of fairness, choose the strongest theme that is most clearly supported and most thoroughly developed in the passage. The theme may be stated explicitly or may be implied by the characters' words and actions.

Here is our passage, "A Boy's Song" by James Hogg.

    Where the pools are bright and deep,
    Where the gray trout lies asleep,
    Up the river and o'er the lea,
    That's the way for Billy and me.

    Where the blackbird sings the latest,
    Where the hawthorn blooms the sweetest,
    Where the nestlings chirp and flee,
    That's the way for Billy and me.

    Where the mowers mow the cleanest,
    Where the hay lies thick and greenest,
    There to trace the homeward bee,
    That's the way for Billy and me.

    Where the hazel bank is steepest,
    Where the shadow falls the deepest,
    Where the clustering nuts fall free.
    That's the way for Billy and me.

    Why the boys should drive away,
    Little sweet maidens from the play,
    Or love to banter and fight so well,
    That's the thing I never could tell.

    But this I know, I love to play,
    Through the meadow, among the hay;
    Up the water and o'er the lea,
    That's the way for Billy and me.

We would probably use call-out boxes to define some of the vocabulary--"lea" and "nestling" stand out as words likely to interfere with student understanding.

If we're writing a multiple-choice item, the stem will look like this:
What is a theme of the poem?
Or we might identify the poem only by its title ("What is a main theme of 'A Boy's Song'?") if we plan to write another item about genre characteristics ("How does the reader know 'A Boy's Song' is a poem?").

Often at the lower grades, we use "theme" and "main idea" as synonyms; depending on curriculum, grade 4 students may not yet be familiar with the specific terms for narrative elements, and we don't want to erect unnecessary obstacles for those students, so we might write a stem that looks like this:
What is a main idea of the story?
I prefer "a" rather than "the" in order to allow for variety in literary interpretation; we'd follow the client's preference on this. In this case, a clear theme is the joy of spending time in nature. Now we have a stem and the correct response:

What is a theme of the poem?
A the joy of spending time in nature
B [TK]
C [TK]
D [TK]

Next we'd write three distractors (wrong answers). Each distractor should have a rationale--that is, each should embody a specific mistake or breakdown in comprehension or literary analysis that might hinder a student en route to determining the theme. The rule in item writing is that, given the evidence in the text, distractors must be "plausible but not possible." The distractors should be clearly wrong to the student who is able to "determine a theme...from details in the text."

Many clients require item writers to provide rationales or justifications for the wrong answer; I support this wholeheartedly as valuable practice for inexperienced item writers. Experienced item writers have rationales in their minds already, so it's just a matter of typing them.

When we write the distractors, we must stay focused on our Big Idea. In order to do that, we'd consider the breakdowns that occur when students attempt to identify a theme. In order to do that, we'd think about the process of making meaning from text. We read the poem and step back and come up with the overarching meaning: the joy of spending time in nature. Then we think about how a student might falter in putting the pieces of the poem together to see that big picture. A student might get stuck on a detail of the poem, and mistake that for a theme. A student might confuse theme and subject. A student might focus too narrowly.

Next up: constructing plausible but not possible distractors.

What I'm reading: The Reivers by Faulkner and Imaginings of Sand by Andre Brinks.