Speaking of Language, Cross-Referenced to Value-Added

Some item writers love multiple-choice language items; some hate them. I pitch my tent in the former category. Several years ago, I wrote all of the language items (writing conventions and writing strategies) that appeared on multiple parallel forms of a statewide high school exit examination. 

Language items may be either standalone or passage-dependent. The former are discrete entities, e.g.:

Read the sentence.

The Supreme Court may rule in favor of restrictions to freedom of speech when words are considered insendiary. 

Which is the correct spelling of the underlined word?

A incendiary *   B incendairy    C ensendiary     D Leave as is.

Passage-dependent language items accompany an editing passage as previously discussed. 

Language items may address any kind of writing skill or content knowledge targeted by an assessment: conventions (punctuation, capitalization, spelling), usage (grammar, diction), style (sentence structure and variety, diction), and organization (focus, elaboration, and support). (Rarely, writing applications skills are also assessed by multiple-choice items; one can easily see the difficulty of assessing any applied skill in this mode.)

There is some overlap (you see "diction" may fall into the usage camp or the style camp, for example, and sometimes grammar items fall into conventions, but basically, you might think of language items as mechanics and style/organization. Proofreading items then generally address mechanics, while editing and revision items may target either mechanics or style/organization, depending on the assessable skills.

Not all item writers can write language items. It's almost more of an editorly than a writerly undertaking, requiring a combination of specialized knowledge, persnicketiness, and an excellent grammar handbook. Some language items produced by unqualified writers are incomprehensible.

Sometimes these slip through the quality control cracks because language items are difficult to review because one must read very closely in order to identify that the error is an error, that the correct response is indeed correct, that the error is the type of error indicated by the standard/objective, and that there are no errors other than those intended. When we read, our brains automatically correct much of the error that we see, even when trained to do otherwise, so to review an item with intentional error that may also contain unintentional error is asking a lot. It's overwhelming, especially if you add in the editing passage, which means you have to do a lot of checking back and forth. Not to mention that a thorough knowledge of English usage and grammar is a rare commodity these days.

I have strong opinions about the content and construction of language items, to wit:

1. A language item should include only one type of error. A spelling item should not be contaminated by punctuation errors.

2. Each wrong answer choice should contain only one error.

3. The errors in the item must be the kinds of errors that students at the targeted grade level would reasonably make.

4. The errors should be obvious to the student who possesses the skill or content knowledge being assessed.

5. Trivial, why-bother sentences (or passages) should not be used to assess language skills. Language items should use actual facts in the stimulus sentences and paragraphs, rather than the easy but lame sentence.

Here is an example of an item with a trivial stimulus sentence:

Read the sentence.

 I haven't seen ______ since December.

Which pronoun should be used in the sentence?
A him *   B she    C they    D we

Here is an example of a language item based in fact:

Read the sentence.

 Langston Hughes had been writing poetry for years before Vachel Lindsay helped ____ publish his work.

 Which pronoun should be used in the sentence?

A he     B him *    C them    D they

I've written language items based on marine biology, space exploration, phenotypic plasticity, you name it. What is interesting in the world is a lot. There's no need to write about the purely meaningless.


UPDATE: Fixed some formatting with the MC items. Those are kind of tricky with the line breaks and indents.


UPDATE: Oh! I forgot to mention a recent dethspicable deplorable practice, that of using sentences from previously published (usually classic literature) as the stimulus for language items, either containing (newly imposed) embedded error or offering students options for improvements to the original (classic) writing.


Your mind is probably as boggled as mine was when first I came across this nastiness in the woodshed unspeakable horror unsound practice.


How might this work, you ask? Not well, as illustrated in the following examples.


Here is an example of the addition of error to a line from classic literature:

Read the sentence from Leo Tolstoy's Anna Karenina. 

Everyone thinks of changing the world, but no one thinks of changing themselves.

Which word would best replace the underlined word in the sentence? 

A himself*   B myself   C ourselves   D Leave as is.  

Here is an example of an invitation to students to improve upon a line from classic literature:

Read the sentence from Leo Tolstoy's Anna Karenina. 

Everyone thinks of changing the world, but no one thinks of changing himself.

To avoid repetition, which word would best replace the underlined word in the sentence? 

A altering*   B becoming   C improving   D renovating   

Why would anyone perform such sacrilege and blasphemy do such a thing? From the best of intentions.


An item writer, bored with the trivial, why-bother stimulus sentences, feels inspired to use sentences culled from the books she loves, sentences that are themselves little masterpieces of beauty, wit, and style. Won't this be good for the students? And there is no one to stop her, as these sentences are stolen borrowed from works now in the public domain.


Good intentions, road to you-know-where, cross-referenced to the law of unintended consequences.