Showing posts with label SAT. Show all posts
Showing posts with label SAT. Show all posts

Thursday, January 24, 2013

Why Literature Matters: Part 1, or What Trollope Says About Assessment

I'm not sure I want to know anyone in this industry who hasn't at some time or another experienced a crisis of conscience. Humans seek meaning in our work, and unless we are psychopaths, we generally prefer doing good rather than evil. When we have a choice that doesn't much inconvenience us.

Why do our consciences rail at us during the dark nights of our souls? You know the answer. You've thought the answer yourself, or you've even asked me the question to elicit the answer, whether you be a friend, family member, or complete stranger sitting next to me on Southwest flight 1207 out of LAX.

If you are a parent or a teacher or a friend of a parent or of a teacher, or if you know and care about a child or if you ever think about what you read in the newspaper and magazines or consider what you see on the news, you know something about K-12 assessment and you probably have opinions about it, opinions that--please forgive the plain speaking, and this is through no fault of your own--are most likely ill-informed. Being as that there are few industries so shrouded in mystery and that excite so little enthusiasm and generate so little va va voom, so little sparkle as that of educational assessment publishing.

However, you probably understand that any conscious (let alone conscientious) person who creates tests for a living has concerns about the quality of the tests, the purposes for which they are used, and the consequences when they are put to use.

A potential salve to our consciences lies in 10 Reasons Why Assessments Make the World a Better Place (compiled by John Kleeman at Questionmark and sent to me by my colleague Frank Brockmann of CenterPoint--you see, we all three of us are soldiers in the same militia).

And so I, dutiful foot soldier that I am, armed with the twin purposes of promoting greater understanding of assessment and its implementation and consequences, and of reminding all of us why the study of literature is crucial to developing higher level thinking skills (and character!), will, with the aid of my betters, review these 10 Reasons Why Assessments Make the World a Better Place.

Today, let's tackle Reason #1: Assessments give equality of opportunity.

The idea that assessment ensures equality of opportunity is one that Victorian writer (and career civil servant) Anthony Trollope treats humorously in The Three Clerks (Chapter 3: The Internal Navigation) and seriously in his autobiography:

Italics mine.

SAT tutoring, anyone? How much does that cost these days, anyway? Who is most able to afford SAT tutoring? How does that disparity affect this playing field that the test is supposed to be leveling?

And here is what Trollope has to say about similar kinds of narrowly targeted test preparation:

Italics mine, yet again.

What Trollope says about himself is that he would certainly have performed poorly on any such public examination, but if he had been rejected based on his poor performance, the government would have just as certainly lost "a valuable public servant." That he was valuable is incontestable; every biographical (and autobiographical) account I've read indicates that Trollope applied himself to his career as devotedly, industriously, and with as much competence as he did to his writing. (You may not know that Trollope wrote a gazillion books, and that he rose every morning at 5 dark-thirty to write for two hours before he began his work day at his day job.)

Which ultimately means that Reason #1 doesn't really reassure me that I am making the world a better place, one test question at a time.

(In addition, a highly inconvenient truth of this industry is that the soundness of the test cannot always be taken for granted, and an unsound test, a test that is constructed in absence of best practices, should not be relied upon to produce data that is itself reliable. I predict this truth will apply to all ten reasons.)

Saturday, September 15, 2012

7%

There was some point in junior high school when I stopped understanding math. I kept attending class, did the homework, and got good grades. In high school, I took algebra and geometry and then scored well enough on the PSAT, SAT, and ACT to have been recruited by colleges thousands of miles away from my home in the Appalachia of the West. What the.

Except for geometry (which I loved, I know not why), I did not understand a bit of anything having to do with numbers (as previously discussed here). How did I continue to do okay at something I did not at all understand?

Maybe my happiest moment in college was when I found out I never had to take another math class again ever, not ever, never. Years passed. I graduated. More years passed. I had a bunch of bad jobs, from hostess at Denny's to secretary at an auto repair shop (previously discussed here) to scheduler at a home health agency (where my biggest responsibility was bringing my supervisor a cup of coffee from the stand on the corner and then sitting in her office and listening to her talk about how the divorce was going). I never needed to know more math than what I was pretty competent with, i.e., adding, subtracting, multiplying, dividing, and finding percentages. Whew.

Then after three years of working in the criminal justice system, I decided that crime-fighting was not for me, and decided to return to my One True Love: English. Which would mean grad school, which would mean taking the GRE. I wasn't worried; I'd always kind of liked taking standardized tests, probably because it gave me a chance to do my favorite thing in the world: sit in a corner and read with no one talking to me. That the reading material wasn't always of the finest didn't trouble me. Like gutter winos who drank Night Train, I'd read whatever was available. (Still do. Yesterday while waiting for my daughter at the orthodontist's, I read Alaska Magazine, FORTUNE, and some other rich people magazine.)

So the first time I took the GRE, I did as well as one might expect on the verbal reasoning and analytical writing, and about as poorly as anyone could possibly do on the quantitative reasoning. I was no longer able to pass as someone with a minimally adequate understanding of math. I did so poorly that when I took the GRE a second time, I bubbled randomly for the quantitative reasoning and improved my score by 7%. I don't mean to mislead anyone; this made no significant improvement. If there had been a cut score for far below proficient, that is where my score would comfortably have settled like a little toad in a pond.

I was thinking about this because my daughters' CA STAR test scores came in the mail yesterday. And because I read this, about a grown man who submits to taking the SAT.

As a side note, I'd like to say that one might think this math handicap extends to data analysis, but it don't. I love data. Love it. I love the patterns--sometimes there is even a narrative.


I was reviewing longitudinal test result data for a high school and saw some patterns that might tell a story: a strong majority of incoming freshmen scored in the advanced category, but that there was a steep downward trajectory, with about half as many grade 11 students scoring so well. I have more investigating to do to be able to draw any meaningful conclusions--is this typical of all high school students in the district, state, country, or is this just this school? Could the difference partly be explained by a large influx of lower-performing students at grade 11? What other factors might influence these results?

These might be numbers, but there is a narrative, there are characters, there's a plot with conflict, action, and, one hopes, resolution.