There was some point in junior high school when I stopped understanding math. I kept attending class, did the homework, and got good grades. In high school, I took algebra and geometry and then scored well enough on the PSAT, SAT, and ACT to have been recruited by colleges thousands of miles away from my home in the Appalachia of the West. What the.
Except for geometry (which I loved, I know not why), I did not understand a bit of anything having to do with numbers (as previously discussed here). How did I continue to do okay at something I did not at all understand?
Maybe my happiest moment in college was when I found out I never had to take another math class again ever, not ever, never. Years passed. I graduated. More years passed. I had a bunch of bad jobs, from hostess at Denny's to secretary at an auto repair shop (previously discussed here) to scheduler at a home health agency (where my biggest responsibility was bringing my supervisor a cup of coffee from the stand on the corner and then sitting in her office and listening to her talk about how the divorce was going). I never needed to know more math than what I was pretty competent with, i.e., adding, subtracting, multiplying, dividing, and finding percentages. Whew.
Then after three years of working in the criminal justice system, I decided that crime-fighting was not for me, and decided to return to my One True Love: English. Which would mean grad school, which would mean taking the GRE. I wasn't worried; I'd always kind of liked taking standardized tests, probably because it gave me a chance to do my favorite thing in the world: sit in a corner and read with no one talking to me. That the reading material wasn't always of the finest didn't trouble me. Like gutter winos who drank Night Train, I'd read whatever was available. (Still do. Yesterday while waiting for my daughter at the orthodontist's, I read Alaska Magazine, FORTUNE, and some other rich people magazine.)
So the first time I took the GRE, I did as well as one might expect on the verbal reasoning and analytical writing, and about as poorly as anyone could possibly do on the quantitative reasoning. I was no longer able to pass as someone with a minimally adequate understanding of math. I did so poorly that when I took the GRE a second time, I bubbled randomly for the quantitative reasoning and improved my score by 7%. I don't mean to mislead anyone; this made no significant improvement. If there had been a cut score for far below proficient, that is where my score would comfortably have settled like a little toad in a pond.
I was thinking about this because my daughters' CA STAR test scores came in the mail yesterday. And because I read this, about a grown man who submits to taking the SAT.
As a side note, I'd like to say that one might think this math handicap extends to data analysis, but it don't. I love data. Love it. I love the patterns--sometimes there is even a narrative.
I was reviewing longitudinal test result data for a high school and saw some patterns that might tell a story: a strong majority of incoming freshmen scored in the advanced category, but that there was a steep downward trajectory, with about half as many grade 11 students scoring so well. I have more investigating to do to be able to draw any meaningful conclusions--is this typical of all high school students in the district, state, country, or is this just this school? Could the difference partly be explained by a large influx of lower-performing students at grade 11? What other factors might influence these results?
These might be numbers, but there is a narrative, there are characters, there's a plot with conflict, action, and, one hopes, resolution.
What I'm thinking about what I do, which includes but is not limited to all matters related to education, reading, writing, editing, content development, and curriculum and assessment design and implementation
Showing posts with label data analysis. Show all posts
Showing posts with label data analysis. Show all posts
Saturday, September 15, 2012
7%
Labels:
35-year-old man takes the SAT,
bubbling randomly,
data,
data analysis,
GRE,
math,
SAT,
test results,
test scores
Wednesday, November 11, 2009
Fun with Point Biserials
A few weeks ago, I was spending a lot of time analyzing test data, specifically, the p values and pt. biserials of a set of English language arts tests for grades 3 through 11, in order to determine What Went Wrong.
In the preponderance of cases, of course, nothing went wrong. The items performed more or less as expected, with the correlations one might expect--i.e., high-achieving students got the easy questions and most of the hard questions right, while struggling students may have gotten the easy questions right, but pretty much got the hard questions wrong. And it follows that no wrong answer in these solid items attracted more students than the right answer.
But there were items with wacky data, items for which the high-achieving students picked wrong answers, or items with wrong answers that lured a higher number of students, leaving the right answer feeling like an awkward wallflower in a darkened gym at the middle school dance. My task was to review these items and figure out the big why. It is a testament to my thorough and absolute geekiness that I LOVE DOING THIS WORK. I could do it all day long, every day. Oh, my goodness. More fun than a barrel of monkeys. It was such a pleasure that I found it difficult to tear myself away at the end of the day (and honesty compels me to add that I did sneak back to it at night after my daughters were asleep).
Why, you may ask. Oh, for so many reasons! One being that it is fun to play detective, to deconstruct an item by conducting an investigative inquiry which concludes in determining the most likely source of the trouble, whether it be a stem of staggering verbosity, or a fundamental unsoundness in the premise of the item, or a bad practice (e.g., attempting to assess two or more skills with one item). It's like picking up a big tangled knotted mess of yarn and, starting with one end, delicately unraveling the knots and twists and then rolling the yarn back into a nice orderly ball. But with your brain. How often do we really have the opportunity to use our brains in this way, and truly, what can possibly be more satisfying that solving a problem?
Because assessment content development, as its own universe, is governed by its own rules, and--to me, in my ridiculous and relentless geekiness--these rules have a sort of simple elegance, the beauty of rigor and orderliness. (Were I to wax biblical on the matter, I would start talking about the necessity for doing things decently and in order.) So to use the data as a mirror to reflect the soundness (or unsoundness) of the item, diagnose the trouble, and then find a means of repair requires the use of a range of tools that include a knowledge of the rules (and knowing when it is acceptable to bend them) and of how students might interpret an item (which is not always how a test developer intends) and of the content area itself. And it also requires a bit of intuition, that ability to sense your way in the dark that just comes from thoroughly knowing something, in the same manner you could navigate your home with the lights off because you know where the couch is and where stands that big coffee table with the treacherously sharp corners.
In the preponderance of cases, of course, nothing went wrong. The items performed more or less as expected, with the correlations one might expect--i.e., high-achieving students got the easy questions and most of the hard questions right, while struggling students may have gotten the easy questions right, but pretty much got the hard questions wrong. And it follows that no wrong answer in these solid items attracted more students than the right answer.
But there were items with wacky data, items for which the high-achieving students picked wrong answers, or items with wrong answers that lured a higher number of students, leaving the right answer feeling like an awkward wallflower in a darkened gym at the middle school dance. My task was to review these items and figure out the big why. It is a testament to my thorough and absolute geekiness that I LOVE DOING THIS WORK. I could do it all day long, every day. Oh, my goodness. More fun than a barrel of monkeys. It was such a pleasure that I found it difficult to tear myself away at the end of the day (and honesty compels me to add that I did sneak back to it at night after my daughters were asleep).
Why, you may ask. Oh, for so many reasons! One being that it is fun to play detective, to deconstruct an item by conducting an investigative inquiry which concludes in determining the most likely source of the trouble, whether it be a stem of staggering verbosity, or a fundamental unsoundness in the premise of the item, or a bad practice (e.g., attempting to assess two or more skills with one item). It's like picking up a big tangled knotted mess of yarn and, starting with one end, delicately unraveling the knots and twists and then rolling the yarn back into a nice orderly ball. But with your brain. How often do we really have the opportunity to use our brains in this way, and truly, what can possibly be more satisfying that solving a problem?
Because assessment content development, as its own universe, is governed by its own rules, and--to me, in my ridiculous and relentless geekiness--these rules have a sort of simple elegance, the beauty of rigor and orderliness. (Were I to wax biblical on the matter, I would start talking about the necessity for doing things decently and in order.) So to use the data as a mirror to reflect the soundness (or unsoundness) of the item, diagnose the trouble, and then find a means of repair requires the use of a range of tools that include a knowledge of the rules (and knowing when it is acceptable to bend them) and of how students might interpret an item (which is not always how a test developer intends) and of the content area itself. And it also requires a bit of intuition, that ability to sense your way in the dark that just comes from thoroughly knowing something, in the same manner you could navigate your home with the lights off because you know where the couch is and where stands that big coffee table with the treacherously sharp corners.
Subscribe to:
Posts (Atom)