Spurious Tuples

(See this post for an introduction to this blog series.)

I got several comments on my presentation that were roughly analogous to this one, which I am quoting exactly (the emphasis is in the original):

So, to me, they’re still evals of what students think they’re learning, not whether or not they’re actually learning. So, it seems assessment would still need to be done…

There are several issues being conflated here, so I’m going to try to unpack them one-by-one:

1. The commenter’s use of “assessment” at the end of the quote suggests that s/he doesn’t think that minute papers are “actual” assessment. Or, to put it another way, minute papers are not a test or exam, and therefore cannot measure learning, and therefore aren’t assessment.  I would counter that both of those latter two premises are incorrect: things that are not tests can measure learning (as anyone who has graded an essay can attest), and therefore can be considered part of assessment.
2. It is true that minute papers, and other informal assessment tools, do rely on students to self-report their learning.  And I do, in fact, have a lot of problems with self-reporting as a research methodology.¹Â  If I had a nickel for every study that’s been published in a peer-reviewed library science journal that essentially relies on “students’ self-reported confidence with X” or “students’ self-reported mastery of Y,” well … I wouldn’t be as rich as if I had a nickel for every “librarians’ perceptions of X” study, but I’d still have a good chunk of change. BUT, at the same time, I also have a lot of problems with assuming that tests accurately measure learning, and especially that tests accurately measure students’ behavior in real-world situations.  Students may very well choose the correct answer on a test that asks them how to evaluate information they find on the web, but then turn right around and cite a completely bogus web site in a paper they have to write for another class.²Â  (Put another way: every single college student in this country can tell you what the legal drinking age is.)
3. It’s important to note that the context in which I’m using the minute paper is formative, rather than summative.  Formative assessment is ongoing, and is used to improve instruction (and, presumably, learning) on a day-to-day basis. Summative assessment happens at the end (of the unit, course, program, etc.) and is used to determine if the students have mastered the specified learning outcomes for the unit, course, program, etc. Nobody is asking seniors, at the end of their college careers, “what’s one useful thing you learned, and one thing you still have questions about?”

Next up: But will it get me tenure?