The Quantified Student
Recently I had the chance to see one of Class for Zoom's demos of their new product. The platform delivers what they promise, an education-focused version of Zoom for both remote and hybrid teaching. They are in the sales phase and it was a polished pitch. One thing left me particularly uneasy...
Like LMS-es and other major edtech tools, the platform quantifies all kinds of behaviors. For example, student boxes are color-coded depending on how much they've spoken during the class. Red for not enough, yellow for not quite, and green for enough minutes yapping. The teacher can sort the active view by who's talked, basically exposing anyone who hasn't talked “enough.” These minutes speaking are tracked, like everything else, and recorded so that not only teachers but (and this was presented as a selling point) administrators can use the analytics for “best practices.”
I've always been a bit skeptical about analytics that are built into edtech products. The problem isn't in the idea of analytics; it's always a problem of their crudeness. To take the example of Class for Zoom, they are measuring minutes of speaking time. The assumption is then that quantity of speaking time is a proxy for something like “engagement.” As a kid who always spoke little but made it matter when I did speak, I find it kind of offensive that this value judgement is baked into an educational product. Why is this the metric built into the product? Is it because it is inherently meaningful?
No, time speaking is a metric simply because it is measurable. Qualitative measures are more difficult to capture. Timeliness of a contribution, knowledge in a contribution, and sophistication of a contribution — those aren't easily quantifiable. But those are things that matter in a conversation. It's like the crude LMS measure of engagement through “page views” or time spent logged in. Those metrics give the impression that they are measuring something meaningful, but in reality they are useful mostly for the edge cases. If someone isn't logging in at all, sure, maybe that tells you something. Beyond that it's a largely meaningless metric. More useful is getting frequent polling data on, for example, how long students are taking to complete assignments, whether they understand things, or more traditional assessments like quizzes or brief written work.
My gripe with analytics is not that data isn't useful. Rather, I don't like getting junk quantification presented to me as if it should mean something. As a teacher, the degree to which a student talks is one factor among many, a behavior that can have multiple causes. Sometimes it is engagement. Sometimes its an attempt to score points. Sometimes it's asking questions. Similarly, quiet is not an indication of lack of engagement. I say this as someone who has been intensely engaged while choosing words sparingly and as a teacher who has seen decades-worth of students all along that spectrum from chatty to taciturn.
While thinking about these forms of quantifying students I ended up looking again at this piece on edtech resistance and why edtech companies need to take the criticism of edtech more seriously: https://nepc.info/blog/edtech-resistance. The 'edtechlash' is necessary not because companies are evil actors but rather because it is so easy to encode values in software and platforms.
Quantifying interactions and behaviors from a classroom space is deeply problematic in general. And I can hear the counter-argument echoing in the void already— we need to measure so we can assess so we can study best practices, etc. etc. My problem isn't with the assessment. Rather, the problem is that the data about student behavior persists long past an individual class session. What can someone now do with that data? Will it get fed into algorithms to predict student success? It's a fine line and barely a full step before we're at the kind of egregious bias described at the end of Williamson's article on edtech resistance linked above, where one quantified version of a student is algorithm-ed into concrete action, denial of opportunity, and false assumptions about the underlying reality of the situation.