Sunday, October 9, 2022

Interrelated Variables Make it Hard to Interpret Results

A student once asked me about two related things in physics.  After explaining the differences, he said, "Oh, okay, they're exactly the same except for all the ways they are different."  We laughed a lot about that, but I liked it and have used it for a number of things.  I have a friend / former colleague with whom I share a similar personality and many similar opinions, but we have a few issues on which our opinions are totally opposed.  I use that sentence to describe our friendship.  

Recently, however, I see a ton of things on Edutwitter that exemplify this in a less funny way.  The assumption is that changing one variable will only change one variable without realizing that our lives are interrelated in ways we cannot predict.

When we went into lockdown in the spring of 2020, no one thought it was the educational ideal, but we adapted as best as we possibly could to virtual learning, keeping kids learning something, even though we were not teaching or assessing in the same ways we would have.  More importantly, virtual learning enabled us to keep the kids connected to each other and to us in some small way.  The following year, most schools either remained virtual or forged ahead with a hybrid model, knowing that it wouldn't be the same as being in school face to face, but doing the best we could to protect our students during the pandemic.

A few weeks ago, some research data was released about the academic and mental health impact of the methods we used to keep education moving during the pandemic.  While these numbers should have been no surprise, some on Twitter reacted to them as though they were a bombshell.  The impact on academic achievement was clearly negative due to many kids choosing not to attend virtually at all and others choosing to cheat their way through virtual tests.  While it will take time to address the lack of gains in those years, it can be done.  

The mental health impacts are tougher to interpret and address.  Lockdown is obviously not the sole source of anxiety, but those who want to criticize lockdowns are treating it that way.  Scientifically, it is hard to parse the impact of that one variable because it was coupled with the racial reckoning following the murder of George Floyd and a difficult election cycle (understatement?), which had their own influence on the mental health of youth facing a world in which they would soon vote.  And we may have forgotten that anxiety rates were already on the rise before 2020.   

Twitter and its 280 characters is, however, no place for nuance, so what we read there is more like, "See, we should never have even gone into lockdown" and "I told you remote learning was a joke" and "If we had just left them in schools, they would not have these mental health issues."  The problem with that is that it imagines an alternate universe in which A only impacts B, which is not the universe in which we live.  To quote some Aaron Sorkin screenwriting, "The world is a more interesting place than that."  We live in a world where A influences B and C and D and where C may be influencing A and B in return.  

If we imagine a world in which there had been no lockdown, we wouldn't just have one in which everything else would be the same except for its impact on academic progress and emotional health.  We would be looking at higher transmission rates, a completely overwhelmed hospital system, greater fear of attending school (as we saw in the hybrid year with some making the choice to stay home anyway).  I can tell you from experience that the death of a classmate has massive mental health consequences.  If staying in school had caused the 1% death rate that Dr. Oz and Dr. Phil deemed acceptable, we would be talking about 7-10 deaths in many schools.  I can promise you that would have a negative impact on both academic progress and mental health.

Lockdowns weren't good.  We know that.  Hindsight is 20/20, but we also knew that when we did it.  But attending school during the height of the pandemic would also not have been good.  Even those who won't admit it know that to be true.  Some schools continued to assess and grade students, while others made a different choice.  Some were synchronous while others utilized videos and self-pacing.  We all made the best decision we could with the information we had in our context.  Monday morning quarterbacking on Twitter isn't helpful because there is no way to know what impact a different combination of decisions would have had.

We, as teachers, need to set a better example.  Rather than constantly criticizing and saying, "See, I knew it," we should be teaching our students that solutions aren't simple in a complicated system.  Knowing that could reduce the polarization we see in the world today because we might not always be so certain that we are right all of the time.  If you'll allow one more Aaron Sorkin quote, "Complexity isn't a vice."

No comments:

Post a Comment

Change, Loss, and Why Your Brain Hates It

According to recent surveys, the most common sources of stress include divorce, the death of a loved one, job loss, marriage, retirement, ha...