Tim Harford, British economist and columnist, recently wrote about impediments to understanding statistical claims and arguments.* Sure, a rigorous understanding of statistics is a major hurdle but one only experts will ever clear. Harford is talking instead about bias and emotion, something he suggests each of us can and should manage.
He begins, “'The best financial advice for most people would fit on an index card.' That’s the gist of an offhand comment in 2013 by Harold Pollack, a professor at the University of Chicago. Pollack’s bluff was duly called, and he quickly rushed off to find an index card and scribble some bullet points — with respectable results. When I heard about Pollack’s notion — he elaborated upon it in a 2016 book — I asked myself: would this work for statistics, too? ... I set out to write my own postcard-sized citizens’ guide to statistics."
Harford's postcard:
1. Observe your feelings.
2. Understand the claim
- what does it mean?
- is this a causal relationship?
- what's being left out?
3. Get the backstory.
4. Put things in perspective.
- is that a big number?
- what is the historical trend?
- beware 'statistical significance'
5. Embrace imprecision
6. Be curious
- go another click
- treat surprises as mysteries
He writes: "With this in mind, my statistical postcard begins with advice about emotion rather than logic. When you encounter a new statistical claim, observe your feelings....What sort of feelings? Defensiveness. Triumphalism. Righteous anger. Evangelical fervor....It’s fine to have an emotional response to a chart or shocking statistic — but we should not ignore that emotion, or be led astray by it. There are certain claims that we rush to tell the world, others that we use to rally like-minded people, still others we refuse to believe. Our belief or disbelief in these claims is part of who we feel we are."
Statistical analysis is a tool that, whatever else it may do, reduces a portion of reality to an objective set of observations. Whether unemployment went up or down and by how much is hardly a matter of how one "feels" about the matter. But how one chooses to define "unemployment" at the outset can be and often is very much a matter of how one feels about the matter, and, per Harford, answering that question is vitally important to understanding another's apparently objective assertions about changes in unemployment rates. His list goes on, but that's the drift of it.
Our great urge to reduce or maybe elevate matters to objective, measurable data, and from those data to draw important conclusions, develop policies, alter priorities, etc., is inescapably shackled by subjectivity. The shackles may be intentional, even sinister, but most often they likely operate beyond our of our conscious reach, despite all aspirations of objectivity.
If so, one might say we should all by cynics, trusting nothing and no one. I disagree. I prefer to acknowledge if not celebrate this feature of being human. Our biases and assumptions come from somewhere. Are the sources good or bad, thoughtful or merely reflexive? It seems far more interesting and even consequential to explore what makes us care about unemployment rates in the first place, rather than just counting the non-working.
Now, back to that postcard. Harford's postcard-sized advice for interpreting statistical claims was inspired by Professor Pollack's claim that he could fit onto a postcard all the best advice for managing personal finances. Then, he did it, squeezing all he could onto 3x5 inches of space:
That seems like fun and a challenge. I'm going to think bigger and draft in the same limited space a list of what I believe to be my broadest assumptions and biases, the kinds of things that inevitably color all I see and read, including claims about unemployment rates. I hope I don't embarrass myself, but if I'm honest about it I'm sure I will.
_______
* Financial Times (2/7/18): https://www.ft.com/content/ba4c734a-0b96-11e8-839d-41ca06376bf2