Last week, I wrote about the author Nassim Taleb and the "black swan" investment strategy that earned him billions of dollars. Another book he wrote is entitled Fooled by Randomness. This is a phenomenon that I believe is extraordinarily common in the practice of medicine, as well as anywhere measurement of processes is used. The science of statistical process control is something I only stumbled upon in the middle of my career as I struggled with ineffective attempts to improve the care processes I was involved in. It was through the Institute for Healthcare Improvement that I was exposed to a more rigorous understanding of how to know whether a process has changed. After formal training in statistical process control, my worldview is forever changed.
Rich asks for the take home!
What a refreshing thought- a simple take home... I don't mean to be fresh- this stuff is hard. You are correct to ask for somehting practical- so here's one: we should be asking for all our time ordered data (any data we collect over time to measure a process) in control chart format (the chart has built in 'macros' that evaluate the data set, apply the right statistical tools, and show the user what is significant change and what is not. There are a number of these tools on the market - at Cooley we use "QI macros" an easy addin to Excel. IT can install it for anyone who wants. And our clever quality folks can help managers and medical staff leaders run these for them. I'll post a blog next week showing some examples.
While we know random variation exists, without the tools we do exactly what you said: we act if the value "seems significantly out of our desired range". The trouble is, without the tools, human judgment about what "seems significant" is really poor, because of what Taleb talks about-our desire to assign causality. Using our intuitive sense has been shown over and over again to be extraordinarily variable, and unreliable. You are correct that we should be displaying SPC charts for data reports routinely. Industry does this, medicine does not.
Cooley Dickinson Hospital • 30 Locust St. (Route 9), Northampton, Mass. • (413) 582-2000