Big Data, Bad Metrics

As technology provides us with more access to more data, a lot of
attention is being directed towards leveraging that data to improve outcomes. The popular notion is that by gleaning insights from so-called “Big Data,” we can make better, faster fact-based decisions
—and that these decisions can move the needle on everything from business performance to human longevity.

This notion is borne out to some extent by empirical evidence. If you collect and analyze customer data, you can sell more. And if you measure your food intake, you can lose weight.

However, not everything that readily lends itself to measurement is important. And not everything that is important readily lends itself to measurement.

Measuring the Wrong Thing
In the June 2014 issue of Harvard Business Review, Clay Christensen and Derek van Bever make this very point about investment. They assert that flawed metrics often drive companies to eliminate jobs rather than create them. They demonstrate that ratios such as return on invested capital (ROIC) incentivize corporate decision-makers to simply whittle down denominators (by, say, outsourcing), rather than do the hard work of adding to the numerator. They further argue that a metric like ROIC is obsolete, because capital is not the scarce resource it once was.

The result is short-term investing that fails to generate growth or jobs. And these results will, if left unchecked, ultimately work against the interests of the very capitalists for whom return ratio metrics have become orthodoxy.

Similarly flawed metrics dominate the contact center industry, where managers chase KPIs such as shorter call-handling times or higher customer satisfaction scores—neither of which have much to do with desired business outcomes such as greater long-term customer loyalty or more market-aligned innovation. But, because these metrics are easily understood and easy to capture, they drive counter-productive and toxic behaviors.

How’m I Doing?
At a recent technology conference, I found myself sitting next to a bright young market analyst who waxed enthusiastic about the Quantified Self. He went on at some length about how the principles of data capture and analysis could be applied to the individual in order to improve physical, financial and even emotional outcomes.

My question to him was a simple one: “What is your ethical metric?”

The question was, of course, rhetorical. And it served its rhetorical purpose by leading to a discussion about whether we are in fact primarily physical, financial and/or emotional beings—or whether we might be in some way moral beings as well. We then discussed how we might rightly incorporate moral imperatives into a life in which physical, financial and emotional imperatives play such a dominant role.

Ethical goodness and long-term socioeconomic impacts don’t easily find their way into our spreadsheets and BI toolkits. So in our zeal for metric-driven personal and institutional behaviors, let us also exercise wisdom. Otherwise we may succeed in making our numbers—but fail in making our world.