A mid-sized airport in the US was getting a lot of complaints from passengers that it took too long for their bags to arrive at the belt after landing. The management of the airport decided to take up and solve this problem by offering incentives to the baggage handlers. They decided that they would install a system to measure the time it took for the first bag to arrive at the belt after the plane landed. They would measure this metric and link it to the bonuses that would be paid out to the baggage handlers. Problem solved, right? All the management had to do was sit back and relax, and performance would automatically improve.
But it didn't. The metric they were measuring definitely showed a big improvement, but the passenger complaints didn't stop. They later figured out what was happening: As soon as a plane landed, one handler would quickly drive up to the plane, take out ONE bag, and take it to the baggage area. The rest of the team would stroll in and get the rest of the bags out of the plane at their own pace – No need to hurry, the bonus was already taken care of.
The point the story is trying to make is that what precisely you are measuring is important. And it is important because metrics have a life of their own. As soon as you start measuring and quantifying something, it touches some deep part of the human psychie, and the people involved start following those numbers, trying to use any means to influence those numbers. At my previous job, there was a TV in hall that would show user signup numbers. Every day, people used to stop by the display, speculating as to why the slightest dip or jump happened, and when the numbers were off from the target, people would try to do anything - from adding new features to praying for it - to get the numbers up.
Metrics can often end up influencing a lot - from what you do on a day-to-day basis to your company's strategy. The Twitter folks, if you read through their blogs and press releases, seem to be obsessed with one metric more than any other - tweets per second. Of course, they track number of users, signups, number of followers etc..., but they seem to have a particular passion for "tweets per second". And in trying to increase that number, the team has made all kinds of interesting decisions, most importantly creating an API that allows third party programs to post and read tweets. They've effectively given up control of their main property, their interfaces on the web and mobile & tablet apps by allowing third-party programs to create twitter frontends. This may be a good or a bad strategic decision, but what happened is clear: The "tweets per second" number has gone up, but that's only what has happened. No revenue, questionable quality of tweets, no assets to monetize.
The management guru Peter Drucker once said: "You get what you measure". For several years, I thought this meant that you should measure what you want and find ways of improving it. Several embarrassing mistakes later, I discovered that it was also meant as a warning: You will get exactly what you measure: nothing more, nothing less. For example, think about how you are measuring customer satisfaction. What exactly is the metric you are using to measure that? If you are measuring click-through-rates, think about what that metric is being used for. Is it really a high click-through rate that you want? Will it be OK if more people came, a fewer % clicked, but the total number that clicked was higher? Are you measuring revenue or profits? Which is more important to you?
Take some time out of your day today to think about what metrics you should be tracking. But be careful what you are measuring - You might just get it.
