Confirmation Bias


Several weeks ago I was ruminating about various types of bias and I ran into one recently that I thought to be very interesting in terms of its implications. This is Confirmation Bias.

According to some definition pulled from somewhere “In psychology and cognitive science, confirmation bias (or confirmatory bias) is a tendency to search for or interpret information in a way that confirms one’s preconceptions, leading to statistical errors.”

When I was at MaRS Discovery District, (or should I say, when I was on MaRS) the then VP of Operations, Don Duval used to report that more than 80% of respondents to their survey gave MaRS a 4 or 5 out of five in rating the service they got from MaRS. Now at the time, not thinking about this stat very much, I thought this was an OK result.

The measure on the surface looks like a good one but I have since learned that if your average on customer satisfaction stats is a 4.2 or so, this is actually not a good result. The way I learned was by conducting a workshop where I got an average score of 4.2. I knew that the workshop, while not totally stinking, wasn’t very good. (These things happen.) But the participants were kind enough to give me a 4.2.

At the University of Toronto School of Continuing Studies, where I teach a course on Strategy Execution, my most recent score was a 4.8 and that I consider reasonable but now I don’t find it superb.

The problem is confirmation bias. People who have selected you as a supplier are subject to this confirmation bias as it is a hard thing psychologically for someone to admit they made a mistake. Having invested time in a supplier, a customer will want to confirm their own choice by not rating a supplier as poor. The way they get around this is by scoring someone as 4, not great, just OK. In the parlance of performance evaluations this is probably a “low meets.”

If your survey population is subject to confirmation bias you should be looking for an average above 4.5, not down around 4. And if you’re doing these types of studies, a better methodology to use is Net Promoter Score because it eliminates confirmation bias.

Avoiding Biases with Statistics

biasesIn the world of biases, I know myself that I’m guilty of recency bias as I have a habit of giving the most credence to that which I learned latest. (Or it could be that I have a bad memory and can’t remember anything I learned earlier.) Whatever it is, we’re all subject to biases in making decisions.

A recent Globe and Mail article on How the Leafs Learned to Love Analytics (no link because it’s behind their pay wall) outlined five types of bias. These were confirmation, recency, information, sample size, and simplicity.

I was impressed thinking that these were a good summary of major types of biases. But then I went to Wikipedia to see more on the subject and was overwhelmed. Their list of types of bias has over 150 entries. Everything from subjective validation bias to reactive devaluation bias.

You really should scan the list to see what you suffer from. I laughed at the last one, being guilty of it myself. This can be like a new type of hypochondria…spot the bias.

Back to the article. The proposition in the article is that statistics can help you avoid the effect of a bias. Now on first blush, that seems a reasonable proposition but I’ve been thinking about it and have decided that it just won’t wash.

Biases are for the most part, an emotional response to a particular situation. Statistics on the other hand are a logical response. Now I don’t know about you but for most people, in a battle between the two, an emotional response trumps a logical response any day.

And this got me wondering, maybe this is why people just don’t pay attention to statistics or analytics. They will just create cognitive dissonance between their emotional bias and the results of the analysis.

Don’t worry, be happy.

So Much Busy Work

Busy WorkLet’s face it, if you’re working more than eight hours a day, you’re not being effective. Lots of studies show that productivity declines rapidly after about eight hours of work. And much of what we’re doing in a day is busy work.

You might have heard of the term busy work but let me explain how I see it. Busy work to me is anything that doesn’t materially help you achieve your objectives and result in success. Let’s take that one slowly as there are a few thoughts embedded in here.

If you don’t know what success is in your role, then you’re doing busy work. In fact in this situation, all the work you’re doing is busy work. After all, if you don’t know what leads to success then what on earth do you know what to do when you get into work?

If you’ve done a good job establishing what success is in your role, then you have probably also figured out what objectives you have in place to achieve that success. If you’re doing anything that doesn’t tie into those objectives, directly then you’re doing busy work.

Next point. If you doing something that ties into your objectives but doesn’t help you meet those objectives in a material way then you’re doing busy work. Why would you do something that’s meaningless and immaterial in any case unless you’re trying to stay busy?

To bring these thoughts home, you should read an article from the Harvard Business Review about Why We Humblebrag About Being Busy. Let’s stop celebrating how much we work and instead start bragging about how little we have to work to get shit done.

Are your co-workers helping or hindering?

co-workersThere is a great study that has been published by the Harvard Business Review that attempts to look at what is going wrong with strategy execution. Some of the results are surprising but given what we have all experienced, they shouldn’t be. One of the things that the study is saying is that a large part of your ability to get your job done, of getting strategy implemented hinges on your co-workers, those people in other departments who you rely on to get things done for you.

The study asked whether people can rely on others around them to meet their promises, to get things done that they committed to. Fully 84% of respondents indicated that they could rely on their boss and direct reports all or most of the time.

But when it comes to relying on other people in the company to meet their promises, shockingly only 9% of managers can rely on co-workers in other departments. Their commitments are only just as good as those given by external parties.

And when they can’t rely it results in duplicated effort, letting promises to customers slip, delaying deliverables, or passing up attractive opportunities. This inevitably leads to conflict that is badly handled.

There is no easy fix to this problem as it appears to result from companies trying to do a good job executing strategy by aligning departments to specific tasks. When peel are hyper-aligned to their own set of tasks, and helping others out doesn’t contribute to getting your own done, then co-workers are left sucking air.

Now I know that none of you reading this blog would be guilty of letting co-workers down but all those other people out there better get their sh*t together right?

Fake It Till You Make It – Redux

keep-calm-and-fake-itI love the stuff that comes out about leadership as so much of it doesn’t make much sense. That’s why I was pleased to see this quote which reminds me of something I was told along time ago: Fake it till you make it.

This quotation comes from the book Act Like a Leader, Think Like a Leader

“The fallacy of changing from the inside out persists because of the way leadership is traditionally studied. Researchers all too often identify high-performing leaders, innovative leaders, or authentic leaders and then set out to study who these leaders are or what they do.

I have found that people become leaders by doing leadership work. Doing leadership work sparks two important, interrelated processes, one external and one internal. The external process is about developing a reputation for leadership potential or competency that can dramatically change how we see ourselves. The internal process concerns the evolution of our own internal motivations and self-definition. It doesn’t happen in a vacuum, but rather, in our relationship with others.”

I’m now wondering if this applies to everything that we learn to do, that we really can’t learn unless we’re doing it and until then we have to fake it.

Change and Ambiguity

mainimageIn the last few days I’ve had a few great examples of why companies both big and small can’t see the competition coming, can’t innovate and frequently get bested by much smaller firms. The point was really driven home by a meeting I had recently with a company which shall remain nameless.

This company is loosely in the media industry. They have had declining revenue for ten years now. In fact revenue has been declining in double digits for that period. As you can imagine, after 10 years of double digit decline, there isn’t much left.

Meanwhile, new upstarts in the same area of the media industry have had double and even triple digit growth, have become the companies that everyone talks about and have been invested in by the world’s best venture capitalists.

So what is it that makes the difference? Two things. The older companies have been around for a while and are populated by people who fundamentally don’t like two things: change and ambiguity.

When a new form of competition enters the market, the first thing an established company has to do is to change. But if it has employees who don’t like change then without replacing all its employees, it is hard to create the type of change that is necessary.

The next problem is ambiguity. Even if people like to change, you have to like changing from a clear situation to one with a great deal of ambiguity. The challenging upstarts are exploring new ways of doing business, new business models, and new products. Creating these things is a highly ambiguous exercise. When you start out, you don’t know where you’re going to end up.

Many people who would embrace change are reluctant to embrace anything ambiguous. They want to know what’s at the end of the change process if they commit to it. They are reluctant to change to an ambiguous situation and with that, entropy sets in.

The lesson to me is that if you’re starting a company or are in one that is being pummelled by new upstarts, look for people who like change and ambiguity or you’ll end up running the equivalent of a buggy whip maker in the automotive age.

Performance hurdles and ceilings

I was looking at a performance plan and bonus scheme for someone today and noticed a rather problematic hurdle. That’s one of those statements that says you get a $10,000 or whatever bonus if revenue exceeds $1 million. These weren’t the numbers but I blanched at the amounts as the bonus was large and the hurdle was high. It was so high, it was almost a cliff.


I’ve worked with people before, who when they realize they don’t have any chance of meeting their bonus, sandbag sales in order to shift them into the next year. The same thing happens with ceilings. In the old days IBM had a rule that no one could earn more that the President. That stopped more than one sales rep from selling late in the year.

The problem is, what do you do about performance hurdles and ceilings? I think ceilings are stupid. You can get burned one year without a ceiling but you can change the comp plan the next year so the expected revenue is more in line with reality.

But hurdles are another matter. How do you set a minimum standard without a hurdle. I don’t think there is any way to deal with them except through one of my favourite tools, trailing 12 month averaging. Paying every quarter or even monthly based on the last twelve months results doesn’t penalize or benefit anyone for sandbagging and you can adjust the hurdle rate to make it more realistic on a regular basis.

I know one company that pays bonuses based on weekly results. You can have an off week and it won’t affect the rest of your month. It also means that you can’t slack off for very long as you’re being measured all the time. They have hurdles and they work when measured weekly.

Complex to administer, maybe but it gets rid of quarter and year end stuffing which inevitably leads to operating issues and it keeps people on their toes all the time.

A New Definition of Success

I think I lost the plot for the last three months and after one of the busiest periods of my life, am finally getting back to having time to think. This got me wondering about the definition of success. It is one of those concepts that plague people and society has developed an elevated level of anxiety about success.

Am I successful because I’m busy, because I’m making money? Because I’m happy? Don’t look at me for an answer as I frequently grapple with the same question. Particularly so as I walk through my own neighbourhood, past $5 million houses that seem to capture today’s definition of success. (The one that amuses me most is the house where the owner bought the house next door and tore it down so that he could have a slightly bigger garage.)

Someone who has grappled successfully with the concept is Alain de Botton. I thought for a treat today, you might enjoy his TED Talk on the subject.


13 Steps to Better Strategy Execution

Screen Shot 2014-08-22 at 2.14.20 PMI’m teaching a course at U of T in Continuing Studies in the fall called Connecting Strategy with Action. It’s all about driving superlative results through better strategy execution. The most remarkable outcome of students who take it is how they end up shifting their whole management focus from activities to results.

In the course we talk about employees three greatest needs:

  • To know exactly what is expected of them.
  • To know how they’re doing.
  • To understand how they can improve.

I prepared this document as an outline of the course and thought I should share it here as well.

13 Steps to Better Strategy Execution

The Organized Mind

mcgill-alumni-mainDaniel Levitin has written an interesting book called the Organized Mind. I saw him speak the other day about the book and was fascinated by the stories he told about highly accomplished people.

Not being one of those people, I listened closely to figure out what I could learn. What struck me most was the way he differentiated between creative thinking and rational decision making.

To Levitin, “Creative thinking means allowing the nonlinear to intrude on the linear and to exercise some control over the output.” It is a mystery though, how linkages are made between the linear and non-linear in order to make creative leaps.

“In contrast to creative thinking is rational decision making.” According to Levitin, “Human brains didn’t evolve to be very good at this…because we have a limited attentional capacity to deal with large amounts of information.”

So if a human brain has two critical functions: creativity and decision making, it turns out that we’re not much good at either of them. And if we don’t have the ability to to pay attention to detail, we’ll never get any better at the latter.

This leaves the human brain to excel at one thing, which is day dreaming, and that I suspect we are all very good at this.