Continuing our series of blogs on pitfalls and best practices of win/loss, today we examine the best practice of confidentiality in conducting win/loss interviews. The original publication of this blog was here.
Ben: Ken, you and I have talked now about one of the key pitfalls of win-loss—not getting buy-in from the sales organization. Let's turn it around this time. I’m ready to hear some best practices.
Ken: Sure. There are so many best practices that you pick up as you gain experience with win/loss. The number one in my book is to conduct win/loss interviews confidentially. And here, I’m not talking about handling client materials under non-disclosure. Of course, it goes without saying that these are all handled confidentially. The best practice I’m referring to here is the use of confidential interviewing to get the most honest and open feedback from customers. We want their frank feedback with minimal bias.
Ben: Ah, so you are talking about promising the interviewed customers confidentiality?
Ken: Yes, and there are many aspects to this that are important to discuss. One size does not fit all.
Ben: Well, can you start with the benefits of a confidential interview?
Ken: Sure. The reason to do this, of course, is to make the respondents feel comfortable to open up and speak honestly and to reduce the bias in what they say. There are two aspects to this. The first is to make the interview anonymous--that is, to promise the person being interviewed that their name and their company name won’t be shared with anyone. The second is to make the interview blind, so that the respondent doesn’t know who the research is being done for.
Ben: Sure, that makes sense. I personally find it hard to give negative feedback to people. Maybe others feel differently, but even if I feel service in a restaurant is not very good I still leave a tip. I won’t go back but it’s not like I’m going to stiff the server.
Ken: Yeah, it’s embarrassing to do that, but less so if you are confident that what you say will be reported anonymously. So, we do all our interviews anonymously so that our respondents feel comfortable to speak as openly as possible about the reasons for their decision, and to feel free to criticize the vendors and their offerings if they have anything negative to say.
Ben: Sure, so an anonymous interview protects the respondent from having their words come back to haunt them.
Ken: Right. So we are very careful to remove identifying information from the interview transcript we prepare for the client. That’s one reason why transcripts I prepare have at least two people review it at least two times each to make sure that we don’t inadvertently include something that could identify our subject. We actually guarantee this to respondents with a promise to pay $10,000 if they ever find a report that we produced with their name in it. We are very, very careful.
Ben: Now what about doing the interviews blind? How is that different from making them anonymous?
Ken: A blind interview is one where the respondent doesn’t know who the interviewer is working for. In a blind interview, we introduce ourselves as independent consultants, but we do not identify our clients. This is a very powerful way to remove bias from the interviews that could otherwise creep in consciously or unconsciously. In non-blind interviews, it’s natural for people to try to tailor their responses based on who they know will be reading the report. We have techniques for countering this but still, a blind interview is generally going to be less biased.
Ben: As a practical matter, what differences do you see between blind and non-blind interviews?
Ken: I find that in blind interviews, respondents are more openly critical of the client's competitors. In part, I think, this is because customers like to keep some cards hidden from their vendors to retain some bargaining power in the on-going relationship. If they tell the winner about all the weaknesses of their competition, then they figure they will have less leverage in pricing when it comes time to renew. Surprisingly enough, though, customers tend to be tough critics of the winning vendors as well in blind interviews. So, if my client won, then there’s very important feedback for them--some tough love--that they should prefer to hear through me than to find out from a disgruntled customer later on. Or, if they lost, good information about some weakness in the competitor who did win.
Ben: So, do you recommend blind and anonymous interviews for all win/loss programs?
Ken: I recommend blind and anonymous interviewing wherever possible but understand that this is not always going to be possible. Specifically, blind interviews are not possible in the European Union, where customer privacy laws require that clients introduce us directly to respondents for an interview. Also, blind interviews tend to be harder to secure; we will usually get a higher response rate when our client introduces us to the respondent than we would approaching them cold. So, some clients choose to do interviews non-blind, when they are working with small numbers of deals. Blind or non-blind, though, we make all of these interviews anonymous, for the reasons we discussed. And in future best-practice discussion, we can get into details about how to conduct an interview--blind or non-blind--that mitigates the bias problems that we discussed.
Ben: Cool, I am looking forward to that. But for today, to summarize, always make your interviews anonymous and do them blind, wherever you can?
Ken: Yes, that’s what we do, as independent consultants. Of course, if you want to do interviews yourself--in house--then you can’t do them blind or anonymous without pretending to be a third party. And that’s unethical. You should never misrepresent yourself. If you do interview your own customers yourself, you must tell them who you are.