A key objective of every win-loss analysis program is to find out why the client wins or loses sales opportunities. This may sound deceptively simple… just ask the customers “Why did/didn’t you pick my client?” and then record the answers. But the reason win-loss consultants like myself are so valuable is our ability to analyze the results – and not just from one interview, but from a set of interviews.
In order to do this, the work starts with the win-loss interview design. A well-designed win-loss questionnaire allows us to put the discussion in a framework that enables sophisticated “apples to apples” analysis – keeping in mind that ultimately, we’ll need to aggregate all of the responses, normalize the data, analyze the findings, and report the results with recommendations. We’ll also want to slice and dice the data.
This blog post focuses on one aspect of conducting your win-loss interviews – that is, how to capture the raw criteria data (or the reasons you win or lose) so that you can do this type of analysis. (For more tips on conducting win/loss interviews, see our other blog on how to engage with a respondent when you are conducting a win/loss interview.)
Research and Questionnaire Development
Before designing an interview questionnaire for a client, I learn as much as I can about the market, the products, the players, and typical customer needs. I ask the client for their take on why they think they win and lose deals. I’ll review their competitive decks and battle cards, read analyst reports, and dig into competitors’ marketing info. If the space is new to me, I may produce a glossary of terms and review the technical architecture of a solution. This due diligence may take a couple of days of work, but it’s worth it in the long run. The more I know about the space, the more flexible I can be during the interviews.
Determining Criteria Buckets for a Win-Loss Analysis
With the research behind me I draw up a criteria list—a comprehensive list of reasons a prospective buyer would select my client’s offerings, as well as any offering from my client’s competitors. The criteria become buckets – which I’ll fill when I conduct my interviews. The buckets should be as non-overlapping as possible, succinct, and easily understood by my client and their buyers.
Mapping Responses to Criteria Buckets in Real Time
During an interview, I’ll ask the buyer for a list of criteria (or buying reasons) that were important. If someone starts talking about “latency”, I know it gets placed in the “Performance” bucket. Or if someone discusses “Integration with LDAP” it goes in the “Openness” bucket. When we feed all of the interview data into the Petronio Insight portal, we will generate charts showing the impact that each criterion had across the deals we interviewed, among other things. So in order to do this, we need a consistent criteria bucket approach across all the interviews, and we need to map responses into the right buckets while each interview is happening.
Not Everything Fits in a Bucket! – Staying Flexible During the Interviews
Wouldn’t life be great if everything followed your expectations? But in a win-loss interview you have to be prepared for variations, and unexpected responses. When I ask the interview subject—say a CIO of Director of IT —for his or her selection criteria, the answer may, or may not, map very well into my predefined criteria.
For instance, I may get a criterion response that is a very customer-specific aggregation of things. They might say something like “how well it met our technical requirements.” Arrgh!! That may very well have been a legitimate reason (from the customer’s perspective) – but what were those technical requirements? I can’t accept that as a criterion, because “technical requirements” will be different from buyer to buyer. So I need to find out more…. “Can you tell me what those technical requirements were?” I may ask to limit the response to the top 5 technical requirements, and I’m hoping that these criteria will now match with my buckets. The truth is that they likely won’t match the names of my buckets. So we take each requirement, one at a time, and discuss it so that I can map that into my predefined criteria list.
Clarifying and Defining Criteria in a Win-Loss Interview
Keep in mind that our language is not perfect, and our words can have multiple meanings. People can misuse words or misunderstand a question. So I don’t just get a list of criteria; I ask for the definition of each criteria and why was it important – among other things—to help me map each criterion correctly.
See an example of this technique in practice in our sample win-loss interview.
Using Experienced Consultants to Conduct Win-Loss Interviews
This brings me back to my main point – in order to understand how to do the mapping and the win-loss analysis you need to do your homework. You must be knowledgeable and fluent – and engaging – if you want to learn as much as possible from the buyer. And you need to be able to talk with business execs, the IT leaders, the developers, and the partners… and everyone in between. You need to understand the technology, the business needs, the ROI and TCO, the risks, the trends – if you are truly to engage like this.
Anyone can read a list of questions to a buyer and check off boxes. But for a win-loss program to succeed—to gain deep analysis and valuable data—it’s important to have a seasoned, experienced, technical interviewer who has prepared well for the interview.
Otherwise, crap in equals crap out.
Looking for more information on how to do win-loss interviewing and analysis? Below are some resources:
- Nimble: The Win/Loss Analysis: It’s How You Play the Game (blog by Laura Patterson)
- Coursera: Video overview of Win-Loss
- MaRS: Win-loss analysis: A tool to gain product and market information