Interviewing for Analysis – Capturing Win-Loss Reasons

A key objective of every win-loss analysis program is to find out why the client wins or loses sales opportunities.  This may sound deceptively simple… just ask the customers “Why did/didn’t you pick my client?” and then record the answers.  But one of reasons a win-loss consultant like myself is so valuable is our ability to analyze the results – and not just from one interview, but from a set of interviews.  And in order to do such an analysis, you have to focus on capturing the win-loss reasons in a methodical way.

It all starts with the design of win-loss interview questionnaire.  A well-designed win-loss questionnaire allows us to put the discussion in a framework that enables sophisticated “apples to apples” analysis – keeping in mind that ultimately, we’ll need to aggregate all of the responses, normalize the data, analyze the findings, and report the results with recommendations.   We’ll also want to slice and dice the data. 

This blog post focuses on one aspect of conducting your win-loss interviews – that is, how to capture the raw criteria data (or the reasons you win or lose) so that you can do this type of analysis.   (For more tips on conducting win/loss interviews, see our other blog on how to engage with a respondent when you are conducting a win/loss interview.)

Research and Questionnaire Development

Before designing an interview questionnaire for a client, I learn as much as I can about the market, the products, the players, and typical customer needs. I ask the client for their take on why they think they win and lose deals. I’ll review their competitive decks and battle cards, read analyst reports, and dig into competitors’ marketing info. If the space is new to me, I may produce a glossary of terms and review the technical architecture of a solution. This due diligence may take a couple of days of work, but it’s worth it in the long run. The more I know about the space, the more flexible I can be during the interviews.

Determining Criteria Buckets for a Win-Loss Analysis

With the research behind me I draw up a criteria list—a comprehensive list of reasons a prospective buyer would select my client’s offerings, as well as any offering from my client’s competitors. The criteria become buckets – which I’ll fill when I conduct my interviews.  The buckets should be as non-overlapping as possible, succinct, and easily understood by my client and their buyers.  

Mapping Responses to Criteria Buckets in Real Time

During an interview, I’ll ask the buyer for a list of criteria (or buying reasons) that were important.  If someone starts talking about “latency”, I know it gets placed in the “Performance” bucket.  Or if someone discusses “Integration with LDAP” it goes in the “Openness” bucket. When we feed all of the interview data into the Petronio Insight portal, we will generate charts showing the impact that each criterion had across the deals we interviewed, among other things.  So in order to do this, we need a consistent criteria bucket approach across all the interviews, and we need to map responses into the right buckets while each interview is happening.

Not Everything Fits in a Bucket! – Staying Flexible During the Interviews

Wouldn’t life be great if everything followed your expectations? But in a win-loss interview you have to be prepared for variations, and unexpected responses. When I ask the interview subject—say a CIO of Director of IT —for his or her selection criteria, the answer may, or may not, map very well into my predefined criteria. 

For instance, I may get a criterion response that is a very customer-specific aggregation of things.  They might say something like “how well it met our technical requirements.”  Arrgh!! That may very well have been a legitimate reason (from the customer’s perspective) – but what were those technical requirements?  I can’t accept that as a criterion, because “technical requirements” will be different from buyer to buyer.  So I need to find out more….  “Can you tell me what those technical requirements were?”  I may ask to limit the response to the top 5 technical requirements, and I’m hoping that these criteria will now match with my buckets. The truth is that they likely won’t match the names of my buckets. So we take each requirement, one at a time, and discuss it so that I can map that into my predefined criteria list.

Clarifying and Defining Criteria in a Win-Loss Interview

Keep in mind that our language is not perfect, and our words can have multiple meanings.  People can misuse words or misunderstand a question.  So I don’t just get a list of criteria; I ask for the definition of each criteria and why was it important – among other things—to help me map each criterion correctly.

See an example of this technique in practice in our sample win-loss interview.

Using Experienced Consultants to Conduct Win-Loss Interviews

This brings me back to my main point – in order to understand how to do the mapping and the win-loss analysis you need to do your homework. You must be knowledgeable and fluent – and engaging – if you want to learn as much as possible from the buyer.  And you need to be able to talk with business execs, the IT leaders, the developers, and the partners… and everyone in between.  You need to understand the technology, the business needs, the ROI and TCO, the risks, the trends – if you are truly to engage like this. 

Anyone can read a list of questions to a buyer and check off boxes. But for a win-loss program to succeed—to gain deep analysis and valuable data—it’s important to have a seasoned, experienced, technical interviewer who has prepared well for the interview.   

Otherwise, crap in equals crap out.


Other Win-Loss Resources

Looking for more information on how to do win-loss interviewing and analysis? Below are some resources:

7 Top Actions and Outcomes from a Win-Loss Analysis

The obvious reason to conduct a win-loss analysis is to discover why you win and lose deals. But knowing why you win or lose is just the beginning. You’re not just looking for good information, you want to drive concrete changes that will increase your competitiveness. The true value of a win-loss analysis program comes when you turn your win-loss insight, into action.

While every company’s situation is unique, the following are 7 typical outcomes from a win-loss analysis program:

1. Better Deal Targeting and Segmentation

Once the win-loss analysis shows you why you win certain kinds of deals, you can gauge which new deals have the highest chance of success. Instead of treating every opportunity equally, focus your salespeople on the most promising deals.

2. Guidelines for Building Relationships with Prospects

Most prospects research your organization before they engage your sales team. During this research, they develop biases, opinions, and perspectives on how they should judge your product. Let your sales people know what your prospects are thinking. This will help them communicate in an intelligent manner. Share the insight from your win-loss analysis to develop the easiest, fastest… dare I say the most “delightful” experience across all touchpoints so that you stand out as the preferred vendor.

3. Product Development Insight

A key outcome of your win-loss analysis is what features or capabilities your product lacks to win – either because competitors have surpassed you or prospects have started looking for capabilities you simply don’t provide. This first-hand data couldn’t be more critical to your product development team as it plans future updates. Your win-loss analysis should help you to prioritize new developments.

4. A Clear, Defensible Vision

You need to present customers and prospects with a vision of where your company and its products are headed, and your vision needs to align with theirs. Win-Loss analysis can help you build a vision that is unique to you and your value proposition. See our blog post on Vision for more information on this subject.

5. Updated Competitive Strategies in Sales Tools and Enablement

Battlecards, training modules, whitepapers—all of your sales tools need to reflect what your prospects and customers actually say about you and your competitors, not what you think they are saying. Drawing on win-loss findings, you can advise your sales and marketing teams on ways to update your tools to better inform them on how your competitors have beaten you and how to better leverage your strengths (or the reasons that you win).

6. Sharpened Messaging

The competitive arena is dynamic. Presentations, websites, social media, collateral, elevator pitches – all need to reflect the latest in terms of customer needs and their outlook on your market. Your win-loss analysis provides superb insight to help you update your messaging on the buyer’s problems, your differentiation, and how your solutions can best meet the customer’s needs.

7. Better Informed Pricing Strategies

Understanding how your competitors have priced opportunities and gaining insight into the buyer’s need and decision process should prove invaluable when developing and updating the price for your solutions.

So, in summary, a win-loss analysis program is not just for identifying why you win and lose… It’s about taking that insight and putting it into action to become a more competitive organization.

Win-Loss Analysis Sampling: Selecting the Right Deals

“How many interviews do we need to do?”  That’s a very common and valid question clients ask me at the outset of each win-loss program engagement. There is no hard and fast answer. Each situation is different. In answering that question, I’ve provided some best practices to consider, along with some cautions about what NOT to do.

Mistake #1: Trying to do too many interviews

Don’t try to come out of the gate with a huge number of interviews. You’ll find that it’s too difficult to get the contacts and participation that quickly.

I always recommend starting small, with very high-quality deliverables. Then grow your program iteratively – because you can’t go from 0 to 100 interviews overnight. You’ll find that you need stakeholder support, and you’ll need to gain trust.  Rich, insightful, and well-summarized interviews, and a very diligent selection, distribution, and analysis approach, will help you gain that trust.

Some win-loss providers will tell you that you need to do a very large number of interviews to get to some level of statistical significance before you can take action.  But I’ve found that even doing a very small number of interviews has been helpful to my clients.  One interview alone can be valuable because, with win-loss analysis, it’s not just purely a numbers game.  The qualitative data can be just as important as the scores for the vendors across the criteria.

Mistake #2: Not properly sampling based on your demographics

You have to know which deals to interview (and which ones to avoid).  An intelligent sampling approach starts by identifying your key demographics (that is, the slices to want to look at) beforehand.  Then sample in a manner that allows you get some patterns and/or statistical significance in each demographic. You’re going to want to look at how the reasons you win and lose differs by demographic.  You do this by looking at subsets of deals. For instance, generate a win-loss chart using just the deals in which you competed against competitor-A. Then you’ll generate the same chart, but using only the deals where you competed against competitor-B.  Then you’ll compare the charts by putting them into a dashboard (as an example, see the dashboard of our win-loss portal that supports this type of analysis).  The differences that you observe will provide you with more insight.

What do I recommend?  To get started, focus your sampling on a few top demographics.  Sample only your top three competitors, selecting hard fought deals that were recently closed (won and lost).  Sample each of your regions.  Choose another demographic, perhaps industry or deal size.  Aim to do between 16 to 32 interviews in your first round.  Focus on delivering very high-quality, insightful interviews that wow your management. Repeat in a regular cadence, which is most often quarterly, but for some companies, this will be yearly.  Look also at the data across all of your demographics and do an analysis.  If you lack some level of statistical significance in one or more of those demographics, increase the number in the next round for that demographic. Now that’s a recipe for success.

Mistake #3: Picking the wrong deals to sample

Interview only those deals that are aligned with your strategy.  For example, if your company wants to go in the direction of selling bigger deals, don’t interview the small ones.  And avoid sole-source deals where there are no other competitors.  Avoid deals that are upsells of seats to existing accounts.  Focus instead on the key competitors.  You don’t need to interview deals for every competitor – and if you believe some of the less frequent competitors are very similar in their offerings or sales approaches, you can group them together in a bucket for sampling and apply the same competitive strategy against both.  Finally, shoot to interview an equal number of wins and losses. You’ll learn from both of them.

Mistake #4:  Waiting to complete your Interviewing before putting findings into action

Once interviewing starts, you’ll see patterns right away.  For instance, you’ve only done 5 deals and you notice competitor X switches its pricing to an all-you-can-eat, enterprise license at the end of every sales cycle. The pattern is clear – we don’t need wait to do an analysis in order to put this into use. Forward the information on to the sales team before they encounter the competitor again.

Mistake #5: Not focusing on interview quality

The quality of the interviews is hugely dependent upon the people involved—the interviewer and respondent. When selecting the respondent, ask your sales person to recommend the most knowledgeable and open person to interview. Qualify that respondent on the phone as part of the scheduling process to make sure he or she will be forthcoming with the details.

Equally if not more important is the interviewer.  The key to delivering insight is having an interviewer who can draw real depth and participation from the respondent, while understanding the context of your competitive arena.  Only use highly knowledgeable, skilled interviewers who have deep and varied backgrounds, and who employ a rigorous approach.  See an example win-loss interview on predictive analytics.

You don’t want a junior level person buzzing through a script; you need someone to engage the respondent in a conversation so that the respondent opens up. Consider this… What will happen if the interviews you produce offer no more insightful than a sales-debrief? Your program will get shot down.

So when a company asks me about how many interviews they need, I like to step back and ask a few questions in return.  Ultimately, your goal should be to execute your program flawlessly with high quality deliverables.  Then you can use your that research to recommend the changes that will increase your competitiveness.

More information about statistics in win-loss and market research:

Why a Win-Loss Analysis Consultant Improves Results

Companies that take the first step of doing a thorough review of why they win and lose deals often ask the question, “Why can’t we just do this internally?” Good question, but there are five good reasons why you should “no” and use a win-loss analysis consultant instead.

1 – You Need an Unbiased Perspective.

If you have your own sales, marketing, or product development staff conduct interviews, review transcripts, and look for competitive advantages or issues, how can they be objective? That’s not to say that they’re trying to shape the truth, it’s that they’re going to start with a biased perspective.

An external win-loss analysis consultant can present themselves as an objective third-party because they have no prior relationship to the customer/prospect, and they have no preconceived ideas.  Buyers are just more candid and open when taking with a third party.

An external win-loss consultant can also ensure the anonymity of the respondent. This is very important if you seek the truth and want your respondent to relax and tell it the way they see it.

Using a third party to conduct the interview removes bias due to these factors.

2 – You Can Benefit from Experience and a Proven Win-Loss Methodology.

Interviewing is an art. A win-loss analysis methodology that uncovers key decision-making criteria is a science. Leveraging win-loss program professionals who, as a team, have performed thousands of interviews and hundreds of analyses can consolidate feedback into actionable recommendations. That’s where the art and science come together.

It also helps to work with win-loss consultants who have experience in your market. You won’t get insider, confidential information, but you’ll be working with people who know the data points and trends that you are looking for.

3 – It’s Cheaper to Outsource the Work to a Win-Loss Consultant. Really.

It costs less to outsource this type of competitive research to a win-loss analysis consultant than doing it internally. (Do I need to remind you ‘time = money’?). There’s the time to set up interviews, transcribe interviews, edit transcripts, translate transcripts from a foreign language, run win-loss analysis across transcripts, etc.

Ask the vendors who provide win-loss programs, “Do you have a toolkit, custom software, or processes that automate many of the tasks needed to produce a thorough win-loss analysis?” This is when you’ll understand that your company is not equipped to do this internally. And by doing your own competitive intelligence, you’ll find some win-loss providers are better equipped than others to get to the bottom of why you win and why you lose.

4 – You’ll Get Results Faster from your Win-Loss Program.

When you engage with an external win-loss consultant, you’ll have a dedicated team focused on delivering your results quickly.  The team approach means you’ll be able to get far more interviews (and an analysis) completed in a much shorter time frame than you can on your own.

5 – You Can Put Your Scarce Resources to Better Use.

Win-loss programs are very time consuming to implement, but easy to outsource to a win-loss analysis consultant. What you can’t outsource is implementing changes change. Focus your time and talent on how you’re going to evolve the parts of your business that will improve win rates: creating new sales messaging, developing JIT marketing materials, coaching sales execs on how to avoid landmines, launching new products that address gaps, responding to competitive “fear, uncertainty, and doubt”, and more.

Simply put, let an experienced, objective resource do the interviewing and analysis. You need to spend your time informing your organization about the findings – and advocating for changes that will deliver more wins.


Additional Win-Loss Resources

Looking for some unbiased opinions on value of win-loss analysis programs?  See these resources:

The Role of Vision in B2B Tech Sales

What’s the role of vision in B2B tech sales?  Through extensive B2B win-loss interviewing in the tech markets, we’ve learned vision can be an important reason for winning or losing a deal.  And it presents an great opportunity to differentiate yourself and claim leadership.

A lot of high tech companies think vision is fluff.  And they come to that conclusion because their customers don’t like to admit that vision played into their buying decision.  But when we conduct in-depth win-loss interviews, vision comes up almost every time. Buyers say things like “Yes, we saw their vision statement, and it rang true for us.”  Vision is a major factor in driving the perception of where a company is going, how successful it will be, and how good its products are. The big tech giants, like IBM, Oracle, Microsoft, and, etc., got big by learning how to sell vision.

If your competitor has a weaker product but more resources and a better-articulated vision, you can bet that customers will have more confidence in them to create an “even better” product in the future.  It’s also true that customers will often overlook product deficiencies if the company has a great vision behind it.

Below are six points to help you understand the role of vision in technology sales:

Vision completes your product story.

You may have the best technology, but where is it going? Where will your company be in five years when this version of the product is obsolete? If the customer grows, where will you be?

Vision is not a pipe-dream. It is simply a means of connecting your products to the evolution of the market. One slide with a short statement may be all you need to establish your vision. In five years, superior customer service will depend on social media listening technology to stay ahead of customer issues.

If your competitors lack vision, it’s an opportunity to define vision for the market.

Customers looking at vendors in certain markets will say, “No one in this market space is selling vision. It just is not on anyone’s radar.”  If a market is so mature that everyone thinks they know where it is heading, then that’s a great opportunity to define the market vision before someone else does.  Assert yourself as the thought leader, the trail blazer who is taking the market in a certain direction.

Make your vision unique to you.

You have to own this vision. It needs to support your unique value proposition. I have talked to customers who say, “A couple of the vendors had roughly the same vision statement. It was nothing I hadn’t heard before.” The customer really wants to know how you see things and what your role is going to be in the future of the market. “Where are you taking us? Why? And, how will you get us there?“

And, if a vendor doesn’t lay out a vision, we’ll hear comments from customers like, “I just don’t see them being around.”

Show customers how your vision maps to theirs.

If you have one slide stating your vision, up the ante with a second slide that matches your vision to your customer’s vision. The proper discovery process will help you understand enough about your customer’s challenges and goals to make the vision match their needs.

This is where the tech giants excel. They are very good at using their relationship with the customer to ensure that they can deliver on a vision. They’ll say, “We’ll do everything we can to make the customer successful.”

Vision does not equal “roadmap.”

The product roadmap is one piece of the puzzle. In effect, it is your third slide. Product roadmaps show your ability to incrementally advance your products toward your vision, one step at a time. If you’ve shown that you’ve been building toward a vision, and the roadmap is an extension, you’ve just gained credibility.

Err on the side of openness, too.

Without a clear roadmap, customers may get nervous. They may think you lack a strategy.  They’ll say “I’m worried about this company because the market is moving so quickly, and they need to adapt. They seem to be falling behind.”  Or they may worry that you’re hiding something.  “The secrecy tells me something else is going on they don’t want us to know.”

Use your competitors’ inability to execute to your advantage.

Lastly, when you’re confident about your vision and your ability to execute on it, look at your competitors’ situation. Even though the tech giants are good at projecting vision, they fall far short on their ability to execute on their vision. I ask customers, “Have you ever looked at the company’s track record? Are they able to execute on their vision?”

What I often hear them say about competitors’ visions is, “Their strategy in this space has been confusing for me,” or “That’s not for us, and probably won’t be a good fit.” Reactions like this can be a gold mine for your competitive messaging.

To sum it up — It is essential that technology companies come up with a true, unique vision on how they’re going to attack the future.  Make sure you communicate your vision clearly, and make it work for the customer.

And be sure to put your strategy regarding vision and roadmap into your battlecards!


Additional Research on Vision:

Tips on Starting a Win-Loss Analysis Program

Before I became a consultant who delivers win-loss analysis programs, I was a Director of a Product Marketing at an enterprise software company where I led the competitive intelligence function.

One day the VP of Product Marketing and Product Management called me into his office one day, can you look into starting a win-loss program?  It was new to me – and we ended up selecting a consultant who was experienced in delivering win-loss analysis programs.

If you are not already familiar with what a win-loss program is, please see this blog, Win-Loss Analysis Programs Add Value to Tech Sales Initiatives.

It was a good move to engage an experienced win-loss consultant, because, as I learned, there is a lot to running a such a program.   Here are some tips that I’d like to share:

TIP #1 – This is a collaborative effort between Marketing and Sales. You need the buy-in from your most senior executives, and it must include the VP of Sales.

Our first task at hand was to meet with our VP of Sales to discuss the benefits of the program to his team — and to get his buy-in. I explained the benefits. “Each interview would bring us invaluable account research, and will also provide some insight into the competitors’ strategy. Once we get enough interviews, the program itself will provide us with an analysis of why we win and lose. We’ll also get recommendations on how we can increase our win rates.“

To be honest, it was an easy sale. We decided to start with a 5-interview trial.

TIP #2 – Select only deals that closed within the last 6 months. Focus on deals that are strategically aligned to the business. Exclude sole-source deals, and smaller deals. All deals need to have a competitor involved. Hard fought deals give you the juiciest interviews.

Our next task was to select the deals we wanted to have interviewed. We wanted a few wins and couple of losses. We only selected deals that had closed in the past six months. Because if a deal was closed more than six months ago, then the interview will be about the client’s experience with the winning vendor – instead of being about the client’s purchase decision.

We also wanted to select bigger deals – that is, those that were aligned with our business strategy. For instance, we didn’t want to select small deals where the customer ordered a few more seats. Finally, we favored deals that had at least one of our five top competitors.

Working with our VP of Sales Operations, we selected the deals that met the above criteria. We selected a total of 15 deals – hoping we’d get at least 5 willing participants. That list was then vetted with our two executive sponsors, who made their changes to the selections. Now we were good to go.

TIP #3 – It really helps to have someone with authority in sales to obtain the contacts for the interviews.

Next, our VP of Sales Operations approached the sales executives for chosen accounts. She obtained the names and telephone numbers of those we wanted to interview. We were seeking to interview the decision makers – or individuals who were intimately involved in the decision process.

TIP #4 – If you are using a skilled executive-level interviewer (with a background in high tech), that person will know what questions to ask. You just need to provide a little education, and discuss some of the questions you think are important.

After having received the contacts, I met with our win/loss consultant to discuss what I knew of the deals. For each deal, the interviewer asked

  • Who won it?
  • Who were the competitors?
  • How much was it worth (what was our price)?
  • When did the deal close (when was it won or lost)?

I spent a bit of time educating him on our competitive playing field — what were our differentiators, and who were our competitors, etc. Finally, I suggested a few questions we’d like to get answered.

TIP #5 – Win-Loss interviews get the attention of the most senior management.

Now – this is where the magic started to happen for me.It wasn’t more than a week or two when I got a call back from the consultant. “Hey John, we just got our first interview, and it’s meaty!” Soon I was reading it. It told us what the customer was looking for, why we lost, and how the vendors were evaluated and ultimately scored. It was great!

I sent it off to our VPs of Marketing/Product Management and Sales. Before I knew it, the interview was forwarded to the CEO by one of my executive sponsors. I was cc’d on it. What came back was something akin to this.

“John – Great work.” (I hadn’t really done much … but who was I to argue?).

“Please cc me and my direct reports on these interviews.”

TIP #6 – Sales people LOVE win-loss presentations! They are keenly attentive to the stories of how we win or lose.

So my next step was to institutionalize the win-loss analysis program within the Sales organization. After completing the 5 interviews, I introduced the program to our sales team in a 40 minute presentation that shared the results.

“Here is how we won at ABC – here is how we beat the competition”.

“This is what the customer said”.

“Here is why we lost at XYZ – here’s what our competitors did, etc.“

Wow. It couldn’t get much better.

Then I explained how every sales person could benefit by leveraging this service.

“It will provide you with invaluable in-depth account research – and you’ll learn all about your competition at the account.”

Everyone at the meeting was excited. “Great work”. “Excellent stuff”. “I’m competing against that same competitor – I know how to counter them!“… You get the idea?  That was how the win-loss program was started.

TIP #7 – Analysis across the interviews tells you why you win and lose, among other things. It should be shared cross-functionally.

Now I come to the 2nd part of the story…what happened when I had 20 interviews (10 wins and 10 losses).

My win-loss consultant and I developed a deck with win-loss charts that showed why we were winning and losing. We looked at each criterion and the supporting quotes. We sliced the data by each competitor – and examined the competitors’ approaches against us. We sliced the data by other demographics – and were able to see variances by region or vertical.

We delivered the presentation to different groups – with about 10 participants in each group. We delivered it to the CEO and his staff. We delivered it to product management, and to product marketing. We delivered it to the services organization, the sales organization, and the greater marketing group. Each group was awed by the results. Everyone got on board, suggesting ideas on how we could each contribute to increasing our competitiveness.

TIP #8 – Win-Loss analysis benefits the company, and it’s a great opportunity to participate across the company.

Before I knew it, the company was benefiting enormously from this level of research and collaboration. Our product team knew what features the customers preferred from our competitors. Our marketing team understood what messages were resonating with our customers…and more.

I also benefited professionally. Suddenly, I was thrust onto the stage. Overnight I was the go-to guy for the competition across the entire company. I got very involved in the deals, strategy discussions, reporting information and trends, researching, and offering recommendations.

And, it was great fun!

I have two important final points to make, and it’s about sales participation. You need your sales team to want to participate in your win-loss program.

TIP #9 – Not every sales person is comfortable with having their customers interviewed.

The first point is that you need to educate the head of Sales about the fact that many sales reps do not want their employer interviewing their accounts. Perhaps its job security, or some other reason – but I’ve seen many excuses from the field, including my favorite – “we’re trying to sign another deal, and I don’t want this interview to raise any bad feelings.” Maybe there are times when it’s best to hold off, but you need a sales management team that is responsible for getting these interviews to happen, and is familiar enough with the pipeline to know when to press for an interview.

TIP #10 – Interviews should only be used to learn, and not to punish.

Finally, if based on the findings of a win/loss interview, your management calls out people for the mistakes that they’ve made (or worse, fires them) — then that may set your program back. You should not publicly admonish anyone based upon findings of a win/loss interview – or that will make your sales team fearful of participating. They may sabotage your program.

They will do things like prep the customer with the responses – or provide you with contact numbers of employees who have recently departed the organization. That’s one of the reasons that when we publish an interview, we like keep it anonymous. It protects both the sales executive, and the respondent.

Sales is hard enough without the frank post-mortem from the customer you lost, but the feedback is critical to learning and development of high-performing sales people. Use the interviews and the information to coach and train and make it clear that the interviews not used for rewards or punishment — only for insight into the sales process to help the company be more successful overall.

More Resources

Win-Loss Analysis Adds Value to Tech Sales Initiatives

I’ve had a number of people recently ask me, “What’s the value of a win-loss analysis program to a sales team?”

After a little thought, I’ve decided to write my first blog to answer this very question with a concrete example that demonstrates the value that a win-loss program brings to a sales organization.

Case Study: Employing Win-Loss Analysis Data to Close a Deal

A sales executive for a leading enterprise software company is proposing a seven-figure deal to a Fortune 100 retailer looking to leverage predictive analytics to deliver real-time offers to its customers. To gain a competitive advantage and improve her chances of winning the business, the software executive turns to her marketing department’s win-loss program. With this win-loss initiative, her company engages a consultant to interview a sampling of the deals that the company competes in – both wins and losses. The marketing team leverages this win-loss data to produce competitor-specific battle cards (“attack plans”) which are then provided to the sales team.

The software company’s competitor is a large stack vendor.

Step 1: Review Relevant Win-Loss Interviews

Initially, our sales executive thinks that her proposal should emphasize the flexibility of her company’s data mining, modeling, and scoring technology. When she reviews a couple of recent win-loss interviews in the win-loss analysis portal with a retailer, however, she learns that the most important buying criteria was the vendor’s ability to implement the solution and the customer’s ability to manage the solution once it’s been rolled out.

Step 2: Research the Competitor

Next she reviews the battle card for this competitor. She learns the competitor’s solution is comprised of three recently acquired technologies. Although the solution is powerful, it’s overly complex. The competitor also requires that its own data management platform and data preparation tools must be used as part of the solution.

Step 3: Craft the Proposal

Based on the findings of the win-loss program analysis, the sales executive changes her approach.

  • Working with her services team, she focuses her proposal on the strength of their implementation.
  • She provides examples of successful projects that were on-time and under budget.
  • She proposes a Center of Excellence (COE) staffed by members of her team to support the adoption of her company’s technology.
  • She structures the program–with governance and executive oversight–to ensure project milestones are reached.
  • The solution architecture will leverage the customer’s existing data systems and infrastructure.

Of course the proposal still mentions her product’s rich tools and flexibility, but, thanks to the data drawn from the win-loss analysis, the approach is far more consultative than her original direction.

Step 4: Exploit thee Competitor’s Weaknesses

Based on her company’s win-loss program’s competitor battle cards, she also plants concerns about her competitor’s product:

  • It requires its own data management platform to be installed — which is costly, slow, and disruptive
  • The solution is overly complex
  • The solution architecture is still changing
  • Few, if any, customers have used all three of its acquired products together

Armed with the knowledge of similar retail buyer’s needs and her competitor’s weaknesses, she was able to assemble an effective account strategy — and win the deal.

When you consider how this win-loss analysis program enables an entire sales organization to compete more effectively, you can begin to appreciate the value that a win-loss analysis program has on company’s sales.

8 Tips on Conducting Win-Loss Analysis Interviews

I love interviewing people. You learn a lot. Most people are in fact enjoyable to talk with. And, the subject matter is fascinating. I’m not going to give away ALL of my secrets on performing win-loss interviews, but below is a list of what I think are some really important ideas to employ when interviewing people.

1.  Remove as much bias as possible

You want to get respondents to tell you their thoughts – without influencing them to give certain responses. Minimize the use of answer “lists.”  If you provide a list, the respondent might select something that they thought they should have done, instead of what they actually did.  Or the list might be incomplete, missing something important, etc.  In order to determine the selection criteria, ask the respondents to explain what was important to them – in their own words.

You can prompt their memory afterwards by reviewing criteria other respondents have found to be important.  But do that after they give you their criteria.

2.  Get the respondent to open up and relax at the beginning of the win-loss interview

I like to start with open-ended questions about the context of the respondent’s decision – including the business need, the process that was used to make the selection, and the respondent’s role, etc.

For instance, my first question is most often “What motivated your company to look into solutions in this space?” These open-ended questions help your respondent to relax, open up, and start talking. Please, when your respondent starts talking, don’t say a word.  Listen only.  And let them talk as long as they want without interrupting them.  Resist the urge to interject comments or questions. Take notes and ask any clarifying questions after they are done.

3.  Build rapport

Enjoy your time with the respondent. Have a laugh together. Relate to them.

4.  Show that you understand your respondent

Who wants to spend their time explaining a complex thought to someone who doesn’t seem to get it? I like to give an indication with a short remark that shows that I understand and appreciate the respondent’s perspective or experience when they are explaining something.

Do your homework before you begin your win-loss interviewing. You need to be well-versed in the market.  Review the key players and their differentiators.  Read the related analyst reports, competitive documents and the product (or service) descriptions, etc. If you are talking with someone in IT, you need to have knowledge of the technical issues. If your respondent starts talking about their integration architecture concerns and you don’t have a clue about REST APIs, OAuth 2,0, and LDAP – how can you ask the related follow-on questions to probe more deeply into the technical issues?

Similarly on the business side, you need to be able to speak the language of your respondent, and engage with them on their concerns.

5.  Be interested in what your respondent has to say

If a respondent says to me “the sky is green”, I might follow up with, “Hmm… that’s interesting, why is the sky green”? Again, you need to be conversant and interactive with your respondents if you expect to learn anything from them.

6.  Get both qualitative and quantitative data

To be most effective, you need both qualitative and quantitative data. Purely qualitative interviews have value, but they lack the ability to prioritize the importance of the information.

For example, you need to ask for the scores of each vendor across the criteria. “And for Company-X, how would you score them on ‘Ease of Use’”? Without scored responses, it’s very difficult to do a detailed, non-subjective analysis.

7.  Ask my favorite question

I often ask this question towards the end of the interview. “If you were meeting with the senior management from each of these vendors, what would you tell them they need to do in order to improve – and to better meet your needs?“

8.  Make sure you are courteous, prepared, and professional.

You only have one shot… So get it right the first time.


Additional Win-Loss References:

A Win-Loss Consultant can be Your Secret Weapon

You Need More Than ‘Internal Sales Debriefs’ To Increase Competitiveness… You need a win-loss consultant to help you run a real win-loss analysis program. 

As we’ve recently spoken with a number of potential clients about Win-Loss Analysis Programs, more than a few people have told us that their company is already doing Win-Loss internally.  When asked for details, we’ve found that many companies think that they are doing Win-Loss when they are doing Internal Sales Debriefs instead.

While these internal debriefs serve a purpose, they are not a substitute for a professional win-loss analysis program.

An “Internal Sales Debrief” is a post-deal discussion with a sales rep (or sales team) in which the details about a deal that recently closed (either won or lost) are discussed. It may be supported by a questionnaire. It’s most often done by the sales manager, but it can be done by product marketing or a competitive intelligence analyst as well.

Debriefs are an important activity and serve a purpose. The main beneficiary of the debrief is the sales team and the management who need to discuss what transpired, review their strategy, discuss the customer’s needs, the power base, and review what the competitors are doing in their account. In the case of a loss, the reasons for the loss should be discussed, etc.

Debriefs are not only helpful to the sales team, they are helpful to the Competitive Strategy and Intelligence team as well. If you attend enough of them  you will find examples of how a competitor operated, and that information should be incorporated into your competitor-specific battle cards.

However, if you are trying to become more competitive you need something more than debriefs, which have limitations, can be misleading and incomplete, and don’t give you a complete story.

  • Telephone Game In a debrief, you are not talking directly to the customer. Instead you are gathering data indirectly through the sales team, and that means that some of the information may not get passed on, or it may be modified before it gets to you. It reminds me of that telephone game I played as a kid – where you whisper something in someone’s ear – and by the time the information comes back around to you, it’s completely different.  And keep in mind that sales reps may be getting their information from someone other than the decision maker, such as a “friendly informant” (or “fox”), or even a partner who is speaking to the fox, etc. You get the idea.
  • Perception rather than fact You are asking the sales rep about his or her perception of what transpired. Some of the information is subject to interpretation.
  • Losing reps know far less than winning reps This may be obvious, but the losing sales reps know far less about the customer’s needs than the winning reps. But you do need to understand why you lose, so that you can understand your weaknesses and work on improving them.
  • Losing reps are concerned about being judged The data you seek in a debrief is masked by the sales rep’s need to be perceived as being valuable to the organization. As a result, some of the data might be “positioned” or changed before it gets to you. And when your losing sales reps are asked questions that they think they should know, but don’t, the losing sales rep may provide guesses. Hopefully they will precede this with “I don’t know that answer”, but these reps are very concerned about how they are being perceived…
  • Most losing sales reps will over emphasize “price” as a reason for losing
  • Sales teams are often not objective about their own performance It’s hard for sales reps to see themselves objectively.  And the problem is further reinforced by the fact that the customer will often not discuss what they didn’t like about the sales team directly with that sales team. Occasionally a sales rep will say “I screwed up”.  But how many times can that be reported before that sales rep is gone?
  • Winning reps use debriefs for self-promotion Have you ever asked a winning sales rep why they won – and the answer was “relationship”?  I remember asking one sales rep who clarified the response by saying “it was ALL relationship”.
  • Your team doesn’t know everything about the customer Your customers won’t tell your sales people everything because they are negotiating with your sales people, or using them for some other purpose. For instance, in some cases, your participation in a deal might be column fodder.  It could be that your participation is designed to lower the price of another vendor. Or it could be that the prospect loves your product so much that they’d have paid twice the price.  There are numerous reasons for why the customer may be holding back information from your sales team.
  • It’s impossible to get the normalized data you seek from your sales team. Except for the occasional spreadsheet that the customer has produced, which finds its way to the salesperson — it’s unusual for the sales team to deliver anything in terms of a full selection criteria list, the relative importance of the criteria, and ratings of the competitors across the criteria, etc.

By contrast, a win-loss analysis program is specifically designed to provide insight and data so that you can identify the real reasons you win and lose, and provide the necessary recommendations across the organization to increase your competitiveness. Win-Loss is not easy to do, but this is the type of discipline that’s required to have a world-class winning team.

What needs to be done differently from an internal sales debrief approach?

  • Interview the Customer directly. The customer is the source of information you seek – so why not debrief the customer directly and bypass some of these issues?
  • Use an experienced, external win-loss consultant. There are a number of reasons for using an external consultant for win/loss interviewing and analysis.
    • Perform unbiased, anonymous interviews. Customers don’t want to disclose certain information to the vendor. It may seem counter-intuitive, but when customers engage in anonymous interviews, they will discuss the issues in a much more open and candid manner than they would if they know they are speaking to the vendor.
    • Remove as much bias as you can. Another benefit of using an external service, is that the approach provides you with an independent, unbiased interview. In contrast, having someone self-report the details of their performance has too much bias and spin.
    • Leverage the consultant’s experience and methodology.  There will be issues that will come up in managing this type of program. If you haven’t done it before, it’s fairly difficult to analyze win/loss data and make conclusions. Experience can help you avoid mistakes that might end your program.
    • Fast Results. A win-loss consultant should be able to work through all of the issues of starting up your program very quickly.  In addition, a win-loss consultant will have the tools, the team to schedule meetings and editor your interviews, and software that can be used to distribute your results quickly to your stakeholders.
  • Gather the right normalized data in a consistent, systematic manner. In order to help your organization become more competitive, you need to be able to draw conclusions and provide actionable recommendations.  Therefore, you need an approach that allows the data to be analyzed across the interviews. Your interview data should include – the customer’s selection criteria, the criteria importance weightings, and scores of the vendors across the criteria – along with the reasons behind the values that the customer gives you and your competitors.

For those of you who still think that an internal sales debrief may be good enough – consider this easy test. Compare the two approaches yourself.

But make sure your comparison includes two key aspects:

  • Interviews. Do some customer win-loss interviews conducted by a win-loss consultant, and then compare them to your internal sales debriefs. Try doing this on the same deals. We recommend a sample set of at least 5 interviews (2-3 wins and 2-3 losses).
  • Analysis. Compare the analysis that can be generated from an internal sales debriefs with an analysis generated by direct customer win-loss interviews. You can do this on the same set of 5 interviews.  Keep in mind that you probably won’t have enough data for statistical significance on such a small sample set.  However, you can still understand the process of how the analysis is performed so that you can get a sense of the types of conclusions that can come from one approach over the other.

Internal sales debriefs are useful – but they are very different from customer win-loss interviewing and analysis.  You need the latter to get the facts directly from the source, in an unbiased and complete manner; then you can draw conclusions and make changes in your approach that will lead to improved competitiveness and increased sales performance.


More Research