A proof of concept (POC) is smartest way for customers to evaluate if a product meets the required objectives, and the best way for vendors to demonstrate why they feel they are best placed to resolve the current outstanding problems. However, not all POCs are successful. Let’s explore why.

What is a failed proof of concept? 

A failed POC is one that has one of the following end results:

1. Vendor(s) fail to prove the concept as originally conceived.
2. Concept is proven but does not provide the expected outcome in terms of value.
3. POC fails to satisfy the intended stakeholders.
4. POC results inconclusive leaving the customer confused and the vendor frustrated, both claiming they have done little wrong.

How to avert failures or fail fast?

I’ve seen at least a few dozen POCs fail to achieve desired objectives, and quite often you see a common pattern among the failures. It is technology agnostic problem and hence cannot be attributed to any particular technology. Rather, it’s often a communication or planning problem. When POCs fail to satisfy the requirements of the customer and vendor, usually both are looking at it from a different perspective.

This blog post will discuss the customer side of the POC process and explain the key reasons that often cause failed POCs.

What do customers and vendors expect from a POC?

The customer, after the extensive RFI and RFP process, wants to ensure that the promised functionalities are there, and all expectations are met. The customer wants to ensure that the product is easy to use, and workarounds aren’t used to meet the requirements.

The vendor, who normally assigns resources to fulfil customer requirements and pays for the POC has a lot at stake as well. He wants to ensure that he is able to meet the customer’s requirements and expectations and demonstrate that the tool is capable of fulfilling the needs as laid out during the RFP process.

On the surface, this seems to be a fairly simple process where the customer and vendor needs align. However, common errors can lead to failure. From my experience, POC’s fail due to the following major reasons:

1. Misunderstood requirements.
2. Lack of executive sponsorship.
3. Costs involved.
4. Lack of ownership.
5. Poor change management.
6. No definitive endpoint.

Let’s look at each in more detail.

1. Misunderstood requirements:

If you take a step back, you will realise that the entire POC process started because a business problem needed solved. The current tools seemed incapable of fulfilling the business need, so a search began for new software or hardware. Customers and vendors should treat the requirement stage as the foundation of a 100-story building, and spend as much time here as needed to understand exactly what the problem is. You need to understand the business case and have a clear understanding of the potential benefits if the business problem is resolved. If you don’t understand the problem resolution, the potential solutions and the value of solving it, please don’t go any further. The POC is not a place where you refine your problem (it does happen a lot at the cost to both the vendor and the customer), but where you prove your concept, which is already a proposed solution.

2. Executive sponsorship:

Once you understand the solution architecture and potential benefits, you need to solicit executive sponsorship, and understand who will write the cheque for the POC. You need to understand if you will have a budget for it, and who the key people in the process are. It will not benefit anyone if one or more vendors fulfil your criteria, but you as a customer do not have a clear plan to go forward.

Never start a vendor evaluation process until you have a clear understanding of the requirements and have identified the person who will eventually need to write the cheque.

3. Costs involved:

It is important for the customer to understand the costs involved with a POC.

The costs are both real costs and opportunity costs:

1. Opportunity cost of executive time on the POC.
2. Opportunity cost of your subject matter expert’s (SME) time on the POC.
3. Cost of vendor equipment (as applicable), including:
   1. Hardware costs.
   2. Resource costs of setting up hardware and software on premise or in the cloud.
   3. Cost of heating and electricity.
   4. Potential security threat costs.
   5. Software costs.

From a customer’s perspective, the POC will not be successful if the key SME’s are not involved in the process. Quite often the key people are the busiest people, and have negative rather than zero availability. This means that even in their current day jobs they are working overtime, and have little time in their calendars to embark upon another activity. The customer needs an executive sponsor who is involved early in the project. These people will be the key to success. In many cases, when executive sponsor is unavailable, random people from teams are assigned to tasks, which not only hinders the POC progress but greatly increases the risk of failure.

Key IT security and administration people need to be involved early as well, even though the rigor of a production solution will not apply to the POC.

4.　Lack of ownership:

Understanding the ownership of the POC is of critical importance and is often the least talked about due to its sensitive nature. I’ve seen political situations within customer environments create a lack of any serious ownership for a POC, which can lead to no ownership or too much ownership. After executive sponsorship, you need to have a proper owner of the POC. A project team needs to be formed at the customer site, with a project manager assigned for the duration of the activity.

5. Change management:

Change management should be a key focus before, during and after POC execution. You need to understand which teams will be affected with the possible procurement of new software and try to communicate the long term vision and objective as soon as possible. If you believe that this is a replacement system, you need to quickly understand if you will have to retrain existing staff or hire new staff. A lot of the time, POCs fail because the internal teams fear that the change brought on by the new system is going to affect their jobs, and in some cases they expect the worst case scenario of being laid off.

6.　End point:

All the POCs need to have a definitive end point. It is important to end the POC as soon as the desired objectives are achieved, and time should not be spent on gold-plating. If you conclude early that the POC is not going to be successful, it is important to call it off as early as possible thus saving time for the customer and the vendor.

Conclusion

Before embarking upon a POC it is important to keep these issues in mind. POCs are definitely a promising way to identify if a vendor can walk-the-talk and meet your requirements, but unless you are ready, no matter who the vendor is, the entire exercise will be futile and is doomed to fail.

Learn more about our process for assessing customer needs - and delivering results.

This blog was a combined effort of Adrian Jones, Yigit Karabag, Marcel Lemahieu & Asif Abbasi. Please stay tuned for the vendor perspective.