PoCs - Protecting your investment with the open road test

In a recent conversation with a vendor, I was asked if there was any way in which we [Celent] could encourage/support the standardization of request for information documents sent out by insurers. Think ACORD Standards for RFIs! At the heart of her question, was a sense of deep frustration at the evaluation process which inhibits the vendor from highlighting or differentiating themselves. I have some sympathy with this view.

IT departments have become very good at comparing the data for RFI/RFPs in two-dimensions. A structured, transparent evaluation process is an important step in choosing what it likely to be a significant IT investment in the case of a core system replacement. The evaluation process can be used to generate hard facts highlighting the differences in system functionality, company and delivery options.

For many insurers, the governance process requires a presentation to the project sponsors of a detailed quantitative analysis of vendor submissions, along with ranking and weightings. This is all well and good but this output can be incorrectly attributed the status of a statistically sound scientific evaluation. I have seen projects where a vendor has been chosen because they had 2 more points in the final analysis on paper.

Most of us would not buy a car based solely on a review of features and functions on the manufacture’s website or brochure. We’d be out there driving and deliberating the real experience of the open road, handling of corners, and comfort of the seats. All these factors are almost impossible to quantify in a brochure but vital in influencing our final choice.

And it’s similar in the world of core system evaluation. The proof of concept is vital addendum to the evaluation process. This is the time to put the vendor to the test. This stage of the evaluation should allow the buyer to really see how easy the product configurator might be, how well a vendor can understand requirements, how responsive/ organised the vendor is, or how much/little coding is required.

And in Celent’s view, proof of concepts should be paid for. It shows the commitment on part of the insurer, and it also encourages the vendor to be focused and committed to this important step. Being paid allows the vendor to justify allocating top staff otherwise committed to paid implementations. A recent evaluation project paid 4-figure sums to both short-listed vendors for a 3 month proof of concept.

The proof of concept is a great process by which to garner more buy-in from the business users, and to generate some excitement about what this change could bring to the organization. Adding the open road test to the brochure comparison will undoubtedly help the insurer make a more robust decision.