There has been much written about the emergence of Big Data, both as a phenomenon and as a set of practices, infrastructure, and algorithms designed to allow modern computing to analyse ever-increasing data in fast and efficient ways. Celent uses the open source Hadoop solution to provide insight into what Big Data is, what it means to the insurance industry, and the kinds of solutions it provides to new and old insurance problems.
In a new report, Big Insurance Data: Drawing Lessons from Amazon, Google, and Facebook, Celent reviews how Big Data is now relevant and accessible to the insurance industry. In this report, Celent offers a discussion on the following topics:
- Grid computing or the use of many distributed, connected machines to undertake large-scale work.
- Distributed file systems, which store large amounts of data across many machines.
- MapReduce, which is an algorithm to split workloads over grid computing.
- Hadoop, which is a specific implementation of the MapReduce algorithm.
“The simple fact is that, with little more than a credit card, today you can rent an infrastructure from Amazon similar to that used by Yahoo to index and search the entire Internet,” says Craig Beattie, Analyst with Celent’s Insurance group and author of the report. “It is important that insurers understand just how much these frameworks do on their behalf and how easy it is to leverage this infrastructure to ask big questions of our data.”
The report draws together recent developments in cloud infrastructures, grid computing, and distributed computing to address issues regarding large volumes of structured and unstructured data. While there are many approaches to leveraging grid infrastructures, this report details one of the most popular approaches and perhaps one of the simpler approaches to implement. While the topic is necessarily technical in nature, the report addresses it in an accessible manner.
