As the discussions around Big Data continue, business professionals are becoming more aware of the challenges and pains of using analytics within their companies. Managers, professionals, and executives are all facing similar problems around investing in expensive IT infrastructure, training their workforce to understand analytic technologies, and rolling out strategies that take into account dashboards and model results. Dealing with these problems is not easy, and a new concept is beginning to emerge in this space: Prescriptive Analytics. If built properly, Prescriptive Analytics tools can help companies deal with the talent shortages and lack of ability to apply models to generate business results.
Prescriptive Analytics is the combination of data mining, model results, business rules and other heuristics to automate the use and application of analytic results. Rather than obtaining model results and deciding how to use them, the workflow is automated to act on the decisions and recommendations automatically. In other words, models (and the IT systems that build them) are designed to actually complete a task or workflow as a result of their analysis.
Case Example: Product Recommenders
A clear and poignant example, and one that nearly everyone is familiar with, is the use of product recommendation engines. Amazon and NetFlix are businesses that have succeeded by providing product recommendations to their customers. Their internal workflow is also fully automated: data is collected, models are built and updated, and their final recommendations are pushed back to the users visiting their websites. While scientists monitor high-level model results, no one is responsible for ensuring that your specific recommendations are updated on the website. The analytic results, associated content, and delivery of said content are all controlled through an automated workflow.
What is brilliant about this case example is that the models prescribe what to do (i.e., which product to recommend), and an automated system actually does it (i.e., the website tells you what to buy next). Big Data meets Prescriptive Analytics, and billion-dollar companies are born.
Big Data, Meet Prescriptive Analytics
While IT infrastructure and analytics tools are a major challenge when it comes to properly using Big Data, these challenges are currently being solved by numerous startups and even open source platforms like Hadoop. The impending problem will be talent shortages and managers not being able to find enough individuals to use the analytic tools properly… IT implementation and scalable infrastructure are just the start of the Big Data value chain.
The challenge with applying Prescriptive Analytics to Big Data is ensuring that the right problems are being solved. The Amazon and NetFlix product recommendation case study is one example, but we are only now starting to see how analytics can be automated to actually implement optimal solutions. Well-known examples, like product recommendation engines or High Frequency Trading (used in investment banking) are well-known, but such examples are still few and far between.
To actually achieve widespread use, Prescriptive Analytics solutions need to overcome a number of challenges. Simply put, many individuals are afraid to outsource what they do to a machine that may or may not know better. In some cases, certain industries and functions really are an art — try outsourcing investigative journalism or cooking a good meal, and you might be unpleasantly surprised with the result. It’s not always clear which business problems can be outsourced to an automated system, and which ones need a human touch.
So where to go from here? The big opportunity for startups, corporations, and business executives is to know when they can automate entire workflows, and where they need a trained analyst to oversee model results. Big Data, combined with Prescriptive Analytics, is how we’ll make the Big Data craze productive while avoiding the talent shortages so many companies fear.