Analytics Part 3: Looking Beyond the Algorithm

Central Role Humans Play in the Age of Analytics

No news flash here, data is growing and an ever accelerating pace and the option of keeping ones figurative head in the sand has long since become obsolete.  Analytics, algorithms and something called Hadoop (high throughput, low latency) are intimately integrated in a wide array o business functions, however they cannot solve this big data problem with out high level experts that can ask the right questions, interpret the output and decide what to do based on what once of these technologies uncovers.  This is an era where it is the intersection of man and machine that determines success or failure.


For companies to truly ride the wave of the analytic revolution  requires a human push, .Sexy new tools may be in the spot light, but the people behind the sexy analytics are the cornerstone for success no matter the business vertical we look at.  High level technologists and knowledge workers need to  driving adoption, and utilization of cutting edge tools no matter how much companies would prefer to avert change.

It is clear that data is increasing at a rapid pace and analytics are necessary to attempt to best leverage this mountain of information.  The former Chief Scientist at Amazon, Andreas Weigend, estimates that the amount of data available about each American doubles every 18-24 months. Much is held by enterprises. In all but two of the U.S. economy’s biggest 17 sectors, companies with more than 1,000 employees store on average more data than the US Library of Congress.1 As data rapidly expands, storage is becoming cheaper. Kevin Kelly, editor of Wired,told the 2011 Web Expo and Conference that a $600 disk drive can store all of the world’s music. A terabyte of disk storage cost $14 million in 1980, costs $30 today, and is moving towards being available for free.

Companies that many of us deal with every day are already making use of data to advance a variety of business goals and to help consumers:

analytics comapnies

  • Kaiser Permanente collects petabytes of health information on its 8-million-plus members, a fantastic amount. Some of this data was used in an FDA-sponsored study to identify risks with Vioxx, Merck’s pain medication, which was pulled shortly after the research identified a greater risk of heart attack in a subset of the patient population.
  • Southern California Edison is collecting hourly (rather than monthly) data on customer usage from new digital smart meters in millions of residences a significant benefit for energy grid management and customer service.
  • Pepsi has an ordering algorithm that lowers the rate of inventory out-of-stocks. The company shares information from this application with partners and retailers, improving its relationships with key stakeholders.
Analytics, Analytics Everywhere

Whether you look in retail, banking, industry or even our own legal space, the massive and ongoing surge of data is driving increased adoption and exploitation of analytics.  But the story does not end here, the pivotal component that is allowing for rapid adoption and deployment of sophisticated technology centric solution for the big data questions is the emerging intellectual capital of key knowledge workers.  Whether its Pepsi, Best Buy, JP Morgan, John Hopkins, or somewhere in big law, analytic solutions require keen intellect to apply, modify and react to the output of big data analytics.

Screen Shot 2013-03-12 at 2.17.11 AM

GigaOM Structure:Data conference

Even at the bleeding edge of application and development of sexy new analytics and applications to crunch through incomprehensibly large volumes of data this fact is not lost.  According to Jordan Novet, in coverage of last week’s GigaOM Structured Data Conference:It is true that with Big Data, as with everything else, there is no easy button.  However, even if there was an easy button a human would need to push it.  The single unifying characteristic across the myriad of applications analytics across the business verticals is the central role that intelligent, well-trained knowledge workers play in application of analytics and understanding what their findings mean.

A few trends emerged in more than 30 talks at this year’s GigaOM Structure:Data conference in New York on March 20-21. The big one: people play a crucial part in the big data equation. 

A key theme across the 30 talks was that a machine can do amazing things if given the correct Human guidance, in eDiscovery an algorithm is only as effective as accurate training in other applications the machine needs to know what question to answer as it splices through a mountain of data.  Humans need to decide which algorithms to apply and how.  And once the system comes back with a smaller still messy set of data they need to know how to use it.

Applying Analytics in a meaningful way as a company requires a human push according to  Paul Maritz, chief strategist at EMC. “Change requires leadership. It requires people to understand what is happening and really get behind it and drive organizations to transform, because none of us really like to change”.

Meanwhile, Amaya Souarez, director of data center services at Microsoft,notes that lots of internal data doesn’t automatically effect changes in strategy. “The data will help …but it’s not everything, t really does take a lot of personal interaction and commitment” to garner insights.

We are no longer at the cross-roads of if analytics will be used, rather we are at the point where along the spectrum of analytic solutions, we, as practitioner and leaders in the space, need to determine which tools work best for each specific matter.  Blindly plowing through the “old way” is cost prohibitive and akin to sticking our head in the sand if analytics are not at least considered on a case by case basis and used when the size, scale, velocity and timeline dictate.  The person at the steering wheel in this decision-making process is critical to the success or failure of an initiative in the age of analytics.

In part 4 of this series, we will look at what it takes to be a knowledge worker in the age of analytics.

Part 1, Part 2

By: Cat Casey

Cat Casey




Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: