Tuesday, March 17, 2015

Big Data and the Knowledge Worker Summit - Feb. 27th Recap

Written by: Udi Dotan

Some of the best and brightest in the Massachusetts data and technology space gathered on the last Friday in February to discuss how the world of big data and computers might impact knowledge workers in the 21st century keynoted by esteemed author Tom Davenport.

Tom spoke on the displacement of workers over time.  In the 18th and 19th centuries, it was farm workers, in the 20th century, it was service jobs, today, it's knowledge workers that are being displaced by technology.  Artificial intelligence is growing and taking over roles that humans used to do and they will take over more work.  Computers can do many tasks faster and more efficiently than humans and computers are cheaper, easier to manage, and don't complain about the cost of healthcare. 

Should we welcome these changes as another in a string of technological revolutions that have enabled humans to flourish, or fear our new computer overlords?  Some technology leaders such as Elon Musk and Bill Gates have voiced concerns about these changes positing that AI is the most dangerous development in history and should be looked upon with skepticism.

Where do these fears come from and how did we get here?
Since the dawn of the age of the internet 20 years ago (yes, it's been more than two decades since you saw your first AOL CD), information has been generated and has flowed more freely.  With more data, they has been more desire to analyze which led to enormous growth of analytics.  Early analytics were descriptive, utilizing simple graphs and charts to understand our world. 

Today, companies are utilizing predictive and prescriptive analytics to help make better decisions (think Amazon's recommender engine).  Going forward, more and more companies are leveraging larger stores of data, more compute power, and sophisticated algorithms to automate analytics.  One such example was given by Ed Macri, Senior Vice President of Marketing and Analytics at Wayfair.  They are using analytics to personalize one million emails a day based upon their prior visits, versus a single email carbon copied to one million people.  

Another example given by a member of our keynote panel, Bruce Weed, Program Director, Global Watson and Big Data Ecosystem Development at IBM, is how Watson, who through the use of its massive library, is helping medical doctors diagnose in a much faster and efficient manner. 

What jobs are computers doing that have or will displace humans and how do we service our new masters?
Of course, no human is capable of personalizing a million emails.  But these aren't the only roles that are ripe for computerization.  According to Tom Davenport, here are some "at risk" jobs that computers can and will do better than humans:
  • Lawyers: e-discovery - combing through thousands of documents to find the nuggets of truth for specific court cases.
  • Accountants: audits, taxes - using intelligence to improve tax preparation (think TurboTax).
  • Radiology: cancer detection - using machines to read radiology reports and highlight areas of concern.
  • Reporters: automated story generation - computers can used data to generate articles for publishing (like this one? - not yet).
  • Marketing: online ad buying and personalized emails as with Wayfair
  • Financial Advisor: "robo-advisors" - generating customized portfolios for clients based on factors such as age, income, and tolerance to risk.
  • Teachers: online content and automated student evaluation - companies such as Kahn Academy, Coursera, and EdX are delivering content online.  Next generation companies such as Dreambox and Knewton are delivering adaptive learning that modifies the material in response to student performance.

Have the machines already won or is there a role for us yet?
Some companies are already leveraging intelligent machines, has this turned their offices into a wasteland where tumbleweeds are rolling through giant data centers?  In short, no, there are still plenty of things that computers can't do without us.  Davenport refers to this work as augmentation.  Computers are good at computationally complex and repetitive tasks, but they can't see the bigger picture.  Humans will be needed to identify the strengths and weaknesses of the analytics systems and algorithms.  Humans will be needed to determine the business problems to solve.  And humans will build and maintain the systems to solve those problems.

As highlighted at the conference, big data technology enables much of the analytics innovation as companies can manage with larger and more varied data stores.  Jeff Kelly, Principal Research Contributor at Wikibon believes that we are moving from early stage adoption of big data implementations built around cost savings to a second generation whereby companies with big data strategies are now focused on revenue generation and operational efficiency.  P. Gary Gregory, SVP & GM, Database Servers and Tools at Rocket Software illustrated that to be successful with such data initiatives, you need to start with a business problem and build data systems to support solutions.  Those systems don't need to include hadoop, but the purpose of the data and the definition of the data sources should be clear, otherwise you end up with a data landfill, not a data lake.

Several of the panelists illustrated that the analytics revolution has led to a greater need for humans, not a lesser need.  Ivan Matviak, Executive Vice President and Head of Data and Analytics Solutions at State Street Global Exchange says they are hiring more people, not fewer, to help build and maintain its advanced analytics capabilities.  In particular, they are aggressively seeking to hire the sexiest workers, data scientists.  EMC's data science practice spends a great deal of effort investigating and rebuilding messy data for analytic purposes. And Wayfair is augmenting automated ad purchases with targeted human buys of online ad space.

Iran Hutchinson, Product Manager & Big Data Software/Systems Architect at InterSystems led a lively panel discussion illuminating success stories at companies leveraging big data and human augmentation to gain remarkable insights.  Joe Dery, a Senior Data Scientist at EMC relayed how EMC increased revenues by mining internal contract data to optimize contract renewals.  The key to optimization was not in the volume or veracity of the data (although there certainly were large volumes of data), but rather in clarifying data definitions and educating the sales team.  According to Joe, the model generation was the simplest part of the two year project.

Gary Sloper, VP of Sales Engineering and Operations at CenturyLink uses big data to proactively monitor network activity and utilize machine learning algorithms that can detect anomalies.  Such techniques can be employed to prevent hacks such as Sony and Anthem have recently experienced.

At Care.com, Co-founder and CTO, Dave Krupinski and his team has focused analytic attention on optimizing the match rate between jobs posted and caregivers seeking jobs.  This has given them guidance on the optimal flow of applications into a job posting, the optimal number of applications per job, and the key terms that are more likely to get a caregiver hired.  The insights have led to an increase in match rate from 70% to over 80% with more opportunity to improve in the pipeline.


As the volume, velocity, variety, and veracity of big data grows and the analytics become more complex and the opportunity for a cooperative relationship between machines and humans will continue to grow and we will continue to find ways to employ technology to advance society.

1 comment:

Raju Kumar said...

I really liked this part of the article,
with a nice and interesting topics have helped a lot of people who do not challenge things people should know...
Hadoop Online Training