Thursday, March 20, 2014

The Next Generation of Big Data Recap

One hundred people gathered at Microsoft NERD for the Big Data Cluster’s seminar on February 28, 2014, where the morning was kicked off by Sastry Chilikuri, Partner at McKinsey & Co. Sastry’s keynote went through four lessons he has discovered are critical to a successful big data program.

  1. Transformation journey. Communication and access to data is key.
  2. Open data. Governments are moving towards open data platforms and sharing more information with the public. Companies are using social media more and more to gather insights on their customers.
  3. Organization and talent. It takes a team of people who can bridge the different gaps required for a successful big data workflow.
  4. Frontline adoption. The end result must be to take advantage of the insights gathered through the data.

Sastry closed his keynote remarking that those organizations who are most successful in the big data journey are those that master the feedback loop. From there our panel got started in on the trends they are seeing and where the biggest opportunities within big data can be found.

Marilyn Matz, Co-Founder and CEO of Paradigm4, sees one of the biggest opportunities for organizations is to take advantage of the multiple data sources now available to them. Jon Pilkington, Vice President of Products at Datawatch spoke of using a combination of real time and stored data to garner insights and then to have the ability to display to the end user. Iran Hutchinson, Product Manager and Software/Systems Architect at Intersystems talked about the need for interoperability and the need to work within a single platform.

Bob Zurek, Senior Vice President of Product at Epsilon went in a slightly different direction, away from data, tools, and platforms and spoke about developing the next generation of data scientists and the adoption of new academic curriculums being offered.

Tools and Applications and Workforce
There was some slight debate on whether one size can fit all within the big data stack. Generally organizations must serve multiple needs across multiple consumer types and there is no one software that can accommodate everything, which is why many organizations use a combination of tools and applications. And with that, we’ll be seeing more pre-packaged and less complicated tools which can be used by business intelligence specialists not just data scientists.

Ultimately the panel understood that to create and implement a big data strategy – ultimately to get the
data to where it needs to be - is hard and you need to hire the people that know how to do it. And what can often help is a large integrator to identify the right tools for the stack and then put them into process.

Final take-aways
Big data is not a new phenomenon. Business intelligence has been done for many years. But the questions we ask and the answers that are provided are more complex. There needs to be a focus on business and an understanding that there are no shortcuts. But overall we have an amazing opportunity with so many benefits to the way in which we work and live.

No comments: