Here is an interesting article about Big Data and a start up here in Massachusetts that talks not only about the technology, but the career and educational requirements to create the manpower to maximize the technology. I'd love to hear thoughts if you agree or disagree that this type of training is required and the opportunities or distractions you see from it.
Let’s wander outside our Boston bubble for a moment and direct our attention to Lowell. That’s right, Lowell. It might seem like the last time that city was any kind of beacon of innovation was back in the Industrial Revolution. But there’s actually a startup out there now that’s doing big things with that beloved buzzword: big data.
Podium Data has created a solution that leverages the power of Hadoop - the software framework that lets people pool and store massive datasets - but ensures companies can find the information they need immediately when they need it. Essentially, they’re enabling businesses to tune out all of the unnecessary noise found in big data to have ready access to the data they’re actually looking for - and without the help of an IT army. So far, the company has secured $2 million in seed funding, as well as big name customers like TD Bank.
Paul Barth, MIT PhD...
As I spoke with Paul Barth, co-founder and CEO of Podium Data, about his venture, it became abundantly clear that he’s been privy to the evolution of data and how we use it over the past several decades. So, I took the opportunity to get his take on where big data is bringing us and how people can best prepare ourselves for the ride.
Barth, who has a PhD in Parallel Processing from MIT, has seen data become king at companies. While businesses of different sectors and sizes now want to leverage data to be competitive, they’re not always equipped with the IT manpower to do so. With that, the field of data science has emerged and become incredibly important for companies. Barth explained:
There’s a term out there today called the data scientist associated with big data...They have a pretty good understanding of the business, how it operates, it runs, how it makes money, what optimization would mean in a business sense. They have a good understanding of statistics and machine learning analytics, and they know how to program in Hadoop. To me, this is the other definition of a unicorn: There are very few of them, if any, and I have never seen one. That actually is a bottleneck for big data. If you have to find people with those skills or even build a team with those skills, they’re in super high-demand and I think those people are great. I support the STEM education that gets you there.
He said there have been studies showing that, in recent years, the number of Master’s programs in data science has grown from a mere handful to more than 170. Although there’s still ample demand for programmers, Barth clarified that data science involves more than knowing how to make a machine do something. It’s a marriage of hard STEM skills and human analysis, which is increasingly coveted in the job market.
“I think it’s seeping into the curriculum,” he shared. “When my kids asked me in high school, ‘Should I take statistics or should I take calculus?’ I said, ‘Take statistics.’ It’s a lot more relevant in the future. Calculus is cool. It’s a beautiful field in math, but, unless you’re going to be a mathematician, calculus is already embedded in computers these days. But statistics is where it still takes a human interaction and you can apply it very well.”
IT IS FUNDAMENTALLY IMPORTANT THAT WE’RE CHANGING PEOPLE’S HEARTS AND MINDS WHEN IT COMES TO USING DATA AND HAVING THE SKILLS READY TO USE DATA.
That’s not to say that there aren’t certain threats to the world of data science. There have been societal and political barriers that have gotten in the way of big data reaching its maximum potential, and education has a lot to do with it.
"When I first got into the field, I was out of MIT, I got this PhD in Parallel Processing and my first project was for American Express, putting up their customer data on a super-computer from Thinking Machines," Barth told me. "I came back to my colleagues after 6 weeks of working on the project and I said, ‘You guys will never guess what the real issue is here...Two things: It’s politics and dirty data.’ They were just astounded, and I will say that it’s very clear to me, as with many things, that the importance of a strong STEM education cannot be undervalued."
Barth was sure to emphasize the importance of Liberal Arts, as well, seeing as not one of his 3 sons pursued engineering. In general, he said he strongly believes in a rich education and that we should be mindful of what it means to our younger generations. With the recent question of whether Boston Public Schools will be cutting its budget by tens of millions of dollars - placing STEM on the chopping block at some schools - Barth’s sentiments are especially relevant. According to him:
In terms of jobs, these kinds of knowledge-worker jobs will be high-valued and we should be investing in them. Getting a society and getting a political system to make the investment for the long-term by spending dollars today is the political decision, just like the political decisions we’ve faced in the past...It is fundamentally important that we’re changing people’s hearts and minds when it comes to using data and having the skills ready to use data. I think it can bleed out into how we invest our public dollars for building out the right educational curriculum.