Live from Hadoop World in New York City

[admin]

This week is the third edition of Hadoop World, and the event has been growing quite a bit from its first edition. From a couple hundred enthusiasts – to a 1400 strong crowd, that only got limited by the fire code restrictions of the meeting rooms. We don’t see a lot of tech conferences that are actually marked as “Sold Out” and refusing walk-in registrations!

I have been sitting in the keynotes and in several breakout sessions, and also got the opportunity to chat with speakers and attendees. The overall impression is overwhelmingly positive: energy is great, content is valuable, Hadoop is the technology.

The last time I saw this kind of energy was in the Web technology conferences of the late 90’s. People then were going crazy over HTML, J2EE, Web application servers… Of course, one could argue that this energy was more hype than anything and led us straight into a bubble – true, but after the bubble had burst, the market consolidated, these technologies were deeply entrenched. *How much of Big Data is a hype that will burst remains to be determined, but one thing is certain: it will have a long lasting impact on IT and the way the business utilizes data.

The twittosphere is abuzz with sound bites from the sessions so I won’t be getting here into the amazing stats we are hearing all day long – just follow the #hw2011 hashtag to get your “Big Data fix”. There are a couple other points I’d like to bring up.

• One third of attendees are non-technical folks. That’s a lot, for a tech conference on Hadoop. And a proof that a lot of Big Data initiatives are business-led. Since IT is badly lacking resources to build all these complicated Map Reduce jobs (cf. the “we are all hiring” stuff below), they need tools to help them. This is why the ecosystem around Hadoop is growing so quickly, and one key capability is the ability to generate optimized Big Data Integration and Big Data Quality jobs, as Talend is delivering with its Big Data solution.
• “We are hiring” is really extreme. *When was the last time you heard a keynote speaker starting his presentation by saying “we are hiring, and we are paying more than the previous speaker”?* It happened in ’99, when everyone who understood 3GL development was crash trained as a Java developer – and everyone else as a Web QA engineer.
• We are witnessing the power – and openness – of open source at work here. Hadoop World is organized by Cloudera, arguably the first to take Hadoop to the enterprise. But Cloudera’s competitors are here, and involved in the program. Including non-Hadoop-based Big Data solutions. I agree with Cloudera that this is the right way to take Hadoop to the next level, but it’s still a gutsy move on their side. Now, you don’t get into the commercial open source business if you are not willing to make a few bets along the way.

Another day is starting in New York, with great content on the agenda. Can’t wait!

Yves




More from the Talend blog ...