With the help of Chris DiBona and Barry Brumitt (speaker) I am pleased to announce that Google is also sponsoring Ignite tonight. As I’ve mentioned on Radar, I use Google for search, mail, and docs (formerly Writely). Using those products certainly made producing Ignite a lot simpler. Thanks Google!
Here’s some more on what Barry Brumitt will be discussing:
Processing and transforming large data sets can be cumbersome and slowon a single machine, while using multiple machines can require significant custom infrastructure to see the advantages of parallelism. Google has many such data sets which are used to build the indexes that provide rapid responses to the very high query rate observed on its web sites. Google engineers nigh-universally use The “Map-Reduce” framework to process large (Gb, Tb, Pb?) datasets across thousands of machines simultaneously. A 20-line program is all that is necessary to perform a simple transformation across a very large cluster. In my work over the last year, I’ve been using Map-Reduce to process large geographic data sets which describe the earth. At IgniteSeattle, I’m going to introduce MapReduce and describe a couple ways it can be effectively used when working with typical geographic data sets.