At Google I/O we announced the Mapper API. Built completely on top of public App Engine APIs today, this API is only the first component of App Engine’s MapReduce toolkit, but can be extremely useful on its own.
The Mapper API can already be of use to many developers who would otherwise need to build their own tool for doing large scale data manipulation. In addition to taking care of the distribution of these jobs over task queues, it provides the ability to store state, batch datastore writes via mutation pools, and ships with an easy to use administrative interface for job management, all optimized for the constraints of App Engine’s dynamic serving environment. Some examples of the types of operations that work with minimal configuration with this tool:
When you’re ready to jump in and start using the tool, head over to the project homepage on Google Code. You’ll want to check out the “Getting Started” page for the language you’re using:
- Fred, Mike, Ikai, Nick + the App Engine team