This tutorial explains the creation of a map/reduce application based on Lokad.Cloud; the application computes the histogram of an image using several Azure workers and handles the data in blob storage and queues.
This tutorial explains the creation of a map/reduce application based on Lokad.Cloud; the application computes the histogram of an image using several Azure workers and handles the data in blob storage and queues.
Microsoft Research recently announced the availability, under Academic Licensing, of Dryad, an infrastructure which allows a programmer to use the resources of a computer cluster or a data center for running data-parallel programs. A Dryad programmer can use thousands of machines, each of them with multiple processors or cores, without knowing anything about concurrent programming.
Â
DryadLINQ is the managed high level programming abstraction used to compose Dryad vertex topology graphs that the Dryad infrastructure uses to partition parallel computations. Here, Erik Meijer and Dryad team member Roger Barga discuss Drayad and DryadLINQ at a high level so that most of us can understand the implications, history and future of Dryad. This is an introductory piece. Erik and I will dive deep into Dryad with one of the scientists behind it in the second part of this Expert to Expert mini series on Dryad. UPDATE: The Going Deep episode on Dryad is now live.
Enjoy! This is incredible and important technology for simplifying the inherent complexity of distributed computation in the cloud. In essence, DryadLINQ enables a sequential programming experience over what will execute across potentially thousands of machines (depending upon the computational complexity of the program) concurrently.  Much to learn here. Channel 9 will help teach.
Microsoft Research recently announced the availability, under Academic Licensing, of Dryad, an infrastructure which allows a programmer to use the resources of a computer cluster or a data center for running data-parallel programs.
A Dryad programmer can use thousands of machines, each of them with multiple processors or cores, without knowing anything about concurrent programming.
That’s a pretty heady statement. What does Dryad do, exactly, to enable this level of abstraction, shielding programmers from the incredibly complex world of distributed parallel computing? Does the level of abstraction impact the degree to which sophisticated programmers can interact with and control some of the low level mechanisms of the Dryad runtime? What is it about LINQ that made it the no-brainer managed programming abstraction for Dryad?
Simply, how does Dryad work? This is the core question that Erik and I had after our conversation with Roger Barga (part one of this E2E mini-series on Dryad and DryadLINQ – perhaps we should focus just on DryadLINQ next time, but for now, all the information in this conversation is certain to keep you very busy and answer many questions you may have after learning about Dryad in part one…).Â
Lots of whiteboarding here. Put on your thinking caps!
Enjoy.