Loading…
This event has ended. Visit the official site or create your own event on Sched.
Click here to return to main conference site. For a one page, printable overview of the schedule, see this.
View analytic
Wednesday, June 29 • 11:42am - 12:00pm
Interactive Terabytes with pbdR

Log in to save this to your schedule and see who's attending!

Historically, large scale computing and interactivity have been at odds. A new series of packages have recently been developed to attempt to rectify this problem. We do so by combining two programming models: client/server (CS) and single program multiple data (SPMD). The client/server allows the R programmer to control from one to thousands of batch servers running as cooperating remote instances of R. This can easily be done from a local R or RStudio session. The communication is handled by the well-known ZeroMQ library, with a new set of package bindings available to R by way of the pbdZMQ package. The client and server are implemented in the new remoter and pbdCS packages. To handle computations, we use the established pbdR packages for large scale distributed computing. These packaegs utilize HPC standards like MPI and ScaLAPACK to handle complex coupled computations on truly large data. These tools use the batch SPMD programming model, and constitute the server portion of the client/server hierarchy. So once the client issues a command, it is transmitted to the SPMD servers and executed in a massively parallel fashion.This talk will discuss the package components and provide timing results for some Terabyte size computations running on hundreds of cores of a cluster.

Moderators
avatar for Dirk  Eddelbuettel

Dirk Eddelbuettel

Debian and R Projects

Speakers
WC

Wei-Chen Chen

pbdR Core Team


Wednesday June 29, 2016 11:42am - 12:00pm
Econ 140 579 Serra Mall, Stanford, CA 94305

Attendees (37)