Introduction to High-Performance Computing with R - Tutorial ...

Tools Measure Faster Compile ImplP ExplP OoMem
Introduction to
High-Performance Computing with R
Tutorial at useR! 2010
Dirk Eddelbuettel, Ph.D.
Dirk.Eddelbuettel@R-Project.org
edd@debian.org
useR! 2010
National Institute of Standards and Technology (NIST)
Gaithersburg, Maryland, USA
Dirk Eddelbuettel Intro to High-Perf. Computing with R Tutorial @useR!2010Tools Measure Faster Compile ImplP ExplP OoMem
Outline
1 Motivation
6 Implicitly Parallel
2 Automation and scripting
7 Explicitly Parallel
3 Measuring and profiling
8 Out-of-memory processing
4 Speeding up
9 Summary
5 Compiled Code
Dirk Eddelbuettel Intro to High-Perf. Computing with R Tutorial @useR!2010Tools Measure Faster Compile ImplP ExplP OoMem
Motivation: What describes our current situation?
Moore’s Law: Processors
keep getting faster and
faster
Yet our datasets get
bigger and bigger and an
even faster rate.
So we’re still waiting and
waiting . . .
Result: An urgent need
for high(er) performance
computing with R.
Source: http://en.wikipedia.org/wiki/Moore’s_law
Dirk Eddelbuettel Intro to High-Perf. Computing with R Tutorial @useR!2010Tools Measure Faster Compile ImplP ExplP OoMem
Motivation: Data sets keep growing
There are a number of reasons behind ’big data’:
more collection: from faster DNA sequencing to larger
experiments to per-item RFID scanning to complex social
networks — our ability to originate data keeps increasing
more networking: (internet) capacity, transmission speeds
and usage keep growing ...