User Tools

Site Tools


spark

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Last revision Both sides next revision
spark [2016/05/12 21:30]
pwolinsk
spark [2016/05/13 14:24]
pwolinsk
Line 1: Line 1:
 ==== Spark ==== ==== Spark ====
-Apache Spark version 1.6.1 is installed in /​share/​apps/​spark. ​+Apache Spark version 1.6.1 is installed in /​share/​apps/​spark. Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It allows users to combine the memory and cpus of multiple compute nodes into into a Spark cluster and use the aggregated cluster memory and cpus to run a single task.
  
 The example PBS script below sets up a 3 node spark cluster in standalone mode using 3 compute nodes on Trestles.  ​ The example PBS script below sets up a 3 node spark cluster in standalone mode using 3 compute nodes on Trestles.  ​
spark.txt · Last modified: 2016/06/02 16:21 by pwolinsk