User Tools

Site Tools


spark

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
spark [2016/05/13 14:24]
pwolinsk
spark [2016/06/02 16:21]
pwolinsk
Line 39: Line 39:
 When the job starts all 3 nodes are running the worker service. ​ The job head node is also running the spark master service. ​ The log file from the spark master is saved to: When the job starts all 3 nodes are running the worker service. ​ The job head node is also running the spark master service. ​ The log file from the spark master is saved to:
  
-**''/​scratch/​$USER/​spark-$USER-org.apache.spark.deploy.master.Master-1-$HOST.out''​**+**''/​home/​$USER/​spark-$USER-org.apache.spark.deploy.master.Master-1-$HOST.out''​**
  
 and worker logs are in: and worker logs are in:
  
-**''/​scratch/​$USER/​spark-$USER-org.apache.spark.deploy.worker.Worker-<​workernum>​-$HOST.out''​**+**''/​home/​$USER/​spark-$USER-org.apache.spark.deploy.worker.Worker-<​workernum>​-$HOST.out''​**
  
 In addition to the log files a new file in the $HOME/​spark-info contains the URL of the spark master web interface: In addition to the log files a new file in the $HOME/​spark-info contains the URL of the spark master web interface:
spark.txt · Last modified: 2016/06/02 16:21 by pwolinsk