User Tools

Site Tools


This is an old revision of the document!

Data Transfer to and from AHPCC Clusters

A special data mover node is available, called from campus and from the world. It should be used for moving data to and from the clusters. tgv/dm is configured with a 10Gb/s network connection and a dedicated 21TB storage system mounted at /local_storage. Regular login shells are blocked. The allowed protocols are

  • scp (secure copy)
  • sftp (secure ftp)
  • rsync

To upload a data file from the current directory on your local desktop machine to your /storage directory on razor:

pawel@localdesktop$# scp localfile.dat

To download a data file from your /storage directory on razor to the current directory on your local desktop machine:

pawel@localdesktop$# scp .

You will also have a staging directory on tgv/dm called /local_storage/$USER/. A new Globus Online instance on tgv/dm is in preparation, and login shells are available for special situations such as batch wget from an http server.

Data Transfer between Razor & Trestles Clusters

A special node named bridge is set aside for the purpose of moving data between storage systems of the Razor and Trestles clusters. The bridge node has home, storage and scratch file systems from both Razor and Trestles nodes mounted under these directories:

Trestles file systems

These are also mounted at:

When you login to bridge, you will be located in your Trestles home.

Razor file systems


Although a direct secure copy to bridge node from either Razor or Trestles cluster nodes or front ends is available, to achieve best performance we recommend logging into bridge node directly and using the cp command to move files between the cluster file systems:

tres-l1:pwolinsk:$ ssh bridge
Last login: Fri Feb 19 14:03:31 2016 from tres-l1
No Modulefiles Currently Loaded.
bridge:pwolinsk:$ cp /trestles/home/pwolinsk/memusage /razor/home/pwolinsk/
bridge:pwolinsk:$ exit
Connection to bridge closed.

cp directly on bridge is faster because it uses 40Gb/s Infiniband while scp uses 1Gb/s ethernet.

moving_data.1457556627.txt.gz · Last modified: 2016/03/09 20:50 by root