User Tools

Site Tools


Data Transfer to and from AHPCC Clusters

A dedicated external data mover node is available, called from campus and from the world. It should be used for moving data to and from the clusters and the Razor parallel file systems. tgv/dm is configured with a 10Gb/s network connection and a dedicated 21TB storage system mounted at /local_storage. Regular login shells are blocked. The allowed protocols are

  • scp (secure copy)
  • sftp (secure ftp)
  • rsync

Linux and MacOS

To upload a data file from the current directory on your local desktop machine to your /storage directory on razor:

pawel@localdesktop$# scp localfile.dat

To download a data file from your /storage directory on razor to the current directory on your local desktop machine:

pawel@localdesktop$# scp .

You will also have a staging directory on tgv/dm called /local_storage/$USER/. A new Globus Online instance on tgv/dm is in preparation, and login shells are available for special situations such as batch wget from an http server. Please contact if you need that.


Windows OS does not include Secure Copy or Secure FTP tools. There are multiple file transfer clients available for download. The most popular command line client is pscp.exe available here:

To transfer files using pscp.exe download and save pscp.exe to your Windows machine. Then open a terminal (i.e. Command Prompt, under “Start→All Programs→Accessories→Command Prompt”) and specify the full path to the location of the downloaded pscp.exe file followed by two arguments <source> and <destination>. Either one or both could specify a file a on remote machine (user@host:path_to_file) or a local file (path_to_file). For example:

C:\Users\Pawel> c:\Users\Pawel\Downloads\pscp.exe filetoupload.txt

The code above uses secure copy protocol to upload a file “filetoupload.txt” to the home directory of user pwolinsk on

Another popular windows transfer client (GUI) is WinSCP:

  <ul> <table border=0><tr><td bgcolor=#aaaaaa><b>NOTE:</b> The data mover node <b><tt></tt></b> can also be accessed as <b><tt></tt></b> from outside of the UofA network. This domain name is assigned an IP address in the network DMZ (demilitarized zone) on a 10Gb/s ethernet network and is the preferred domain name to use for UofA external transfers.</td></tr></table>

Data Transfer between Razor & Trestles Clusters

A dedicated internal node named bridge is set aside for the purpose of moving data between storage systems of the Razor and Trestles clusters. The bridge node has home, storage and scratch file systems from both Razor and Trestles nodes mounted under these directories:

Trestles file systems

These are also mounted at:
There are also multiple privately owned storage areas /storage[x].

When you login to bridge, you will be located in your Trestles home. Also please recall that the Trestles persistent /scratch/$USER partition is being phased out and will in the future be only /scratch/$PBS_JOBID for each batch job.

Razor file systems


Although scp to the bridge node is possible from either cluster, we recommend logging into bridge node directly and using the cp or mv commands to move files, thus using 40Gb/s Infiniband instead of scp using 1Gb/s ethernet:

tres-l1:pwolinsk:$ ssh bridge
Last login: Fri Feb 19 14:03:31 2016 from tres-l1
No Modulefiles Currently Loaded.
bridge:pwolinsk:$ cp /trestles/home/pwolinsk/memusage /razor/home/pwolinsk/
bridge:pwolinsk:$ exit
Connection to bridge closed.

rsync is better for copying whole directories:

bridge:pwolinsk:$ cd /razor/home/pwolinsk
bridge:pwolinsk:/razor$ rsync -av XSEDE /home/pwolinsk/
sending incremental file list
...omitted every file listed with -v option...

sent 185047 bytes  received 1258 bytes  124203.33 bytes/sec
total size is 180459  speedup is 0.97

On bridge, /home/username and /trestles/home/username are equivalent.

bridge is not reachable from outside the clusters, please use tgv/dm. A data transfer node to the Trestles file systems is planned.

moving_data.txt · Last modified: 2020/09/21 19:52 by root