Differences
This shows you the differences between two versions of the page.
| Both sides previous revision
Previous revision
|
Next revision
Both sides next revision
|
moving_data [2021/03/10 20:04]
pwolinsk |
moving_data [2021/03/10 20:04]
pwolinsk |
| GLOBUS [[https://globus.org/]] is a service for secure, reliable research data management. It allows users to move, share, & discover data via a single interface using a web browser. GLOBUS is designed to move large data sets. | GLOBUS [[https://globus.org/]] is a service for secure, reliable research data management. It allows users to move, share, & discover data via a single interface using a web browser. GLOBUS is designed to move large data sets. |
| |
| scp, sftp, rsync have the advantage of being very simple and do not require any initial setup. However large data sets often fail to transfer correctly using these protocols. GLOBUS does require some initial setup but is much more reliable. It has many features, such as splitting the transfer into multiple simultaneous streams, encrypting the data in flight, automatically retransmitting data on network failure/timeouts, verifying data integrity after transfer. In addition our installation of GLOBUS is connected to the **100Gb/s** network (where our cluster login nodes used for scp/sftp/rsync are on the 10Gb/s network). | scp, sftp, rsync have the advantage of being very simple and do not require any initial setup. However large data sets often fail to transfer correctly using these protocols. GLOBUS does require some initial setup but is much more reliable. It has many features, such as splitting the transfer into multiple simultaneous streams, encrypting the data in flight, automatically retransmitting data on network failure/timeouts, verifying data integrity after transfer. In addition our installation of GLOBUS is connected to the **100Gb/s** network (while our cluster login nodes used for scp/sftp/rsync are on the 10Gb/s network). |
| |
| GLOBUS service moves data between GLOBUS Endpoints. Each Endpoint is a server process running on a machine which can send and receive data. One such endpoint, named **UARK-Pinnacle**, is set up on the Pinnacle cluster. It is a //public// endpoint (visible to all GLOBUS users), and accessible by anyone with an account on Pinnacle. To transfer data between your account on Pinnacle and your personal workstation/laptop you will need to set up a //private// GLOBUS endpoint, which is only visible and accessible by you. This requires the installation of GLOBUS personal connect server on your workstation/laptop. | GLOBUS service moves data between GLOBUS Endpoints. Each Endpoint is a server process running on a machine which can send and receive data. One such endpoint, named **UARK-Pinnacle**, is set up on the Pinnacle cluster. It is a //public// endpoint (visible to all GLOBUS users), and accessible by anyone with an account on Pinnacle. To transfer data between your account on Pinnacle and your personal workstation/laptop you will need to set up a //private// GLOBUS endpoint, which is only visible and accessible by you. This requires the installation of GLOBUS personal connect server on your workstation/laptop. |
