Data Transfer

RCC provides a number of methods for accessing and transferring data in/out of our systems. We recommend the scp protocol for transferring files to/from RCC systems. RCC hosts a managed Globus Online endpoint that can be used for moving very large amounts of data. When connecting from the UChicago network or through the UChicago VPN it is also possible to connect to the RCC file systems using SAMBA.

SCP

Secure copy or SCP is a means of securely transferring computer files between a local host and a remote host. It is based on the Secure Shell (SSH) and Secure File Transfer (SFTP) protocols.

Command-Line Operation

Most UNIX-like operating systems (Mac OS X, Linux, etc) provide a scp command which can be accessed from the command line. To transfer files from your local computer to your home directory on Midway1, open a terminal window and issue the command:

Single files: $ scp <some file> <CNetID>@midway.rcc.uchicago.edu:
Directories:  $ scp -r <some dir> <CNetID>@midway.rcc.uchicago.edu:

    Or, to connect to a directory on Midway2 (/project2, for example) :
    $ scp -r <some dir> <CNetID>@midway2.rcc.uchicago.edu:/project2

When prompted, enter your CNet password.

WinSCP GUI for Windows Clients

WinSCP is a scp client software that can be used to move files to and from Midway and a Windows machine. WinSCP can be obtained from http://www.winscp.net.

When setting up your connection to Midway1 in WinSCP, use the following information:

Hostname: midway.rcc.uchicago.edu
Port:     22
Username: CNetID
Password: CNet password

If you are connecting to Midway2 in WinSCP, use the following information:

Hostname: midway2.rcc.uchicago.edu
Port:     22
Username: CNetID
Password: CNet password

After connecting, if you are prompted to accept the server’s host key, select “yes.”

Upon successfully connecting, the main WinSCP window allows you to move files from your local machine (left side) to Midway (right side).

SAMBA

SAMBA allows uses to connect to (or “mount”) their home and project directories on their local computer so that the file system on Midway appears as if it were directly connected to the local machine. This method of accessing your RCC home and project space is only available from within the UChicago campus network. From off-campus you will need to first connect through the UChicago VPN.

Your SAMBA account credentials are your CNetID and password:

Username: ADLOCAL\CNetID
Password: CNet password
Hostname: midwaysmb.rcc.uchicago.edu

Note

Make sure to prefix your username with ADLOCAL\

Connecting from Windows

On a Windows computer, use the “Map Network Drive” option:

  • Enter one of the following UNC paths depending on which location on Midway you wish to connect to:

    Home:     \\midwaysmb.rcc.uchicago.edu\homes
Project:  \\midwaysmb.rcc.uchicago.edu\project
Project2: \\midwaysmb.rcc.uchicago.edu\project2
Scratch:  \\midwaysmb.rcc.uchicago.edu\midway-scratch

  • When prompted for a username and password, select Registered User.

  • Enter ADLOCAL\CNetID for the username and enter your CNet password.

Note

Make sure to prefix your username with ADLOCAL\

Connecting from Mac OS X

To connect on a Mac OS X computer follow these steps:

  • Open the Connect to Server utility in Finder:
../_images/finder-connect_to_server.jpg

  • Enter one of the following URLs in the input box for Server Address depending on which location on Midway you wish to connect to:

    Home:     smb://midwaysmb.rcc.uchicago.edu/homes
Project:  smb://midwaysmb.rcc.uchicago.edu/project
Project2: smb://midwaysmb.rcc.uchicago.edu/project2
Scratch:  smb://midwaysmb.rcc.uchicago.edu/midway-scratch

  • When prompted for a username and password, select Registered User.

  • Enter ADLOCAL\CNetID for the username and enter your CNet password.

Note

Make sure to prefix your username with ADLOCAL\

Connecting from Ubuntu

From Ubuntu, follow the steps here: https://wiki.ubuntu.com/MountWindowsSharesPermanently. Specifically,

  • Install cifs sudo apt-get install cifs-utils.

  • Create a folder to contain the mounted filesystem, e.g. mkdir /media/midway

  • Put your RCC credentials in a text file. Create a file .smbcredentials in your home directory containing:

    username=[midway username]
password=[midway password]

  • Change permissions so only you can read the file chmod 600 ~/.smbcredentials.

  • Edit the file /etc/fstab as root and add the following on one line at the end:

    //midwaysmb.rcc.uchicago.edu/project /media/midway cifs credentials=/home/[username]/.smbcredentials,domain=ADLOCAL,iocharset=utf8,sec=ntlm 0 0

    You can change “project” in the above line to “homes”, “project2” or “midway-scratch” depending on which location you wish to mount.

  • Remount everything in /etc/fstab with the command sudo mount -a.

Your Midway folders should now be accessible at /media/midway.

HTTP (web access)

RCC provides web access to data on their storage system via public_html directories in both users’ home and project directories.

Be sure your home or project directories and public_html have the execute bit set, and that public_html has read permissions if you would like to allow indexing (directory listings and automatic selection of index.html).

chmod o+x $HOME
mkdir -p $HOME/public_html
chmod o+rx $HOME/public_html

Files in public_html must also be readable by the web user (other), but should not be executable.

Note

Use of these directories must conform with the RCC usage policy (https://rcc.uchicago.edu/about-rcc/rcc-user-policy). Please notify RCC if you expect a large number of people to access data hosted here.

Globus Online

Globus Online is a robust tool for transferring large data files to/from Midway. The RCC has a customized Globus Online login site at https://globus.rcc.uchicago.edu and uses Single Sign On capabilities of CILogon. If you have already signed up, here is the connection information:

URL:       https://globus.rcc.uchicago.edu
End Point: ucrcc#midway

Follow these instructions to get started:

../_images/globus-login1.png
  • Select “University of Chicago” for the Identity Provider
../_images/globus-login2.png
  • Enter your CNetID and password when prompted
../_images/globus-login4.png
  • You will need to link your University of Chicago credentials to a Globus Online account. Either create a new Globus Online account or sign in to your existing account if you have one.
../_images/globus-login6.png
  • Once you are signed in, enter ucrcc#midway as the Endpoint and hit the Go button
../_images/globus-login5.png
  • If you want to transfer files from your local computer, click the Get Globus Connect link on the transfer page and follow the instructions.
../_images/globus-login7.png

There is extensive documentation on the Globus Online site as to how to transfer files in different modes. Please refer to their documentation for more details or contact us with any RCC specific issues.