/
Copying files from TACC to your personal computer.

Copying files from TACC to your personal computer.

Jun 17, 2023

Overview:

Stampede2 is tremendously powerful and capable of doing many things, but as most of you are probably being slightly frustrated by, it doesn't have much in the way of a GUI (graphical user interface). The lack of a GUI means it can't visualize graphs or other meaningful representations of our data. In order to do these types of things, we have to get our data off of stampede2 and onto our own computers.

You recall one of the first things you did in this class was use the cp command to copy files from the the BioITeam directories to your home directory. Unfortunately sampede2 can't access the directories on your personal computer so the cp command won't work. Instead we have to use the "secure copy command" (scp). Copying using  scp works using the same 3 parts as the cp command:

  1. the command itself (now scp instead of cp)

  2. The file you want to securely copy 

  3. The location you want the new file to go to

The difference is the scp command includes more detailed information on the path of where the file is, or where the file is going using syntax similar to that used in the other "secure" command you are familiar with: ssh.

Objectives:

  1. Transfer a file from stampede2 to your local computer.

  2. Transfer a file from your local computer to stampede2.

Tutorial:

  • Setup

It is often useful to set up 2 different terminal windows to scp between a remote computer and your local computer. This is not required by any means, but is often helpful. The image below shows 2 separate windows where I have transferred a file named README from a folder inside the BioITeam materials to my local computer. Notice the one on the left corresponds to my local computer while the other is actually inside of the BioITeam folder on TACC. 

 

  • Transfer from stampede2 to your computer

Before you begin, be sure you have 2 terminal windows open as shown in the "setup" section above. For simplicity the window corresponding to your Local computer will be referred to as "Left" while the window where you have logged into the Remote computer will be referred to as "Right" as is depicted in the image above.

In the Right window (Remote):

  1. Navigate to the directory that contains the file you want to transfer.

    cd /corral-repl/utexas/BioITeam/ngs_course

  2. Use the pwd command to print the current working directory

    pwd

  3. Copy the address by highlighting the text and hitting control/command + C

In the Left window (Local):

  1. Navigate to the directory that you want to copy the file to.

    cd Downloads

  2. The scp command will have the following parts: scp <USERNAME>@<Remote_computer_address>:<pathway_to_file_you_want_to_transfer_starting_with_a_/_mark_and_ending_with_the_filename> <location_you_want_the_file_copied_to>

    use the scp command to transfer the remote file to to your local computer

    scp <USERNAME>@stampede2.tacc.utexas.edu:/corral-repl/utexas/BioITeam/ngs_course/README .

    *note the required punctuation:

    1. @ symbol following your username (same as is done in the ssh command)

    2. : following the remote computer address. Missing this mark is the 2nd most common mistake people make

    3. / immediately after the : mark. Missing this mark is the most common mistake people make.

  3. Upon hitting enter you should be prompted for your tacc password, and 1 time code.

  4. You can then navigate to that folder using your operating system to see the new file and choose what to open it with (in this case it is a text file)

  • Transfer from your computer to stampede2

Before you begin, be sure you have 2 terminal windows open as shown in the image in "setup" section above. For simplicity the window corresponding to your Local computer will be referred to as "Left" while the window where you have logged into the Remote computer will be referred to as "Right" as is depicted in the image above.

In the Right window (Remote):

  1. Navigate to the directory that you want to transfer the file to..

    cd $SCRATCH

  2. Use the pwd command to print the current working directory

    pwd

  3. Copy the address by highlighting the text and hitting control/command + C

In the Left window (Local):

  1. Navigate to the directory that has the file you want to copy.

    cd Downloads

  2. The scp command will have the following parts: scp <filename> <USERNAME>@<Remote_computer_address>:<location_you_want_the_file_copied_to_starting_with_a_/_mark>

    use the scp command to transfer the remote file to to your local computer

    scp README <USERNAME>@stampede2.tacc.utexas.edu:<pwd_you_copied_in_the_right_window>

    *note the required punctuation:

    1. @ symbol following your username (same as is done in the ssh command)

    2. : following the remote computer address. Missing this mark is the 2nd most common mistake people make

    3. / immediately after the : mark. Missing this mark is the most common mistake people make.

  3. Upon hitting enter you should be prompted for your tacc password, and 1 time code.

  4. Switch back to the other window and use the ls command to see the file.

Alternatives:

  1. Students sometimes ask if you can use scp in the window logged into TACC and transfer a file to their local computer. The answer is yes you can, but it is more complicated. For those wanting to figure this out for themselves, this question has been asked and discussed on stackexchange and should work for you. This is something your instructor has never done personally but is willing to answer questions about, or provide additional resources to help you do this for yourself if you think it important.

  2. For those using putty, I remind you that using the power shell with openSSH installed from windows10 will allow you to use the scp command listed above. For those wanting help using power shell see this tutorial.

  3. Another alternative for putty users is WinSCP. This link will take you to the installation instructions.

    1. Once installed, recommend using the 'commander' interface until you are more familiar with things. 

    2. .

 

 

Since this page is linked to from many different tutorials, you want to use the back arrow on your browser to return to the page you were just on.