2022 Linux fundamentals

This page should serve as a reference for the many "things Linux" we use in this course. It is by no means complete – Linux is **huge** – but offers introductions to many important topics.

You need a Terminal program in order to ssh to a remote computer.

• Macs and Linux have a Terminal program built-in

• Windows options:

  • Windows 10+

    • Command Prompt program and PowerShell have ssh and scp (may require latest Windows updates)

      • Start menu → Search for Command

    • Windows Subsystem for Linux – Windows 10 Professional includes a Ubuntu-like bash shells

      • See https://docs.microsoft.com/en-us/windows/wsl/install-win10

      • We recommend the Ubuntu Linux distribution, but any Linux distribution will have an SSH client

  • or

    • Putty – http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

      • simple Terminal and file copy programs

      • download either the Putty installer (https://the.earth.li/~sgtatham/putty/latest/w64/putty-64bit-0.77-installer.msi)

        • or just putty.exe (terminal) and pscp.exe (secure copy client)

    • Cygwin – http://www.cygwin.com/

      • A full Linux environment, including X-windows for running GUI programs remotely

      • Complicated to install

Type as little and as accurately as possible by using keyboard shortcuts!

The Tab key is your best friend! Hit the Tab key once or twice - it's almost always magic! Hitting Tab invokes shell completion, instructing the shell to try to guess what you're doing and finish the typing for you. On most modern Linux shells, Tab completion will:

• single Tab – complete file or directory names up to any ambiguous part

  • if nothing shows up, there is no unambiguous match

• Tab twice – display all possible completions

  • you then decide where to go next

• work for shell commands too (like ls or cp)

• Use Up arrow to retrieve any of the last 500 commands you've typed, going backwards through your history.

  • You can then edit them and hit Enter (even in the middle of the command) and the shell will use that command.

• The Down arrow "scrolls" forward from where you are in the command history.

• Right arrow and Left arrow move the cursor forward or backward on the current command line.

• Use Ctrl-a (holding down the "control" key and "a") to jump the cursor to the beginning of the line.

• Use Ctrl-e to jump the cursor to the end of the line.

• Arrow keys are also modified by Ctrl

  • e.g. Ctrl-right-arrow will skip by word forward; Ctrl-left-arrow backward.

• Use Backspace to remove text before the cursor; Delete to remove text after the cursor.

The shell has shorthand to refer to groups of files by allowing wildcards in file names.

* (asterisk) is the most common filename wildcard. It matches "any length of any characters".

This technique is sometimes called filename globbing, and the pattern a glob.

Other useful ones are

• brackets ( [ ] ) to allow for any character in the list of characters between the brackets.

  • and you can use a hyphen ( - ) to specify a range of characters (e.g. [A-G])

• braces ( {  } ) enclose a list of comma-separated strings to match (e.g. {dog,pony})

For example:

• ls *.bam – lists all files in the current directory that end in .bam

• ls [A-Z]*.bam – does the same, but only if the first character of the file is a capital letter

• ls [ABcd]*.bam – lists all .bam files whose 1st letter is A, B, c or d.

• ls *.{fastq,fq}.gz – lists all .fastq.gz and .fq.gz files.

Three special file names:

1. . (single period) means "this directory".

2. .. (two periods) means "directory above current." So ls .. means "list contents of the parent directory."

3. ~ (tilde) means "my home directory".

Every command and Linux program has three "built-in" streams: standard input, standard output and standard error.

Open

It is easy to not notice the difference between standard output and standard error when you're in an interactive Terminal session – because both outputs are sent to the Terminal. But they are separate streams, with different meanings. When running batch programs and scripts you will want to manipulate standard output and standard error from programs appropriately.

• To take the standard output of a program and save it to a file, you use the > operator

  • a single > overwrites any existing target; a double >> appends to it

  • since standard output is stream #1, this is the same as 1>

• To redirect the standard error of a program you must specify its stream number using 2>

• To redirect standard output and standard error to the same place, use the syntax 2>&1