Linux fundamentals

Linux fundamentals

This page should serve as a reference for the many "things Linux" we use in this course. It is by no means complete – Linux is **huge** – but offers introductions to many important topics.

See also this page, which provides lists of the most common Linux commands, by category, as well as their most useful options: Some Linux commands

Some Linux commands

  • This page provides lists of the most common Linux commands, by category, as well as their most useful options:

Terminal programs, shells and commands

You need a Terminal program in order to ssh to a remote computer.

  • Macs and Linux have a Terminal program built-in

  • Windows options:

Use ssh (secure shell) to login to a remote computers.

SSH to a remote computer
# General form: ssh <user_name>@<full_host_name> # For example ssh abattenh@ls6.tacc.utexas.edu

The bash shell REPL and commands

When you type something in at a bash command-line prompt, it Reads the input, Evaluates it, then Prints the results, then does this over and over in a Loop. This behavior is called a REPL – a Read, Eval, Print Loop. The shell executes the command line input when it sees a linefeed, which happens when you press Enter after entering the command.

The input to the bash REPL is a command, which consists of:

  • The command name (any of the built-in Linux/Unix commands, or the name of a user-written script or program)

  • One or more (optional) options, usually noted with a leading dash (-) or double-dash (--).

    • short (1-character) options can be provided separately, prefixed by a single dash (-)

      • or can be combined with the combination prefixed by a single dash

    • long (multi-character or "word") options are prefixed with a double dash (--) and must be supplied separately.

    • Both long and short options can be assigned a value

  • One or more command-line arguments, which are often (but not always) file names

Some examples using the ls (list files) command:

ls # example 1 - no options or arguments ls -l # example 2 - one "short" (single character) option only (-l) ls --help # example 3 - one "long" (word) option (--help) ls .profile # example 4 - one argument, a file name (.profile) ls --width=20 # example 5 - a long option that has a value (--width is the option, 20 is the value) ls -w 20 # example 6 - a short option w/a value, as above, where -w is the same as --width ls -l -a -h # example 7 - three short options entered separately (-l -a -h) ls -lah # example 8 - three short options that can be combined after a dash (-lah)
  • The arguments to ls are one or more file or directory names. If no arguments are provided, the contents of the current directory are listed.

    • If an argument is a directory name, the contents of that directory are listed.

  • Some handy options for ls:

    •  -l shows a long listing, including file permissions, ownership, size and last modified date.

    • -a shows all files, including dot files whose names start with a period ( . ) which are normally not listed

    • -h says to show file sizes in human readable form (e.g. 12M instead of 12201749)

A good place to start learning built-in Linux commands and their options is on the Some Linux commands page.

Getting help

How do you find out what options and arguments a command uses?

  1. In the Terminal, type in the command name then the --help long option (e.g. ls --help)

    • Works for most Linux commands; 3rd party tools may use -h or -? or even /? instead

    • May produce a lot of output, so you may need to scroll up quite a bit, or pipe the output to a pager

      • e.g. ls --help | more (a space advances the output by one screen/"page", and typing Ctrl-C will exit more)

  2. Use the built-in manual system (e.g. type man ls)

    • This system uses the less pager

    • For now, just know that a space advances the output by one screen/"page", and typing q will exit the display.

  3. Ask the Google, e.g. search for ls man page

    • Can be easier to read

Many 3rd party tools, especially bioinformatics tools, may bundle a number of different functions into one command. For these tools, just typing in the command name then Enter may provide top-level usage information. For example, the bwa tool that aligns sequencing reads to a reference genome:

Use the program name alone as a command to get help
bwa

Produces something like this:

bwa top-level help information
Program: bwa (alignment via Burrows-Wheeler transformation) Version: 0.7.16a-r1181 Contact: Heng Li <lh3@sanger.ac.uk> Usage: bwa <command> [options] Command: index index sequences in the FASTA format mem BWA-MEM algorithm fastmap identify super-maximal exact matches pemerge merge overlapping paired ends (EXPERIMENTAL) aln gapped/ungapped alignment samse generate alignment (single ended) sampe generate alignment (paired ended) bwasw BWA-SW for long queries shm manage indices in shared memory fa2pac convert FASTA to PAC format pac2bwt generate BWT from PAC pac2bwtgen alternative algorithm for generating BWT bwtupdate update .bwt to the new format bwt2sa generate SA from BWT and Occ Note: To use BWA, you need to first index the genome with `bwa index'. There are three alignment algorithms in BWA: `mem', `bwasw', and `aln/samse/sampe'. If you are not sure which to use, try `bwa mem' first. Please `man ./bwa.1' for the manual.