Linux fundamentals
This page should serve as a reference for the many "things Linux" we use in this course. It is by no means complete – Linux is **huge** – but offers introductions to many important topics.
See also this page, which provides lists of the most common Linux commands, by category, as well as their most useful options: Some Linux commands
- 1 Some Linux commands
- 2 Terminal programs, shells and commands
- 3 Terminal input
- 4 Getting around in the shell
- 5 Streams and Piping
- 5.1 Standard streams and redirection
- 5.2 Piping
- 5.2.2 piping a histogram
- 5.2.2.1 The power of chaining pipes
- 6 Viewing text in files
- 6.1 cat, more or less
- 6.2 Introducing grep
- 6.3 head and tail
- 7 More Linux concepts
- 8 Bash control flow
- 9 File attributes
- 9.1 Owner and Group
- 9.2 Permissions
- 10 Copying files between TACC and your laptop
- 11 Editing files
- 11.1 nano
- 11.1.1 Start the nano text editor
- 11.2 emacs
- 11.2.1 Start the emacs text editor
- 11.3 Line ending nightmares
- 11.4 Komodo Edit for Mac and Windows
- 11.5 Notepad++ for Windows
- 11.1 nano
- 12 Other bash resources
Some Linux commands
This page provides lists of the most common Linux commands, by category, as well as their most useful options:
Terminal programs, shells and commands
You need a Terminal program in order to ssh to a remote computer.
Macs and Linux have a Terminal program built-in
Windows options:
Windows 10+
Command Prompt and PowerShell programs have ssh and scp (may require latest Windows updates)
Start menu → Search for Command
Putty – http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
simple Terminal and file copy programs
download either the Putty installer or just putty.exe (Terminal) and pscp.exe (secure copy client)
Windows Subsystem for Linux – Windows 10 Professional includes a Ubuntu-like bash shells
See https://docs.microsoft.com/en-us/windows/wsl/install-win10
We recommend the Ubuntu Linux distribution, but any Linux distribution will have an SSH client
Use ssh (secure shell) to login to a remote computers.
SSH to a remote computer
# General form:
ssh <user_name>@<full_host_name>
# For example
ssh abattenh@ls6.tacc.utexas.eduThe bash shell REPL and commands
When you type something in at a bash command-line prompt, it Reads the input, Evaluates it, then Prints the results, then does this over and over in a Loop. This behavior is called a REPL – a Read, Eval, Print Loop. The shell executes the command line input when it sees a linefeed, which happens when you press Enter after entering the command.
The input to the bash REPL is a command, which consists of:
The command name (any of the built-in Linux/Unix commands, or the name of a user-written script or program)
One or more (optional) options, usually noted with a leading dash (-) or double-dash (--).
short (1-character) options can be provided separately, prefixed by a single dash (-)
or can be combined with the combination prefixed by a single dash
long (multi-character or "word") options are prefixed with a double dash (--) and must be supplied separately.
Both long and short options can be assigned a value
One or more command-line arguments, which are often (but not always) file names
Some examples using the ls (list files) command:
ls # example 1 - no options or arguments
ls -l # example 2 - one "short" (single character) option only (-l)
ls --help # example 3 - one "long" (word) option (--help)
ls .profile # example 4 - one argument, a file name (.profile)
ls --width=20 # example 5 - a long option that has a value (--width is the option, 20 is the value)
ls -w 20 # example 6 - a short option w/a value, as above, where -w is the same as --width
ls -l -a -h # example 7 - three short options entered separately (-l -a -h)
ls -lah # example 8 - three short options that can be combined after a dash (-lah)The arguments to ls are one or more file or directory names. If no arguments are provided, the contents of the current directory are listed.
If an argument is a directory name, the contents of that directory are listed.
Some handy options for ls:
-l shows a long listing, including file permissions, ownership, size and last modified date.
-a shows all files, including dot files whose names start with a period ( . ) which are normally not listed
-h says to show file sizes in human readable form (e.g. 12M instead of 12201749)
A good place to start learning built-in Linux commands and their options is on the Some Linux commands page.
Getting help
How do you find out what options and arguments a command uses?
In the Terminal, type in the command name then the --help long option (e.g. ls --help)
Works for most Linux commands; 3rd party tools may use -h or -? or even /? instead
May produce a lot of output, so you may need to scroll up quite a bit, or pipe the output to a pager
e.g. ls --help | more (a space advances the output by one screen/"page", and typing Ctrl-C will exit more)
Use the built-in manual system (e.g. type man ls)
This system uses the less pager
For now, just know that a space advances the output by one screen/"page", and typing q will exit the display.
Ask the Google, e.g. search for ls man page
Can be easier to read
Many 3rd party tools, especially bioinformatics tools, may bundle a number of different functions into one command. For these tools, just typing in the command name then Enter may provide top-level usage information. For example, the bwa tool that aligns sequencing reads to a reference genome:
Use the program name alone as a command to get help
bwaProduces something like this:
bwa top-level help information
Program: bwa (alignment via Burrows-Wheeler transformation)
Version: 0.7.16a-r1181
Contact: Heng Li <lh3@sanger.ac.uk>
Usage: bwa <command> [options]
Command: index index sequences in the FASTA format
mem BWA-MEM algorithm
fastmap identify super-maximal exact matches
pemerge merge overlapping paired ends (EXPERIMENTAL)
aln gapped/ungapped alignment
samse generate alignment (single ended)
sampe generate alignment (paired ended)
bwasw BWA-SW for long queries
shm manage indices in shared memory
fa2pac convert FASTA to PAC format
pac2bwt generate BWT from PAC
pac2bwtgen alternative algorithm for generating BWT
bwtupdate update .bwt to the new format
bwt2sa generate SA from BWT and Occ
Note: To use BWA, you need to first index the genome with `bwa index'.
There are three alignment algorithms in BWA: `mem', `bwasw', and
`aln/samse/sampe'. If you are not sure which to use, try `bwa mem'
first. Please `man ./bwa.1' for the manual.