Content Comparison

...

Notice that when you call more, it outputs some text then stops and indicates that there is additional text available (--More-- text available).

• Just press the spacebar to see the next page.
• If there is additional output, you'll see the  --More-- indicator again; if not, the command prompt appears again.
• To end the more display, just type q (quit) or Ctrl-c.

So more is really simple, and is great for scanning through a large file. But what if you're looking for something in particular? Or want to go back and forth? That's where the less pager comes in, providing navigation and search capabilities using its own special characters (metacharacters).

...

The word grep stands for general regular expression parser, and nearly .

Nearly every programming language offers grep functionality, where a pattern you specify – a regular expression or regex – describes how the search is performed.

In Unix, the grep program performs regular-expression text searching, and displays lines where the pattern text is found.

There are many grep regular expression metacharacters that control how the search is performed. This is a complex topic and we defer it to the Intermediate Unix workshop (for a preview, see the grep command).

For now, we will just use grep <pattern> <file> where <pattern> just contains alphanumeric characters (A-Z, a-z, 0-9). And note that , and use options:

• grep -i will perform a case-insensitive search

...

• grep -n will display line numbers where the pattern was matched

...

...

Standard streams and piping

...

The power of the Linux command line is due in no small part to the power of piping. The pipe operator ( | ) connects one program's standard output to the next program's standard input.

Image RemovedImage Added

The key to the power of piping is that most Unix commands can accept input from standard input instead of from files. So, for example, these two expressions appear equivalent:

...

1. In the 1st, the more command reads some input from the jabberwocky.txt file
   • then writes the output to standard output, which is displayed on your Terminal
   • it pauses at page boundaries (--More--) waiting for input on standard input
   • when it receives a space on character on standard input it reads more input from jabberwocky.txt
   • then writes the output to standard output, which is displayed on your Terminal
2. In the 2nd, the cat command reads its input from the jabberwocky.txt file
   • then writes its output to standard output
   • the pipe operator ( | ) then connects the standard output from cat to standard input of the more command
   • the more command then reads its input from standard input, instead of from a file
     • then writes its output to standard output, which is displayed on your Terminal
     • more continues its processing similar to #1, except reading from its standard input instead of the file

...

• In #2, the cat command "blocks" writing to its standard output until more says it's ready for more input
  • This "write until block" / "read when input available" behavior makes streams a very efficient means of inter-process communication.
• In #1, more can report how much of the file has been read, e.g. --More-- (24%) because it has access to the size information for the file it is reading.
  
  • But in #2, the text is "anonymous input" – from standard input – so more doesn't know how much of the total has been provided.

...

NAME
    cat - concatenate files and print on the standard output
SYNOPSIS
    cat [OPTION]... [FILE]...
DESCRIPTION
    Concatenate FILE(s) to standard output.
    With no FILE, or when FILE is -, read standard input.

...

With no options, head shows the first 10 lines of its input and tail shows the last 10 lines. You can use the -n option followed by a number to specify how many lines lines to view, or just put the number you want after a dash (e.g. -5 for 5 lines or -1 for 1 line).

...

But what if you want to see lines in the middle of a file? Here's where a special feature of tail comes in handy. If you use tail and put a plus sign (+) in front of the number (with or without the -n option), tail will start its output at that line.

Let's pipe line-numbered output from cat to tail to see how this works:

...

...

. Note we use cat -n to provide input with line numbers because neither head nor tail has line numbering options.

cat -n haiku.txt | tail -n 75   # display the last 75 lines of haiku.txt

cat -n haiku.txt | tail -n +75  # display text in haiku.txt starting at 
line      7 cat -n haiku.txt | tail +10    # display text in haiku.txt starting at line 10

When you use the tail -n +<integer> syntax it will display all output from that line to the end of its input. So to view only a few lines starting at a specified line number, pipe the output to head:

# display 2 lines of haiku.txt starting at# line 95
cat -n haiku.txt | tail -n +96    | head# -2display cattext -nin haiku.txt | head -10 | tail -2

Exercise 2-4

Use cat, head and tail to display the middle stanza of haiku.txt.

 starting at 
                               # line 6

When you use the tail -n +<integer> syntax it will display all input starting from that line until the end of its input. So to view only a few lines starting at a specified line number, pipe the output to head:

# display 2 lines of haiku.txt starting at line 9
cat -n haiku.txt | tail -n +9 | head -

2
cat -n haiku.txt | tail +

9 | head -

n 2

cat -n haiku.txt | head -

10 | tail -

Text lines and the Terminal

...

2

Exercise 2-4

Use cat, head and tail to display the middle stanza of haiku.txt.

cat -

n 

haiku.txt | 

tail -n 

+5 |

Note that most Terminals let you increase/decrease the width/height of the Terminal window. But there will always be single lines too long for your Terminal width and too many lines of text for its height.

So how long is a line? And how many lines are there in a file? The wc (word count) command can tell us this.

• wc -l reports the number of lines in its input
• wc -c reports the number of characters in its input (including invisible linefeed characters)
• wc -w reports the number of words in its input (groups of space-separated text characters)

Examples:

 head -n 3
cat -n haiku.txt | tail +5 | head -3

cat -n haiku.txt | head -7 | tail -3

Text lines and the Terminal

Sometimes a line of text is longer than the width of your Terminal. In this case the text is wrapped. It can appear that the output is multiple lines, but it is not. We can see that by looking at lines of the mobydick.txt file, that has some very long lines:

tail -1 mobydick.txt
cat -n mobydick.txt | more

Note that most Terminals let you increase/decrease the width/height of the Terminal window. But there will always be single lines too long for your Terminal width or too many lines of text for its height.

So how long is a line? And how many lines are there in a file? The wc (word count) command can tell us this.

• wc -l reports the number of lines in its input
• wc -c reports the number of characters in its input (including invisible linefeed characters)
• wc -w reports the number of words in its input (groups of space-separated text characters)

Examples:

wc -l mobydick.txt            # Reports the number of lines in the
                              # mobydick.txt file
cat mobydick.txt | wc -l      # Reports the number of lines of its input

tail -1 mobydick.txt | wc -c  # Reports the number of characters in 
                        # Reports the number of lines in# the last mobydick.txt file
catline
head -5 mobydick.txt | wc -l -c  # Reports the total number of characters in
        # Reports the number of lines of its input tail -1 mobydick.txt | wc -c  # Reports the number of characters of# the first last5 mobydick.txt linelines

When you give wc -l multiple files, it reports the line count of each, then a total.

...

...

We've talked about viewing text using various Unix commands – but what exactly is text? That is, what is stored in files that the shell interprets as text?

Inside of files, text isn't characters at all – it is all numbers, because that's all computers know.

On standard Unix systems, each text character is stored as one byte – eight binary bits – in a format called ASCII (American Standard Code for Information Interchange). Eight bits can store 2^{^8} = 256 values, numbered 0 - 255.

...

However not all ASCII "characters" are printable -- in fact the "printable" characters start at ASCII 32 (space).

ASCII values 0 - 31 have special meanings. Many were designed for use in early modem protocols, such as EOT (end of transmission) and ACK (acknowledge), or for printers, such as VT (vertical tab) and FF (form feed).

The non-printable ASCII characters we care most about are:

• Tab (decimal 9, hexadecimal 0x9, octal 0o011)
  • backslash escape: \t
• Linefeed/Newline (decimal 10, hexadecimal 0xA, octal 0o012)
  • backslash escape: \n
• Carriage Return (decimal 13, hexadecimal 0xD, octal 0o015)
  • backslash escape: \r

...

• The numeric offset of the 16-character line, in hexadecimal (base 16)
  
  • 16 decimal is 0x10 hex
• The numeric value (ASCII code) for each character, again in hexadecimal
  • each 2-digit hex number represents one 8-bit byte/character
• The translated text
  • The display character associated with each ASCII code, or a period ( . ) for non-printable charactersthe , written between a greater than ( > ) and less than ( < ) sign

Notice that spaces are ASCII 0x20 (decimal 32), and the newline characters appear as 0x0a (decimal 10).

Why hexadecimal? Programmers like hexadecimal (base 16) because it is easy to translate hex digits to binary, which is how everything is represented in computers. And it can sometimes be important to know which binary bits are 1s and which are 0s. See Decimal and Hexadecimal for more information.

...