Basic AWK Usage

Consider network connection data:

$ cat connections.log
Protocol  RecvQ  SendQ  LocalAddress         RemoteAddress           Status
tcp       0       0      0.0.0.0:3306         0.0.0.0:*              LISTEN
tcp       0       0      0.0.0.0:80           0.0.0.0:*              LISTEN
tcp       0       0      127.0.0.1:9000       0.0.0.0:*              LISTEN

Extract specific colums:

$ awk '{print $1, $4}' connections.log
Protocol LocalAddress
tcp 0.0.0.0:3306
tcp 0.0.0.0:80
tcp 127.0.0.1:9000

Formatted output:

$ awk '{printf "%-8s %-12s %s\n", $1, $4, $5}' connections.log
Protocol LocalAddress  RemoteAddress
tcp      0.0.0.0:3306  0.0.0.0:*
tcp      0.0.0.0:80    0.0.0.0:*
tcp      127.0.0.1:9000 0.0.0.0:*

Filetring and Variables

Filter records using conditions:

$ awk '$3==0 && $6=="LISTEN"' connections.log
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN

Common built-in variables:

Custom field separator:

$ awk -F: '{print $1, $3}' /etc/passwd
root 0
bin 1
daemon 2

Pattern Matching

Filter using regular expressions:

$ awk '$6 ~ /ESTABLISHED/' connections.log
tcp 0 0 localhost:42376 remote:5231 ESTABLISHED

Inverse matching:

$ awk '$6 !~ /WAIT/' connections.log
tcp 0 0 localhost:80 remote:4521 ESTABLISHED

Data Processing

Split files by field value:

$ awk 'NR!=1{print > $6}' connections.log
$ ls
ESTABLISHED  LISTEN  WAIT

Statistical calculations:

$ ls -l *.log | awk '{total+=$5} END{print total}'
2456712

Count status occurrences:

$ awk 'NR!=1{status_count[$6]++} END{for(s in status_count) print s, status_count[s]}' connections.log
ESTABLISHED 12
WAIT 8
LISTEN 3

AWK Script Structure

Sample student data:

$ cat grades.txt
Alice 101 85 92 78
Bob   102 72 85 91
Carol 103 93 88 84

Script example:

#!/usr/bin/awk -f
BEGIN {
  math_sum = 0
  printf "%-10s %-5s %s\n", "Name", "ID", "Average"
}
{
  math_sum += $3
  avg = ($3+$4+$5)/3
  printf "%-10s %-5d %.1f\n", $1, $2, avg
}
END {
  printf "Math average: %.1f\n", math_sum/NR
}

Advanced Techniques

Environment variable usage:

$ threshold=80
$ awk -v limit=$threshold '$3 > limit' grades.txt
Alice 101 85
Carol 103 93

Multi-field separator:

$ awk -F'[: ]' '{print $2}' data.txt

Custom calculations:

seq 5 | awk '{sum += $1} END{print sum}'
15