AWK Text Processing Fundamentals

Column Extraction and Field Separation

In AWK, you can reference columns using $0 (entire line) through $NF (last column). The following examples demonstrate column extraction:

# Display entire line
awk '{print}'

# Display first column
awk '{print $1}'

# Display second column
awk '{print $2}'

# Display last column
awk '{print $NF}'

The field separator (-F) allows you to specify custom delimiters:

# Use comma as separator
awk -F "," '{print $2}'

# Use multiple separators (comma, pipe, semicolon)
awk -F "[,;|]" '{print $2}'

String Pattern Matching

AWK provides powerful pattern matching capabilities:

# Print lines containing "pattern"
awk '/pattern/'

# Print second column of lines containing "pattern"
awk '/pattern/{print $2}'

# Print second column if first column matches "10010"
awk '$1 ~ /10010/{print $2}'

# Exact match using word boundaries
awk '$1 ~ /\<10010\>/{print $2}'

# Multiple conditions
awk '$1 ~ /10010/ && $NF ~ /liantong2/{print $2}'

Conditional Statements in AWK

AWK supports conditional logic with if statements:

# Print lines where second column >= 30
awk '$2 >= 30{print}'

# Print first and second columns where second column >= 30
awk '$2 >= 30{print $1, $2}'

# Multiple conditions
awk '($2 >= 30) && ($1 ~ /wang/){print $3}'

# If-else structure
awk '{if (NR % 2 == 0){print $0} else {printf "%s ", $0}}'

# Nested if-else
awk '{ 
  if (NR == 2){print $0/4} 
  else if (NR == 4){print $0/3} 
  else {print} 
}'

Loop Structures in AWK

AWK supports both for and while loops:

# For loop with condition
echo "5 6 7 8" | awk '{for (i=1; i<=NF; i++) {if ($i > 6) {print $i}}}'

# Sum using for loop
echo "5 6 7 8" | awk '{for (i=1; i<=NF; i++) {sum+=$i}; {print sum}}'

# Calculate average
awk '{total+=$1} END {print total/NR}'

# Find maximum value (with numeric conversion)
awk 'BEGIN {max = 0} {if ($1+0 > max+0) max=$1} END {print max}'

# Find minimum value (with numeric conversion)
awk 'BEGIN {min = 999999} {if ($1+0 < min+0) min=$1} END {print min}'

# While loop for summation
awk 'BEGIN{total=0; i=1; while(i<=100) {total+=i; i++} print total}'

# Do-while loop for summation
awk 'BEGIN{total=0; i=1; do {total+=i; i++} while(i<=100) print total}'

Sample Data File

user1 23 1500
user2 32 2300
user3 43 3200
===========
user1,23,1500
user2,32,2300
user3,43,3200
===========
user3,43;3200
user3;43,3200
user3,43,3200
==========Pattern Match=
10086 cmc mobile
100860 ccc mobile2
10010 cnc联通
100101 ccc联通2
10000 ctc典型
==========if 条件=
192.168.1.17
down
192.168.1.103
open
=======if-else结构=
125
32
126
54
==========
cc 750
aa 181
bb 37
bb 725
aa 42
cc 72
=====BEGIN END=====
one:16.32
two:13.28
three:15.65