Overview and Mechanics

The sed (stream editor) is a powerful utility designed for parsing and transforming text streams. Unlike interactive editors, it operates automatically based on scripts or command-line arguments. Internally, sed processes input line by line. Each line is copied into a temporary area known as the Pattern Space. The commands defined in the script are executed against this buffer, and if the -n option is not active, the result is printed to standard output. The process repeats until the end of the file is reached. By default, the original file remains unchanged unless output redirection or the in-place flag is utilized.

Syntax and Configuration Options

The general usage follows two primary patterns:

# Inline execution
sed [options] 'script_expression' filename

# External script execution
sed [options] -f script_file filename

Common flags include:

• -e: Execute the provided expression immediately (default behavior).
• -f filename: Load the sequence of commands from an external file.
• -i: Modify files directly without generating separate output files.
• -n: Suppress automatic printing; allows explicit control via p.
• -r / -E: Enable extended regular expressions syntax.

Addressing Lines and Ranges

Commands can target specific lines using numbers or address patterns.

# Target line 10 specifically
sed '10d' data.txt

# Delete lines 5 through 15
sed '5,15d' config.yml

# Delete anything starting with '#comment'
sed '/^#/d' log.file

To operate only on matched lines, you can specify ranges between addresses:

# Print lines between START and END markers
sed -n '/START/,/END/p' inventory.csv

Text Transformation Commands

The core capability of sed lies in modifying the pattern space content.

Substitution (s)

The substitute command replaces characters or substrings. The syntax is s/search/replace/flags.

# Replace 'client' with 'server' globally in one line
sed 's/client/server/g' users.conf

# Replace from the third occurrence onwards
echo "alpha beta alpha beta alpha" | sed 's/beta/GAMMA/3'
# Output: alpha beta alpha GAMMA alpha

If the delimiter character conflicts with your search string, any character can be used instead of forward slashes:

# Using pipe as delimiter avoids escaping slashes
sed 's|/usr/local/bin|/opt/app/bin|g' path.list

Insertion and Change

You can add new text before or after matches, or replace entire lines.

# Add text below matching lines
sed '/Production/a\    # Environment is live' app.yaml

# Insert text above line number 3
sed '3i\New Header Row' table_data.csv

# Replace line 2 entirely
sed '2c\Updated Status Line' status.txt

Deletion (d)

Rapidly remove unwanted sections.

# Remove all blank lines
sed '/^$/d' report.txt

# Drop lines where the word "DEBUG" appears
sed '/DEBUG/d' logs/system.log

Advanced Operations and Logic

Regular Expressions and Markers

sed supports basic regex anchors. Special characters help refine matching:

• ^: Anchors to the beginning of a line.
• $: Anchors to the end of a line.
• \{m,n\}: Matches the preceding element m to n times.
• [...]: Defines character sets.

Capture groups allow referencing previously matched text in the replacement:

# Swap first and second words using backreferences
echo "Hello World" | sed 's/\([^ ]*\) \(.*\)/\2 \1/'
# Output: World Hello

# Wrap every digit in brackets
echo "Version 2.4 Build 9" | sed 's/[0-9]/&/g'

Buffer Management

Beyond the Pattern Space, sed uses a Hold Space for temporary storage.

• h: Copy Pattern Space to Hold Space.
• H: Append Pattern Space to Hold Space.
• g: Retrieve content from Hold Space to Pattern Space.
• x: Exchange contents of Pattern and Hold Spaces.

# Save line 5 to hold space, append to last line when finished
sed -n '5h; $G; p' notes.txt

Script Control Flow

You can loop or jump between command labels using b (branch) and t (test).

# Skip next line if test is found
sed '/ERROR/{ n; b }' error_log.txt

Batch Processing with Scripts

Complex tasks are best organized in a script file passed via the -f option. Comments start with #, and multiple commands on one line must be separated by semicolons.

#!/bin/bash
# script.sed
/^LOG_START/,/^LOG_END/{
    /INFO/d
}
s/:time:/TIMESTAMP:/g

Usage:

sed -f script.sed server_output.txt

External Input and Output

seD can read content from other files or write specific output streams.

# Read 'footer.txt' content after matching 'header' lines
sed '/header/r footer.txt' index.html

# Write lines matching 'TODO' to a separate list
sed -n '/TODO/w todo_list.txt' readme.md

Variable Expansion

When shell variables are needed inside the command, double quotes allow expansion:

OLD="staging"
NEW="production"
sed "s/$OLD/$NEW/" deployment_map.json