Mastering Advanced sed Operations

sed operates with two internal buffers: the pattern space (active working area) and the hold space (auxiliary storage).

  • Both buffers start empty.
  • For each input line, sed performs a cycle:
    • Reads a line, strips its trailing newline, and loads it into the pattern space.
    • Applies commands sequentially; each may be constrained by an address — a condition that must evaluate to true before the command executes.
    • Unless suppressed with -n, the pattern space is printed to stdout upon reaching the end of the script.
    • The cycle repeats with the next input line.
  • The pattern space is cleared between cycles unless preserved explicitly.
  • The hold space persists across cycles, enabling stateful processing.

Addressing Lines

Numeric Addresses

  • 3s/foo/bar/ applies only to line 3.
  • 2~3p prints lines 2, 5, 8, … (i.e., 2 + n * 3 where n ≥ 0).
  • $ matches the final line — behavior remains consistent unless modified via script logic.

Regex-Based Addresses

By default, sed uses Basic Regular Expressions (BRE); -E or -r enables Extended Regular Expressions (ERE).

  • /pattern/command — forward slashes can be replaced with any non-alphanumeric delimiter (e.g., \%pattern%command or \~pattern~command).
  • GNU extensions support modifiers:
    • I: case-insensitive matching.
    • M: multi-line mode — ^ and $ match line boundaries within multi-line pattern space.

Range Addresses

Defined by two comma-separated addresses:

  • 4,8d: deletes lines 4 through 8 inclusive.
  • 2,/END/d: deletes from line 2 up to and including the first line matching /END/.
  • /START/,/END/s/old/new/g: applies substitution across the entire block bounded by the first START and subsequent END.
  • GNU-specific variants:
    • 0,/first/: allows matching on line 1 (unlike 1,/first/, which skips line 1 if no prior match exists).
    • 5,+2: targets lines 5–7.
    • 5,~3: targets lines 5, 6, 7, 8 (i.e., up to but not beyond the next multiple of 3).

Multi-Line Processing

Commands ending in uppercase (D, G, H, N, P) preserve embedded newlines and enable line-aware buffering:

  • N: appends the next input line (with leading newline) to pattern space.
  • H: appends pattern space (with leading newline) to hold space.
  • G: appends hold space (with leading newline) to pattern space.
  • D: deletes up to the first newline in pattern space, then restarts the cycle without printing.
  • P: prints up to the first newline only.

Example: extract overlapping line pairs:

seq 6 | sed -n 'N; l; D'

Output:

1\n2$
2\n3$
3\n4$
4\n5$
5\n6$

This works by building a two-line window, printing its raw representation, then discarding the first line to slide the window forward.

Paragraph-Oriented Processing

To treat contiguous non-empty lines as logical blocks (e.g., paragraphs), use:

sed '/./{H;$!d}; x; s/SEARCH/REPLACE/g'

  • /./{H;$!d}: for every non-blank line, append it to hold space; skip printing and reload unless it's the last line.
  • x: swaps accumulated content into pattern space.
  • s///: performs substitution across the full block (including internal newlines).

Flow Control and Branching

sed’s default flow is linear and per-line. These commands alter control flow:

  • d: clears pattern space and restarts the cycle — no further commands execute, and nothing prints.
  • D: like d, but only removes up to the first newline; useful in multi-line contexts.
  • b label: unconditionally jumps to :label. If no label is given, it restarts the cycle.
  • t label: jumps to :label only if the most recant s/// succeeded.
  • T label: jumps only if the last s/// failed.

Labels are declared as :name (alphanumeric, no spaces). Key distinction:

  • Restarting the cycle prints the current pattern space and reads the next line.
  • Branching to a label bypasses both actions — ideal for loops or conditional reprocessing.

Simple infinite loop:

seq 3 | sed ':loop; b loop'

Break it using n (print & clear, then read next line) or N (append next line without printing):

# Print first line, then exit
seq 3 | sed ':a; =; q; :b; b a'

Tags: sed regex shell-scripting linux-cli text-processing

Posted on Fri, 28 Aug 2026 16:07:51 +0000 by Defibber