SH

Text Processing

Bash & Linux · 9 entries

Search Text with Patterns

syntax
grep [options] pattern [file...]
example
grep -rn 'TODO' src/
grep -i 'error' /var/log/syslog
grep -c 'SELECT' queries.sql
grep -v '^#' nginx.conf
output
src/api/handler.go:42:  // TODO: add rate limiting
src/lib/cache.ts:18:  // TODO: implement TTL

Note -r searches recursively through directories. -n shows line numbers. -i is case-insensitive. -v inverts the match (shows non-matching lines). -c counts matches. -l lists only filenames with matches. Use -E for extended regex or egrep.

Stream Editor for Find & Replace

syntax
sed [options] 's/pattern/replacement/flags' file
example
sed 's/localhost/0.0.0.0/g' config.ini
sed -i.bak 's/v1\.2/v1.3/g' version.txt
sed -n '10,20p' access.log
sed '/^$/d' notes.txt

Note -i edits in place. -i.bak creates a backup before editing (strongly recommended). g flag replaces all occurrences on a line, not just the first. -n suppresses default output; use with p to print specific lines. /d deletes matched lines.

Column-Based Text Processing

syntax
awk 'pattern { action }' file
example
awk '{print $1, $4}' access.log
awk -F',' '$3 > 1000 {print $1, $3}' sales.csv
awk '{sum += $5} END {print "Total:", sum}' report.tsv
output
192.168.1.40 [28/Mar/2026:10:15:23
Alice 2340
Total: 87450

Note By default, awk splits on whitespace. -F sets a custom delimiter. $0 is the whole line, $1 is the first field, NR is the line number, NF is the number of fields. BEGIN runs before processing; END runs after.

Extract Fields or Characters

syntax
cut [options] file
example
cut -d',' -f1,3 employees.csv
cut -c1-10 logfile.txt
output
name,department
Alice,Engineering
Bob,Marketing

Note -d sets the delimiter (default is tab). -f selects fields by number. -c selects character positions. For more complex extraction, awk is usually more flexible.

Translate or Delete Characters

syntax
tr [options] set1 [set2]
example
echo 'Hello World' | tr 'A-Z' 'a-z'
cat data.csv | tr ',' '\t'
tr -d '\r' < windows_file.txt > unix_file.txt
output
hello world

Note tr reads only from stdin (it cannot take a filename argument). -d deletes characters in set1. -s squeezes repeated characters. Converting \r is handy for fixing Windows line endings.

Build Commands from Stdin

syntax
xargs [options] command
example
find . -name '*.tmp' -print0 | xargs -0 rm
cat urls.txt | xargs -P 4 -I {} curl -sO {}
grep -rl 'oldFunc' src/ | xargs sed -i 's/oldFunc/newFunc/g'

Note -0 with find -print0 handles filenames with spaces and special characters safely. -I {} replaces {} with each input line. -P runs N processes in parallel for significant speedups on I/O-bound tasks.

Split Output to File and Stdout

syntax
tee [options] file...
example
make build 2>&1 | tee build.log
echo 'new entry' | sudo tee -a /etc/hosts

Note -a appends instead of overwriting. tee is essential for writing to root-owned files because 'sudo echo x > /root/file' fails (the redirect runs as your user, not root). Use 'echo x | sudo tee /root/file' instead.

Format Tabular Output

syntax
column [options]
example
cat data.csv | column -t -s','
mount | column -t
output
name     age  department
Alice    30   Engineering
Bob      25   Marketing

Note -t creates a neatly aligned table. -s sets the input delimiter. Great for making CSV data or command output readable in the terminal.

Merge Lines Side by Side

syntax
paste [options] file1 file2...
example
paste -d',' names.txt scores.txt
seq 10 | paste - - - -
output
Alice,95
Bob,87
Carol,92

Note -d sets the delimiter (default is tab). Using - multiple times reads successive lines from stdin into columns. The seq example arranges 1-10 into a 4-column layout.