Unlocking the Power of SED: 5 Practical Applications in Text Processing
Written on
Understanding SED: An Introduction
SED, short for Stream Editor, is a powerful utility in Unix that excels in text processing, specifically for parsing and substituting text. As a command-line based, line-oriented editor, it takes in files and supports regular expressions. Although its syntax may appear daunting to new Linux command line users, this guide will illustrate practical applications of SED that allow beginners to tackle complex tasks with just a single line of command.
Before diving in, ensure you have a basic understanding of navigating a Linux or Unix-like environment.
Let's set up our example. Create a text file with the following content and save it as jobclass.txt:
1 Warrior Melee Str
2 Knight Melee Str
3 Mage Range Int
4 Priest Melee Int
5 Hunter Range Agi
6 Assassin Melee Agi
7 Summoner Range Int
8 Bard Range Int
9 Paladin Melee Str
10 Templar Melee Str
11 Wizard Range Int
12 Warlock Range Int
13 Ranger Range Agi
14 Monk Melee Agi
15 Engineer Range Str
If you’re into role-playing games, this content might seem familiar. Now, let’s get started!
Substituting Text with SED
SED is widely used for text substitution. The basic syntax is as follows:
sed 's/current text/new text/g' filename
For example, to replace "Agi" with "Dex," you would use:
sed 's/Agi/Dex/g' jobclass.txt
The resulting output will be:
1 Warrior Melee Str
2 Knight Melee Str
3 Mage Range Int
4 Priest Melee Int
5 Hunter Range Dex
6 Assassin Melee Dex
7 Summoner Range Int
8 Bard Range Int
9 Paladin Melee Str
10 Templar Melee Str
11 Wizard Range Int
12 Warlock Range Int
13 Ranger Range Dex
14 Monk Melee Dex
15 Engineer Range Str
In this instance, all occurrences of "Agi" have been replaced with "Dex."
Replacing Whitespaces
In cases where you need to substitute spaces, use s instead of a space bar:
sed 's/s/new text/g' filename
For example, to replace all spaces with commas in jobclass.txt, you would run:
sed 's/s/,/g' jobclass.txt
The output will be:
1,Warrior,Melee,Str
2,Knight,Melee,Str
3,Mage,Range,Int
4,Priest,Melee,Int
5,Hunter,Range,Agi
6,Assassin,Melee,Agi
7,Summoner,Range,Int
8,Bard,Range,Int
9,Paladin,Melee,Str
10,Templar,Melee,Str
11,Wizars,Range,Int
12,Warlock,Range,Int
13,Ranger,Range,Agi
14,Monk,Melee,Agi
15,Engineer,Range,Str
Here, all spaces have been replaced with commas.
Removing Specific Patterns
To eliminate specific text patterns, use the following syntax:
sed 's/current text//g' filename
For instance, to remove "Int" from jobclass.txt, the command would be:
sed 's/Int//g' jobclass.txt
The output will show:
1 Warrior Melee Str
2 Knight Melee Str
3 Mage Range
4 Priest Melee
5 Hunter Range Agi
6 Assassin Melee Agi
7 Summoner Range
8 Bard Range
9 Paladin Melee Str
10 Templar Melee Str
11 Wizard Range
12 Warlock Range
13 Ranger Range Agi
14 Monk Melee Agi
15 Engineer Range Str
As seen, all instances of "Int" have been removed.
Deleting Lines with Specific Patterns
To remove entire lines based on a specific text pattern, use the following syntax:
sed '/current text/d' filename
For instance, to delete lines containing "Str," you would execute:
sed '/Str/d' jobclass.txt
The result will be:
3 Mage Range Int
4 Priest Melee Int
5 Hunter Range Agi
6 Assassin Melee Agi
7 Summoner Range Int
8 Bard Range Int
11 Wizard Range Int
12 Warlock Range Int
13 Ranger Range Agi
14 Monk Melee Agi
All lines with the word "Str" have been removed.
Eliminating Blank Lines
For removing blank lines, use a similar syntax as above, where a blank line is represented by ^$:
sed '/^$/d' filename
Before proceeding, create a new file with the following content and save it as jobclass2.txt:
1 Warrior Melee Str
2 Knight Melee Str
3 Mage Range Int
4 Priest Melee Int
5 Hunter Range Agi
Once ready, run:
sed '/^$/d' jobclass2.txt
The output will be:
1 Warrior Melee Str
2 Knight Melee Str
3 Mage Range Int
4 Priest Melee Int
5 Hunter Range Agi
Bonus Content: Combining SED with Other Commands
SED can be combined with other Linux/Unix commands for enhanced functionality. The syntax is as follows:
command | sed syntax
For example, to list files while removing the word "job," you could use:
ls -l | sed 's/job//g'
The output would appear as:
total 8
-rw-rw-r-- 1 user user 295 Jan 18 11:16 class.txt
-rw-rw-r-- 1 user user 104 Jan 18 16:02 class2.txt
This guide aims to provide a foundational understanding of SED’s practical uses.
May the code be with you,
-Arc
This video explores the capabilities of the grep and sed commands, highlighting their usefulness in Linux and Unix environments.
This video provides an intermediate overview of essential Bash commands, including grep, sed, awk, tar, less, and gzip.