bash编程——sed 50例
sed是GNU/Linux中有用的文本处理工具。sed的完整英文名称是Stream Editor。使用sed
命令可以很容易地完成许多简单和复杂的文本处理任务。可以使用带有sed命令的正则表达式来搜索、替换和删除文本或文件中的任何特定字符串。但是此命令会临时执行所有类型的修改,并且默认情况下不会更改原始文件内容。如果需要,用户可以将修改后的内容存储到另一个文件中。sed命令的基本用法在本教程中通过使用50个独特的示例进行了说明。
在开始本教程之前,您必须通过运行以下命令来检查操作系统中sed的安装版本。
$ sed --version
sed (GNU sed) 4.4
sed命令基本格式:
sed [options]… [script] [file]
1、基本字符串查找替换
将Bash替换成Perl,'s’表示search和replace任务。
jun@ubuntu:~$ echo "Bash Scripting Language" | sed 's/Bash/Perl/'
Perl Scripting Language
创建weekday.txt文件内容如下:
jun@ubuntu:~$ cat weekday.txt
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
Sunday
jun@ubuntu:~$ sed 's/Sunday/Sunday is holiday/' weekday.txt
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
Sunday is holiday
2、替换指定行匹配的所有字符串
创建Python.txt文件内容如下:
利用行号、‘s’、‘g’,替换第2行中所有Python为perl
jun@ubuntu:~$ cat Python.txt
Python is a very popular language.
Python is easy to use. Python is easy to learn.
Python is a cross-platform language
jun@ubuntu:~$ sed '2 s/Python/perl/g' Python.txt
Python is a very popular language.
perl is easy to use. perl is easy to learn.
Python is a cross-platform language
3、替换每一行中匹配第二次之后的字符串
'g’后面添加匹配每行第几次之后
jun@ubuntu:~$ sed 's/Python/perl/g2' Python.txt
Python is a very popular language.
Python is easy to use. perl is easy to learn.
Python is a cross-platform language
4、正则匹配每行,部分保留,部分替换
创建lang.txt文件内容如下:
保留Programming前面部分,后面替换为Scripting。
\1 指的是前面.*匹配保留的部分。
jun@ubuntu:~$ cat lang.txt
C Language
Shell Programming
Python Programming
Perl Programming
jun@ubuntu:~$ sed 's/\(.*\)Programming/\1Scripting/' lang.txt
C Language
Shell Scripting
Python Scripting
Perl Scripting
5、替换第一行中第一个匹配的字符串
jun@ubuntu:~$ sed '1 s/Python/perl/' Python.txt
perl is a very popular language.
Python is easy to use. Python is easy to learn.
Python is a cross-platform language.
6、替换最后一行第一个匹配的字符串
jun@ubuntu:~$ sed '$ s/Python/Perl/' Python.txt
Python is a very popular language.
Python is easy to use. Python is easy to learn.
Perl is a cross-platform language.
7、替换文件路径
jun@ubuntu:~$ echo /home/ubuntu/code/perl/add.pl | sed 's#/#\\/#g'
\/home\/ubuntu\/code\/perl\/add.pl
8、去掉路径获取文件名
jun@ubuntu:~$ echo "/home/ubuntu/temp/myfile.txt" | sed 's/.*\///'
myfile.txt
jun@ubuntu:~$ echo "/home/ubuntu/temp/myfile.txt" | sed 's#.*/##'
myfile.txt
9、替换有关键字的行中的字符串
替换包含CSE的行中的Count为95,替换包含Civil的行中的Count为100
jun@ubuntu:~$ cat dept.txt
CSE - Count
EEE - Count
Civil - Count
jun@ubuntu:~$ sed '/Civil/ s/Count/100/; /CSE/ s/Count/95/;' dept.txt
CSE - 95
EEE - Count
Civil - 100
10、和以上相反,替换无关键字的行中的字符串
jun@ubuntu:~$ sed '/EEE/! s/Count/100/;' dept.txt
CSE - 100
EEE - Count
Civil - 100
11、在关键字前后添加内容
jun@ubuntu:~$ cat lang.txt
C Language
Shell Programming
Python Programming
Perl Programming
jun@ubuntu:~$ sed 's/\(Programming\)/is \1 Language/' lang.txt
C Language
Shell is Programming Language
Python is Programming Language
Perl is Programming Language
12、删除匹配的行
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/OS/d' os.txt
Windows
Linux
Android
13、删除匹配行之后2行
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/Android/,+2d' os.txt
Windows
Linux
14、删除每行结尾空格和tab等
jun@ubuntu:~$ hexdump -c os.txt
0000000 W i n d o w s \t \n \n L i n u x
0000010 \n \n A n d r o i d
0000020 \n O S \n
0000028
jun@ubuntu:~$ sed 's/[[:blank:]]*$//' os.txt >os2.txt
jun@ubuntu:~$ hexdump -c os2.txt
0000000 W i n d o w s \n \n L i n u x \n \n
0000010 A n d r o i d \n O S \n
000001b
jun@ubuntu:~$
15、删除关键字两次出现的所有行
16、删除空行
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/^$/d' os.txt
Windows
Linux
Android
OS
17、删除无法打印的字符
jun@ubuntu:~$ hexdump -c os.txt
0000000 \t W i n d o w s \n
0000010 \t L i n u x \n
0000020 \t A n d r o i d \n
0000030 \t O S \n
0000035
jun@ubuntu:~$ sed 's/[^[:print:]]//g' os.txt >os2.txt
jun@ubuntu:~$ hexdump -c os2.txt
0000000 W i n d o w s \n
0000010 L i n u x \n
0000020 A n d r o i d \n O S
0000030 \n
0000031
18、在匹配行的末尾添加内容
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/Windows/ s/$/ 10/' os.txt
Windows 10
Linux
Android
OS
19、在匹配行之前插入内容
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/Linux/ s/^/iOS\n/' os.txt
Windows
iOS
Linux
Android
OS
20、在匹配内容之后插入
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed 's/droid/&\nUbuntu/' os.txt
Windows
Linux
Android
Ubuntu
OS
21、在不匹配行之后添加内容
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/Linux/! s/$/ Operating System/' os.txt
Windows Operating System
Linux
Android Operating System
OS Operating System
22、删除不匹配的行
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/Linux/!d' os.txt
Linux
23、多次匹配,在匹配字符串后添加空格
jun@ubuntu:~$ cat Python.txt
Python is a very popular language.
Python is easy to use. Python is easy to learn.
Python is a cross-platform language.
jun@ubuntu:~$ sed 's/to /& /g' Python.txt
Python is a very popular language.
Python is easy to use. Python is easy to learn.
Python is a cross-platform language.
24、列表2中第一列替换成列表1的第三列
jun@ubuntu:~$ cat list1.txt
1001 => Jafar Ali
1023 => Nir Hossain
1067 => John Michel
jun@ubuntu:~$ cat list2.txt
1001 CSE GPA-3.63
1002 CSE GPA-3.24
1023 CSE GPA-3.11
1067 CSE GPA-3.84
jun@ubuntu:~$ sed `cat list1.txt | awk '{print "-e s/"$1"/"$3"/"}'` <<< "`cat list2.txt`"
Jafar CSE GPA-3.63
1002 CSE GPA-3.24
Nir CSE GPA-3.11
John CSE GPA-3.84
25、替换匹配字符串并断行
jun@ubuntu:~$ echo "Bash Perl Python Java PHP ASP" | sed 's/Python/Added Text\n/'
Bash Perl Added Text
Java PHP ASP
26、替换’\n’为’,’
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed -z 's/\n/,/g' os.txt
Windows,Linux,Android,OS,jun@ubuntu:~$
27、替换’,‘为’\n’
jun@ubuntu:~$ echo "Kaniz Fatema,30th,batch" | sed "s/,/\n/g"
Kaniz Fatema
30th
batch
28、忽略大小写,删除匹配的行
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '/linux/Id' os.txt
Windows
Android
OS
29、忽略大小写,替换成新字符串
jun@ubuntu:~$ echo "I like bash programming " | sed 's/Bash/PHP/i'
I like PHP programming
30、关键字替换成大写
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed 's/\(Windows\)/\U\1/Ig' os.txt
WINDOWS
Linux
Android
OS
31、关键字替换成小写
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed 's/\(os\)/\L\1/Ig' os.txt
Windows
Linux
Android
os
32、把文件中所有大写转成小写
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed 's/\(.*\)/\L\1/' os.txt
windows
linux
android
os
33、检索数字并在数前面添加内容
jun@ubuntu:~$ cat items.txt
HDD 100
Monitor 80
Mouse 10
jun@ubuntu:~$ sed -E 's/([[:digit:]]+)/$\1/g' items.txt
HDD $100
Monitor $80
Mouse $10
34、分组数字,3个数字加个’,’
jun@ubuntu:~$ echo "5098673" | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/ \1,\2/;ta'
5,098,673
35、将\t替换成4空格
jun@ubuntu:~$ echo -e "1\t2\t3" | sed 's/\t/ /g'
1 2 3
jun@ubuntu:~$ echo -e "1\t2\t3" | sed 's/\t/++++/g'
1++++2++++3
36、替换4个连续空格为tab
jun@ubuntu:~$ echo -e "1 2" | sed 's/\s\{4\}/\t/g' | hexdump -c
0000000 1 \t 2 \n
0000004
37、截断所有行为80个字符
jun@ubuntu:~$ cat in.txt
PHP is a server-side scripting language.
PHP is an open-source language and PHP is case-sensitive. PHP is platform-independent.
The following `sed` command will truncate each line of in.txt file into 80 characters.
jun@ubuntu:~$ sed 's/\(^.\{1,80\}\).*/\1/' in.txt
PHP is a server-side scripting language.
PHP is an open-source language and PHP is case-sensitive. PHP is platform-indepe
The following `sed` command will truncate each line of in.txt file into 80 chara
38、检索关键字并附加内容
jun@ubuntu:~$ echo "hello, how are you?" | sed 's/\(hello\)/\1 John/'
hello John, how are you?
39、每行匹配第二次出现的关键字,并在关键字后面附加内容
jun@ubuntu:~$ cat input.txt
PHP is a server-side scripting language.
PHP is an open-source language and PHP is case-sensitive.
PHP is platform-independent.PHP PHP
jun@ubuntu:~$ sed 's/\(PHP\)/\1 (New Text added)/2' input.txt
PHP is a server-side scripting language.
PHP is an open-source language and PHP (New Text added) is case-sensitive.
PHP is platform-independent.PHP (New Text added) PHP
40、把sed命令放入文件执行
jun@ubuntu:~$ cat input.txt
PHP is a server-side scripting language.
PHP is an open-source language and PHP is case-sensitive.
PHP is platform-independent.PHP PHP
jun@ubuntu:~$ vim sedcmd
jun@ubuntu:~$ cat sedcmd
s/PHP/ASP/
s/independent/dependent/
jun@ubuntu:~$ sed -f sedcmd input.txt
ASP is a server-side scripting language.
ASP is an open-source language and PHP is case-sensitive.
ASP is platform-dependent.PHP PHP
41、连续多行匹配,并同时替换成其他字符串
42、调换关键字的顺序
jun@ubuntu:~$ echo "perl python" | sed -e 's/\([^ ]*\) *\([^ ]*\)/\2 \1/'
python perl
43、执行多条sed命令
jun@ubuntu:~$ echo "Ubuntu Centos Debian" | sed -e 's/Ubuntu/Kubuntu/; s/Centos/Fedora/'
Kubuntu Fedora Debian
44、sed命令拼接
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ cat os.txt | sed 's/Linux/Fedora/'| sed 's/windows/Windows 10/i'
Windows 10
Fedora
Android
OS
45、在每行下面插入一个空行
jun@ubuntu:~$ cat stdlist
#ID #Name
[101] -Ali
[102] -Neha
jun@ubuntu:~$ sed G stdlist
#ID #Name
[101] -Ali
[102] -Neha
jun@ubuntu:~$
46、将所有英文数字替换成空格
jun@ubuntu:~$ cat stdlist
#ID #Name
[101] -Ali
[102] -Neha
jun@ubuntu:~$ sed 's/[A-Za-z0-9]/ /g' stdlist
# #
[ ] -
[ ] -
47、使用’&'打印匹配字符串
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed -n 's/^L/Matched String is - &/p' os.txt
Matched String is - Linux
48、每行中交换关键字顺序
jun@ubuntu:~$ cat course.txt
PHP ASP
MySQL Oracle
CodeIgniter Laravel
jun@ubuntu:~$ sed 's/\([^ ]*\)\s*\([^ ]*\)/\2 \1/' course.txt
ASP PHP
Oracle MySQL
Laravel CodeIgniter
49、单词首字母大写
jun@ubuntu:~$ echo "I like bash programming" | sed 's/\([a-z]\)\([a-zA-Z0-9]*\)/\u\1\2/g'
I Like Bash Programming
50、打印文件行号
jun@ubuntu:~$ cat os.txt
Windows
Linux
Android
OS
jun@ubuntu:~$ sed '=' os.txt
1
Windows
2
Linux
3
Android
4
OS