Home » Shell Scripting » BASH » A Sed (Stream Editor) Example

About Andreas Pomarolli

Andreas Pomarolli
Andreas has graduated from Computer Science and Bioinformatics at the University of Linz. During his studies he has been involved with a large number of research projects ranging from software engineering to data engineering and at least web engineering. His scientific focus includes the areas of software engineering, data engineering, web engineering and project management. He currently works as a software engineer in the IT sector where she is mainly involved with projects based on Java, Databases and Web Technologies.

A Sed (Stream Editor) Example

This is a Sed (Stream Editor) example. A Stream Editor is used to perform basic transformations on text read from a file or a pipe. The result is sent to standard output. The syntax for the sed command has no output file specification, but results can be saved to a file using output redirection. The editor does not modify the original input.

Want to master BASH scripting?

Subscribe to our newsletter and download the BASH Programming Cookbook right now!

In order to help you start using BASH scripting, we have compiled a kick-ass guide with all the major commands and use cases! Besides studying them online you may download the eBook in PDF format!

 

What distinguishes sed from other editors, such as vi and ed, is its ability to filter text that it gets from a pipeline feed. You do not need to interact with the editor while it is running; that is why sed is sometimes called a batch editor. This feature allows use of editing commands in scripts, greatly easing repetitive editing tasks. When facing replacement of text in a large number of files, sed is a great help.

The following table shows an overview of the whole article:

1. SED Commands

The sed program can perform text pattern substitutions and deletions using regular expressions, like the ones used with the grep command. The editing commands are similar to the ones used in the vi editor.

1.1 SED editing commands

CommandResult
a\Append text below current line.
c\Change text in the current line with new text.
dDelete text.
i\Insert text above current line.
pPrint text.
rRead a file.
sSearch and replace text.
wWrite to a file.

1.2 Sed options

OptionEffect
-e SCRIPTAdd the commands in SCRIPT to the set of commands to be run while processing the input.
-fAdd the commands contained in the file SCRIPT-FILE to the set of commands to be run while processing the input.
-nSilent mode.
-VPrint version information and exit.

2. Interactive editing

2.1 Printing lines containing a pattern

This is something you can do with grep, of course, but you can’t do a “find and replace” using that command. This is just to get you started.

This is our example text file:

Test.txt

We shall not spend a large expense of time
Before we reckon with your several loves,
And make us even with you. My thanes and kinsmen,
Henceforth be earls, the first that ever Scotland
In such an honour named. What's more to do,
Which would be planted newly with the time,
As calling home our exiled friends abroad
That fled the snares of watchful tyranny;
Producing forth the cruel ministers
Of this dead butcher and his fiend-like queen,
Who, as 'tis thought, by self and violent hands
Took off her life; this, and what needful else
That calls upon us, by the grace of Grace,
We will perform in measure, time and place:
So, thanks to all at once and to each one,
Whom we invite to see us crown'd at Scone.
Macbeth, William Shakespeare

We want sed to find all the lines containing our search pattern, in this case “erors”. We use the p to obtain the result:

A SED Search Pattern Example.

A SED Search Pattern Example.

As you notice, sed prints the entire file, but the lines containing the search string are printed twice. This is not what we want. In order to only print those lines matching our pattern, use the -n option:

A SED Search Pattern Example without duplicates.

A SED Search Pattern Example without duplicates.

2.2 Deleting lines of input containing a pattern

We use the same example text file. Now we only want to see the lines not containing the search string:

A SED Search Pattern Example without lines containing "you".

A SED Search Pattern Example without lines containing “you”.

The d command results in excluding lines from being displayed. Matching lines starting with a given pattern and ending in a second pattern are showed like this:

A SED Search Pattern Example with a given pattern and ending in a second pattern.

A SED Search Pattern Example with a given pattern and ending in a second pattern.

Note that the last dot needs to be escaped in order to actually match. In our example the expression just matches any character, including the last dot.

2.3 Ranges of lines

This time we want to take out the lines containing the errors. In the example these are lines 2 to 4. Specify this range to address, together with the d command:

A SED range of line Example.

A SED range of line Example.

To print the file starting from a certain line until the end of the file, use a command similar to this:

Another SED range of line Example.

Another SED range of line Example.

This only prints the first two lines of the example file. The following command prints the first line containing the pattern “a text”, up to and including the next line
containing the pattern “a line”:

A SED range of line Example with two patterns.

A SED range of line Example with two patterns.

2.4 Find and replace with sed

In the example file, we will now search and replace the errors instead of only (de)selecting the lines containing the search string.

A simple SED Find and Replace Example.

A simple SED Find and Replace Example.

To insert a string at the beginning of each line of a file, for instance for quoting:

A SED String insertion Example.

A SED String insertion Example.

Insert some string at the end of each line:

Another SED String insertion Example.

Another SED String insertion Example.

Multiple find and replace commands are separated with individual -e options:

A Sed Example with multiple find and replace commands.

A Sed Example with multiple find and replace commands.

Keep in mind that by default sed prints its results to the standard output, most likely your terminal window. If
you want to save the output to a file, redirect it:

sed option 'some/expression' file_to_process > sed_output_in_a_file

3. Non-interactive editing

3.1 Reading sed commands from a file

Multiple sed commands can be put in a file and executed using the -f option. When creating such a file,
make sure that:

  • No trailing white spaces exist at the end of lines.
  • No quotes are used.
  • When entering text to add or replace, all except the last line end in a backslash.

3.2 Writing output files

Writing output is done using the output redirection operator >. This is an example script used to create very simple HTML files from plain text files.

script.sed

1i\
<html>\
<head><title>generated html by sed</title></head>\
<body bgcolor="#ffffff">\
<pre>
$a\
</pre>\
</body>\
</html>

txt2html.sh

#!/bin/bash
# This is a simple script that you can use for converting text into HTML.
# First we take out all newline characters, so that the appending only happens
# once, then we replace the newlines.
echo "converting $1..."
SCRIPT="script.sed"
NAME="$1"
RESULT="$2"
TEMPFILE="sed.$PID.tmp"
sed "s/\n/^M/" $NAME | sed -f $SCRIPT | sed "s/^M/\n/" > $TEMPFILE
mv $TEMPFILE $RESULT
echo "done."

$1 holds the first argument to a given command, in this case the name of the file to convert:

input.txt

This
is
a
sed
generated
html
file

After transforming we get the following output:

output.txt

<html>
<head><title>generated html by sed</title></head>
<body bgcolor="#ffffff">
<pre>
This
is
a
sed
generated
html
file
</pre>
</body>
</html>

This is not really how it is done; this example just demonstrates sed capabilities.

4. Summary

The sed stream editor is a powerful command line tool, which can handle streams of data: it can take input lines from a pipe. This makes it fit for non-interactive use. The sed editor uses vi-like commands and accepts regular expressions.

The sed tool can read commands from the command line or from a script. It is often used to perform find-and-replace actions on lines containing a pattern.

Do you want to know how to develop your skillset to become a sysadmin Rockstar?

Subscribe to our newsletter to start Rocking right now!

To get you started we give you our best selling eBooks for FREE!

 

1. Introduction to NGINX

2. Apache HTTP Server Cookbook

3. VirtualBox Essentials

4. Nagios Monitoring Cookbook

5. Linux BASH Programming Cookbook

6. Postgresql Database Tutorial

 

and many more ....

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Want to take your sysadmin skills to the next level?

Grab our programming books for FREE!

Here are some of the eBooks you will get:

  • Linux BASH Programming Cookbook
  • NAGIOS Monitoring Cookbook
  • Introduction to NGINX
  • Apache HTTP Server Cookbook
  • VirtualBox Essentials
  • PostgreSQL Database Handbook