How-To: Bash Parameter Expansion and String Manipulation

3 minute read

Last time we saw how bash can help us in handling default values out of the box using parameter expansion. This time we will see how basic string operations (nonetheless common and useful) can also be achieved using bash.

There is many ways to do string manipulation with bash, like finding a filename extension using expr, separating the directory part from a filename using dirname and basename…. or even more sophisticated ones based on regex, sed….

Why using a sledgehammer to crack a nut when you could use bash builtin functionalities!

Bash built-ins expansion can be used to:

  • Get substrings
  • Substitute part of a string
  • Case modification
  • Removing prefix and suffix of a string

Let see how we can do that!

Substrings

Bash substring works rather like any substring function available in scripting languages, e.g. We provide a string and give an offset… and optionally a length.

the form to use is:

${parameter:offset}
or
${parameter:offset:length}

This will expand to up to length character of parameter starting from character positioned at offset. If length is omitted, if will expand till the end of parameter.

If offset is negative, the value is used as an offset starting from the end of parameter

$ str="this is a string with spaces and words"
$ echo ${str:10}
string with spaces and words
$ echo ${str:10:6}
string
$ echo ${str: -5}
words
$ echo ${str:(-5)}
words

Mind that when using a negative offset, you should either leave a whitespace after the colon : or add parenthesis around the number. If you do not do so, bash will believe you are using the form ${parameter:-word} !

Nice, let now see how we can substitute part of a string.

String substitution

To substitute a pattern in a string, we will use the form:

${parameter/pattern/string}

If pattern begins with /, all matches of pattern will be replaced, if pattern begins with % it must match at the end, if pattern begins with # if must match at the beginning.

If string is null, matches of pattern are deleted.

$ str="this is a string with spaces and words"
$ echo ${str/i/o}
thos is a string with spaces and words
$ echo ${str//i/o}
thos os a strong woth spaces and words
$ echo ${str/#t/i}
ihis is a string with spaces and words
$ echo ${str/%s/i}
this is a string with spaces and wordi
$ echo ${str/i}
ths is a string with spaces and words
$ echo ${str//i}
ths s a strng wth spaces and words

sweet, lets move on to the case modification

Case Modification

Case modification is done by using the following forms:

${parameter^pattern}
or
${parameter^^pattern}
or
${parameter,pattern}
or
${parameter,,pattern}

The versions using ^ will be used to change the letter to uppercase and version using , change the letter to lowercase.

When we use the form ^^ or ,,, all letter of the string will be modified.

Here are some examples:</em>

str="this is a string with spaces and words"
$ echo ${str^}
This is a string with spaces and words
$ echo ${str^^}
THIS IS A STRING WITH SPACES AND WORDS
$ str2="THIS IS A STRING WITH SPACES AND WORDS"
$ echo ${str2,}
tHIS IS A STRING WITH SPACES AND WORDS
$ echo ${str2,,}
this is a string with spaces and words
$ echo ${str2,T}
tHIS IS A STRING WITH SPACES AND WORDS
$ echo ${str2,,T}
tHIS IS A StRING WItH SPACES AND WORDS
$ echo ${str2,,[AEIOU]}
THiS iS a STRiNG WiTH SPaCeS aND WoRDS

Last but not least, let see how we can remove prefixes and suffixes

Removing Prefix and Suffix

To remove prefixes, we will use the form:

${parameter#word}
or
${parameter##word}

For suffixes, it will be the form:

${parameter%word}
or
${parameter%%word}

In both cases, when we use only 1 #</m> or % we will remove the shortest matching pattern from the beginning or the end, respectively.

When using ## or %%, we will remove the longest matching pattern.

Let see with a couple of examples how this works:

$ file="/a/b/c/d/e/myfile.txt"
$ echo ${file#*.}
txt
$ echo ${file#*/}
a/b/c/d/e/myfile.txt
$ echo ${file##*/}
myfile.txt
$ echo ${file%/*}
/a/b/c/d/e
$ echo ${file%%/*}

$ echo ${file%%.*}
/a/b/c/d/e/myfile

Now, you could be using the form ${file##*/} to find the filename given a path, and ${file%/*} to find th directory name of that same path.

That’s it for this tutorial, I wish this will give you some useful tips and will help you working with string manipulation in your bash scripts to come.