Evaluating filename wildcards

When working with a command, it will not be unusual for you to specify several filenames in the argument list. Rather than requiring you to list each and every filename, the command shell provides the filename wildcard (pathname expansion ) mechanism. With this mechanism, you may specify part of the filename and one or more "wildcards" that symbolically represents a character or set of characters. The shell will look through possible matches in the targeted directory list and fills in any matches.

#(Prep for exercises. As each meta-character is described, try the examples in
the description. )

#( Create a directory, testdir or your choice of names. Change to it. )
mkdir testdir
cd testdir

#(run the following commands to create files )
ls -al > list
ln list list.h
ln list list.h.list
touch ./-l
touch a b b.f bad bad.f1 c c.f ch.01 ch.02 ch.03 ch.0e e.f
mkdir emptydir

#( make sure ls is not defined by an alias )
unalias ls
The filename wild card meta-characters are :

* - the asterisk
The asterisk represents any number of any characters in the filename.

ls list* matches all files beginning with list. This will even match list with no additional characters in its name.

The * by itself expands to the list of all non-hidden names it the directory.

Depending on the shell and shell options, if the shell cannot expand the *, it leaves it as is, resulting in error. In bash, this behavior is controlled by the shell nullglob option.

The nullglob option allows the shell to remove the word containing the unmatched filename wildcard from the command line arguments. The default is off (don't remove). Turning it on will cause ls * to behave like ls without any arguments. They both will list all files in the directory. However, To turn it on use :

shopt -s nullglob
and back off :
shopt -u nullglob

To see a list of all shell options set :
shopt

#( Try the following : )

cd emptydir
shopt -u nullglob

#( The following lists nothing but displays no errors and returns success)
ls
echo $?

#( The following lists . and .. )
ls -a
echo $?

#( The following lists nothing. It displays an error and returns failure)
ls -a *
echo $?

#( Allow null replacement of unmatched words with wildcards )
shopt -s nullglob
ls -a *
echo $?

shopt -u nullglob

#( Change back to parent for later exercises )
cd ..

#( Notice that you get a long listing. )
ls *
In the command above, the filename wildcards are expanded before the command and its options are run. Therefore bash attempts to expand ls -a * but fails. With nullglob unset, it leaves it in the form ls -a * which literally looks for the file * in the empty directory and fails.

With the nullglob set, the unmatched * is replaced with a null character and the command becomes ls -a and the hidden files are listed.

The final ls gives along listing even though you did not specify it. Remember the wild card is expanded before the command is run. If you review the list of filenames, you will notice a -l. When the ls * is expanded, the command line looks something like :

ls a b bad.f1 b.f c c.f ch.01 ch.02 ch.03 e.f emptydir -l list list.h list.h.list
Remember from the command_line module, we noted that whether something is interpreted as an option or an argument is the decision of the command. In this case, when ls does run, it sees the -l as the long listing option and not a filename.

? - the question mark
The question mark represents one character of any value.

ls ? matches all 1 character filenames. ls ch.?? lists all files called ch. with a two character extension after the period. ls ??? lists all 3 character names. In Unix, the period is just another character in a filename. ls ?.? lists all three character filenames with the middle character specifically being a .

[] - the brackets
The brackets represent a single character of a specified list or range.

ls ch.0[1-9] matches only five letter filenames starting with ch.0 and having a digit as the third character.

You may use brackets to provide a specific list. ls ch.0[135] will match ch01, ch03, and ch05 if they exist. If at least one exists, ls will list what is matched and return a success status.

The bash shell also provide a set of symbolic definitions called character classes for common long lists, these are :

[:alnum:] - a-zA-Z0-9
[:alpha:] - a-zA-Z
[:blank:] - the [space] and [tab]
[:cntrl:] - such as [ctrl]c, ascii characters 1-31 and 127
[:digit:] - 0-9
[:graph:] - all printable characters except [space], ascii 33-126
[:lower:] - a-z
[:print:] - all printable characters including [space], ascii 33-126
[:punct:] - all printable characters not [a-zA-Z0-9], ascii 33-64,
            ascii 91-96, or ascii 123-126
[:space:] - [tab],[space],[verttab],[formfeed],[carriagereturn]
[:upper:] - A-Z
[:xdigit:] - 0-9a-fA-F

An example : ls ch.0[[:xdigit:]] matches all five letter filenames beginning with ch.0 and a fifth character in the range 0-9, a-f, or A-F. Notice that the whole character class identifier including the brackets have to be specified inside the wildcard brackets.

The bash shell provides an additional mechanism that may be used when designating filenames, the braces {}. Braces are is not part of the pathname expansion mechanism but are most often used with variables. However they can be used on the command line to perform some unusual expansions.

#( Add two additional files to our exercise directory. ) touch cathouse doghouse

#( Try the following : )

ls {cat,dog}house
echo $?

ls {bird,dog}house
echo $?

The braces allow you to provide a comma delimited list which gets expanded to separate elements on the command line. In the example above, the list was concatenated with another word and that word gets added to each element.

The brace expansion simply performs the string expansion specified. It does no checks against the directory. As a result, the second ls generates an error because, although the file "doghouse" exists, the file "birdhouse" doesn't. And, after expansion, both names were specified as arguments to ls.

If you want to see what the expanded command looks like before being executed, use :

set -x

to turn on this display. You will see the expanded command displayed before it is executed. Be aware, you may see a lot of other things that happen each time you issue a command, so the output may be half a page or more. But occasionally, this feature may help debug an unusual behavior.

To turn the expanded display off, use :

set +x

bash has a variety of shell options that affect the filename expansion. See shopt in the bash man page. For now, you should work with the defaults already configured.