How to add a progress bar to a bash script?

Question

When scripting in bash or any other shell in *NIX, while running a command that will take more than a few seconds, a prgress bar is needed.

For example, copying a big file, opening a big tar file.

What ways do you recommend to add progress bars to bash scripts?

Answer 1

Most unix commands will not give you the sort of direct feedback from which you can do this. Some will give you output on stdout or stderr that you can use.

For something like tar you could use the -v switch and pipe the output to a program that updates a small animation for each line it reads. As tar writes out a list of files it's unravelled the program can update the animation. To do a percent complete you would have to know the number of files and count the lines.

cp doesn't give this sort of output as far as I know. To monitor the progress of cp you would have to monitor the source and destination files and watch the size of the destination. You could write a small c program using the stat (2) system call to get the file size. This would read the size of the source then poll the destination file and update a % complete bar based on the size of the file written to date.

Answer 2

You can implement this by overwriting a line. Use \r to go back to the beginning of the line without writing \n to the terminal.

Write \n when you're done to advance the line.

Use echo -ne to (1) not print \n and (2) to recognize escape sequences line \r.

Here's a demo:

echo -ne '#####                     (33%)\r'
sleep 1
echo -ne '#############             (66%)\r'
sleep 1
echo -ne '#######################   (100%)\r'
echo -ne '\n'

Answer 3

I found this bash script which wraps around cp & tar commands to provide a progress bar: http://www.theiling.de/projects/bar.html

Answer 4

While getting the actual progress information can be tricky, as Nigel Campbell indicated above, the second problem of how to draw the progress bar can be resolved nicely with the dialog command from http://invisible-island.net/dialog/dialog.html.

Answer 5

Some posts have showed how to display the command's progress. In order to calculate it, you'll need to see how much you've progressed. On BSD systems some commands, like dd(1), accept a SIGINFO signal, and will report their progress. On Linux systems some commands will respond similarly to SIGUSR1. If this facility is available, you can pipe your input through dd to monitor the number of bytes processed.

Alternatively, you can use lsof to obtain the offset of the file's read pointer, and thereby calculate the progress. A command like the following could do the trick.

lsof -o0 -o -p $PID |
awk '
            BEGIN { CONVFMT = "%.2f" }
            $4 ~ /^[0-9]+r$/ && $7 ~ /^0t/ {
                    offset = substr($7, 3)
                    fname = $9
                    "stat -f %z '\''" fname "'\''" | getline
                    len = $0
                    print fname, offset / len * 100 "%"
            }
    '

I've posted Linux and FreeBSD shell scripts on my blog.

Answer 6

A simpler method that works on my system using the pipeview ( pv ) utility.

srcdir=$1
outfile=$2


tar -Ocf - $srcdir | pv -i 1 -w 50 -berps `du -bs $srcdir | awk '{print $1}'` | 7za a -si $outfile

Answer 7

You may also be interested in how to do a spinner

Answer 8

GNU tar has a useful option which gives a functionality of a simple progress bar.

(...) Another available checkpoint action is ‘dot’ (or ‘.’). It instructs tar to print a single dot on the standard listing stream, e.g.:

$ tar -c --checkpoint=1000 --checkpoint-action=dot /var
...

The same effect may be obtained by:

$ tar -c --checkpoint=.1000 /var

Answer 9

My solution displays the percentage of the tarball that is currently being uncompressed and written. I use this when writing out 2GB root filesystem images. You really need a progress bar for these things.What I do is use gzip --list and some sed to get the total uncompressed size of the tarball. From that I calculate the blocking-factor needed to divide the file into 100 parts.Finally, I print a checkpoint message for each block. For a 2GB file this gives about 10MB a block. If that is too big then you can divide the BLOCKING_FACTOR by 10 or 100, but then it's harder to print pretty output in terms of a percentage.

Assuming you are using Bash then you can use the following shell function:

untar_progress () 
{ 
    TARBALL=$1;
    BLOCKING_FACTOR=$(($(gzip --list ${TARBALL} | sed -n -e "s/.*[[:space:]]\+[0-9]\+[[:space:]]\+\([0-9]\+\)[[:space:]].*$/\1/p") / 51200 + 1));
    tar --blocking-factor=${BLOCKING_FACTOR} --checkpoint=1 --checkpoint-action='ttyout=Wrote %u%  \r' -zxf ${TARBALL}
}

If you want to use some other shell that doesn't support shell arithmetic expressions then replace the $((...)) expression with a call to expr or dc or bc to calculate the blocking-factor.

-- Noah Spurrier