Instead of flying blind, use the Linux
progress commands to track a command’s progress. These utilities will give you progress bars for commands that don’t normally have any. You’ll see an estimated time until completion, too.
If you are on a long haul flight on an aircraft without video screens in the seatbacks, it isn’t easy to know how far through your journey you are. You know when you took off. You know how long the flight is expected to take. But how do you know if you are on track, on time, or way behind schedule? If you don’t want to watch the in-flight movie you can usually switch your video screen to show a map with the position of your aircraft on it. You also get some statistics, such as an expected time of arrival (ETA), which is great.
Starting a command from the terminal window can sometimes feel like a long-haul flight without a video screen. You have nothing to give any indication if all is well or if the process has hung, nor how close to completion it is. A flashing cursor isn’t very informative.
progress commands give you some statistics and a little visual feedback. You can see how close the process is to complete. That means you get an ETA for your running processes. Compared with staring at a cursor, that wins hands down.
You must install
pv on Ubuntu use this command:
sudo apt-get install pv
pv on Fedora use this command:
sudo dnf install pv
pv on Manjaro use this command:
sudo pacman -Syu pv
pv stands for pipe viewer. Piping has to be involved in the command somewhere. Here’s an example where we’re piping an ISO image through
zip to make a compressed zip file of the ISO.
To slow the commands down enough that a screenshot could be taken, some of the files in the examples used for this article were stored on an old, slow, external USB called SILVERXHD.
pv /media/dave/SILVERXHD/gparted-live-1.0.0-1-amd64.iso | zip > gparted.zip
pv gives us can be seen in the bottom line of the display.
From left to right, the information that is displayed is:
- The data transferred so far.
- The time elapsed fo far.
- The data transfer rate (throughput).
- A progress bar and a percentage completed figure.
- The estimated time left before completion (ETA).
Copying a File with pv
To copy a file with output from
pv, use this command:
pv /media/dave/SILVERXHD/gparted-live-1.0.0-1-amd64.iso > gparted.iso
We get a progress report as the file is copied.
Copying Multiple Files with pv
To copy multiple files and folders with
pv we need to use a little trick. We use
tar to move the files for us.
tar -c help-files/ | pv | tar -x -C Documents/
tar -c help-files/ portion of the command instructs
tar to create (
-c) an archive of the files in the help-files folder. This is piped through
pv so that we get a display of the progress. It is then piped back into
tar for the last portion of the command. The archive is extracted (
-x) and the directory is changed (
-C) to Documents before the extraction.
So, the files and folders that are in help-files are copied to the Documents folder, with a progress display.
The output is slightly different this time.
We do not get an ETA. The progress bar now displays a moving indicator. It shows that the process is active, but it doesn’t grow from left to right like a traditional progress bar.
pv is limited to displaying the information it can extract from the process that is being piped.
Using pv and tar to Create an Archive
Copying files with
tar does not leave us with an archive file. A sort of “virtual” archive is created by
tar, which is fed straight back into
tar to extract the files. If our objective is to copy files, that is accomplished. But what if we want to create an archive file?
We can still use
tar to create an archive file and get a progress report from
pv. The options used with
-c (create archive),
-z (compress with gzip) and
-f (filename of the archive).
Note that we’re using
- as the filename, which causes
tar to use stdout, and to write its output to the terminal window. We don’t see that output because it is piped through
The actual name of the archive is going to be the filename that we pipe the output from
pv into. In this case, it is “help-files.tgz”.
tar -czf - ./help-files/ | pv > help-files.tgz
We get the same progress indicators as before, and the archive file is created for us.
The pv Display Options
There are a number of options you can use with
pv to change the details of its report.
If you use any of these options, all of the other options are turned off. So if you want to have three of the display options in use, then you need to specify those three options.
pv without any options is the same as using the
- -p: display the percentage complete. This is the progress bar and the percentage completed figure.
- -t: display the elapsed time.
- -e: display the ETA.
- -r: display the rate of data transfer.
- -b: display the byte count (data transferred so far).
- -n: display the percentage as an integer. This prints the percentage completed as an integer figure, with each new update on a new line.
Let’s repeat the last command and pass the
-p (percentage completed) option to
tar -czf - ./help-files/ | pv - p > help-files.tgz
This turns off all of the other display options.
pv provides the percentage completed element only.
pv doesn’t get a percentage completed figure from
tar, the progress bar is replaced with a moving indicator. There is no percentage figure.
Using pv With wc
We can use
pv to pipe a text file (or files) into
wc will then count the carriage returns, characters, and words and
pv will give us a progress report.
Here we are piping all of the “.page” files in the help-files directory into
wc completes we can see our count of carriage returns (lines), characters and words from all of the “.page” files in the help-files folder.
Installing the progress Command
progress command gives the same sort of useful information as
pv, but it works with a specific set of Linux commands.
progress in Ubuntu, use this command:
sudo apt-get install progress
progress in Fedora, use this command:
sudo dnf install progress
progress in Manjaro, use this command:
sudo pacman -Syu progress
The Commands progress Works With
progress in a terminal window and pressing Enter will give you a list of the commands that
progress works with.
Using Progress With Pipes
There are two techniques we can use to monitor commands with
progress. The first is to use pipes.
tar command is in the list of supported commands that
progress can monitor, so let’s use
The options we’ll use are the standard
-c (create archive),
-z (compress with gzip) and
-f (filename) options. We’re going to create a compressed archive of everything in the help-files folder, and the archive will be named “help.tgz”.
We’re piping that into
progress and using the
-m (monitor) option so
progress keeps reporting on the process until it has completed.
tar -czf help.tgz ./help-files/ | progress -m
The terminal window will show the progress of the
tar command as it creates the archive.
As each file is processed, it is listed, with the following information:
- The process ID.
- The process name.
- Percentage completed.
- Data processed and total size of the file.
- Data rate (throughput).
- Estimated time remaining (ETA).
You might be surprised to see a second data set appear. This first data set is for
tar. The second one is for
gzip to perform the compression. Because
gzip is in the list of supported commands,
progress reports on it.
Using Progress in Continual Monitor Mode
You can use
progress in a real-time continual monitor mode by using the -M (monitor) option.
Type the following command in a terminal window:
progress will report that there are no commands running for it to monitor. But you are not returned to the command line.
progress waits until a command that it can monitor starts. It will then automatically start reporting on it.
In another terminal window, type a command that is in the list of commands that progress can monitor.
We’re going to use
cat. Commands that are over too quickly won’t register with
progress, so we’ll list the contents of a very long text file.
In the terminal window with
progress in it, you’ll see statistics for the
cat command as it executes and works towards completion.
cat finishes listing the file
progress returns to its waiting state.
Each time one of the commands it can report on performs a sizeable task,
progress will automatically monitor it and report on it.
That’s pretty neat.
Take the guesswork out of wondering how a long-running command is doing, and take a break from contemplating your cursor with
|Files||tar · pv · cat · tac · chmod · grep · diff · sed · ar · man · pushd · popd · fsck · testdisk · seq · fd · pandoc · cd · $PATH · awk · join · jq · fold · uniq · journalctl · tail · stat · ls · fstab · echo · less · chgrp · chown · rev · look · strings · type · rename · zip · unzip · mount · umount · install · fdisk · mkfs · rm · rmdir · rsync · df · gpg · vi · nano · mkdir · du · ln · patch · convert · rclone · shred · srm|
|Processes||alias · screen · top · nice · renice · progress · strace · systemd · tmux · chsh · history · at · batch · free · which · dmesg · chfn · usermod · ps · chroot · xargs · tty · pinky · lsof · vmstat · timeout · wall · yes · kill · sleep · sudo · su · time · groupadd · usermod · groups · lshw · shutdown · reboot · halt · poweroff · passwd · lscpu · crontab · date · bg · fg|
|Networking||netstat · ping · traceroute · ip · ss · whois · fail2ban · bmon · dig · finger · nmap · ftp · curl · wget · who · whoami · w · iptables · ssh-keygen · ufw|