a reliable way to get the progress of an ffmpeg conversion in BASH

Question

Hi

I've done a bit of searching on here and on other sites, it seems this question has been asked a few times, but rather than giving the actual code, people only give the theory.

I wish to create a reliable way to get the progress of a transcode. It seems the best way is to use the total frames of the source file and then get the current frame that ffmpeg is on. As people point out, because ffmpeg bizarrely outputs its progress using Carriage Returns (/r or ^M) and because there is sometimes a space between the output and sometimes not, this can be unreliable at best. Here is a sample of the output:

frame=73834 fps=397 q=0.0 size=      -0kB time=2462.14 bitrate=  -0.0kbits/s frame=74028 fps=397 q=0.0 size=      -0kB time=2468.64 bitrate=  -0.0kbits/s frame=74133 fps=397 q=0.0 Lsize=      -0kB time=2472.06 bitrate=  -0.0kbits/

I have written function that is called whilst the ffmpeg conversion is going ahead. Here is what I have got so far:

First, to get the total frames of the source file:

duration=( $(ffmpeg -i "$SourceFile" 2>&1 | sed -n "s/.* Duration: \([^,]*\), start: .*/\1/p") )
fps=( $(ffmpeg -i "$SourceFile" 2>&1 | sed -n "s/.*, \(.*\) tbr.*/\1/p") )
hours=( $(echo $duration | cut -d":" -f1) )
minutes=( $(echo $duration | cut -d":" -f2) )
seconds=( $(echo $duration | cut -d":" -f3) )
# Get the integer part with cut
frames=( $(echo "($hours*3600+$minutes*60+$seconds)*$fps" | bc | cut -d"." -f1) )
if [ -z $frames ]; then
    zenity --info --title "$WindowTitle" --text "Can't calculate frames, sorry."
    exit 
echo ""$SourceFile" has $frames frames, now converting" > $ffmpeglog
echo ""$SourceFile" has $frames frames, now converting"

Then this is the progress function I call during the conversion:

progress() {
sleep 10
#some shenanigans due to the way ffmpeg uses carriage returns
cat -v $ffmpeglog | tr '^M' '\n' > $ffmpeglog1
#calculate percentage progress based on frames
cframe=( $(tac $ffmpeglog1 | grep -m 1 frame= | awk '{print $1}' | cut -c 7-) )
if [ -z $cframe ]; then
    cframe=( $(tac $ffmpeglog1 | grep -m 1 frame= | awk '{print $2}') )
fi
if is_integer $frame; then
    percent=$((100 * cframe / frames))
    #calculate time left and taken etc
    fps=( $(tac $ffmpeglog1 | grep -m 1 frame= | awk '{print $3}') )
    if [ "$fps" = "fps=" ]; then
        fps=( $(tac $ffmpeglog1 | grep -m 1 frame= | awk '{print $4}') )
    fi
    total=$(( frames + cframe + percent + fps ))
    #simple check to ensure all values are numbers
    if is_integer $total; then
        #all ok continue
        if [ -z $fps ]; then
            echo -ne "\rffmpeg: $cframe of $frames frames, progress: $percent"%" and ETA: error fps:0"
        else
            if [ -z $cframe ]; then
                echo -ne "\rffmpeg: total $frames frames, error cframes:0"
            else
                remaining=$(( frames - cframe ))
                seconds=$(( remaining / fps ))
                h=$(( seconds / 3600 ))
                m=$(( ( seconds / 60 ) % 60 ))
                s=$(( seconds % 60 ))
                echo -ne "\rffmpeg: $cframe of $frames frames, progress: $percent"%" and ETA: "$h"h "$m"m "$s"s"
            fi
        fi
    else
        echo "Error, one of the values wasn't a number, trying again in 10s."
    fi
else
    echo "Error, frames is 0, progress wont work, sorry."

fi
}

And here, for completeness, is that is_integer function:

is_integer() {
    s=$(echo $1 | tr -d 0-9)
    if [ -z "$s" ]; then
        return 0
    else
        return 1
    fi

}

As you can see, my approach is rather crass, in that I am writing one log file to another and then processing that 2nd log file, to try to deal with the carriage returns. The main problem I have found is that no command that BASH can call seems to be able to cut ffmpegs output to just the last line, due to the frankly infuriating use of carriage returns. My awk skills aren't very good and the progress function fails when there is no space between the frame=xxxxxx part, as is sometimes the case. I was wondering if any BASH headz can make my code a little more error resilient and reliable.

Answer 1

The reason for the carriage returns, as I'm sure you're aware, is so the progress lines overwrite each other (the output stays on one line since there's no linefeed).

You can try this method for extracting the string after the last carriage return:

$ sample=$'cherry\rbanana\rapple'
$ echo "$sample"
applea
$ cr=$'\r'                    # A
$ extract="${sample##*$cr}"   # B
$ echo "$extract"
apple

The lines marked "A" and "B" are the essentials, the rest are just for demonstration. Using this method, you may be able to eliminate a few of the gyrations you're having to use now.