Handling integer out of range in bash

Question

I have collected vmstat data in a file . It gives details about free, buffer and cache . SInce i'm interested in finding the memory usage , I should do the following computation for each line of vmstat output -- USED=TOTAL - (FREE+BUFFER+CACHE) where TOTAL is the total RAM memory and USED is the instantaneous memory value. TOTAL memory = 4042928 (4 GB)

My code is here

grep -v procs $1 | grep -v free | awk '{USED=4042928-$4-$5-$6;print $USED}' > test.dat
awk: program limit exceeded: maximum number of fields size=32767
        FILENAME="-" FNR=1 NR=1

Answer 1

You should not be printing $USED for a start, the variable in awk is USED:

pax> vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0 1402804 258392 2159316    0    0    54    79  197  479  1  2 93  3

pax> vmstat | egrep -v 'procs|free' | awk '{USED=4042928-$4-$5-$6;print USED}'
222780

What is most likely to be happening in your case is that you're using an awk with that limitation of about 32000 fields per record.

Because your fields 4, 5 and 6 are respectively 25172, 664 and 8520 (from one of your comments), your USED value becomes 4042928-25172-664-8520 or 4008572.

If you tried to print USED, that's what you'd get but, because you're trying to print $USED, it thinks you want $4008572 (field number 4008572) which is just a little bit beyond the 32000 range.

Interestingly, if you have a lot more free memory, you wouldn't get the error but you'd still get an erroneous value :-)

By the way, gawk doesn't have this limitation, it simply prints an empty field (see, for example, section 11.9 here).

Answer 2

you can just do it with one awk command

vmstat | awk 'NR>2{print 4042928-$4-$5-$6 }' file