Python drawing cumulative plot (matplotlib)

Question

Hi,

I have not used matplotlib, but looks like it is main library for drawing plots. I want to draw CPU usage plot. I have background processes each minute making record (date, min_load, avg_load, max_load). date could be timestamp or nice formatted date.

I want to draw diagram which show min_load, avg_load and max_load on the same plot. On X axis I would like to put minutes, hours, days, week depending on how much data there is.

There are possible gaps. Let's say monitored process crashes and because no one restarts it there might be gaps for several hours.

Example of how I imagine it: http://img714.imageshack.us/img714/2074/infoplot1.png This does not illustrate gaps, but in this situation on readings go to 0.

I am playing with matplotlib right now I will try sharing my results too. This is how data might look like:

1254152292;0.07;0.08;0.13
1254152352;0.04;0.05;0.10
1254152412;0.09;0.10;0.17
1254152472;0.28;0.29;0.30
1254152532;0.20;0.20;0.21
1254152592;0.09;0.12;0.15
1254152652;0.09;0.12;0.14
1254152923;0.13;0.12;0.30
1254152983;0.13;0.25;0.32

Or it could look something like this:
Wed Oct 06 08:03:55 CEST 2010;0.25;0.30;0.35
Wed Oct 06 08:03:56 CEST 2010;0.00;0.01;0.02
Wed Oct 06 08:03:57 CEST 2010;0.00;0.01;0.02
Wed Oct 06 08:03:58 CEST 2010;0.00;0.01;0.02
Wed Oct 06 08:03:59 CEST 2010;0.00;0.01;0.02
Wed Oct 06 08:04:00 CEST 2010;0.00;0.01;0.02
Wed Oct 06 08:04:01 CEST 2010;0.25;0.50;0,75
Wed Oct 06 08:04:02 CEST 2010;0.00;0.01;0.02

-david

Answer 1

Try:

from matplotlib.dates import strpdate2num, epoch2num
import numpy as np
from pylab import figure, show, cm

datefmt = "%a %b %d %H:%M:%S CEST %Y"
datafile = "cpu.dat"

def parsedate(x):
    global datefmt
    try:
        res = epoch2num( int(x) )
    except:
        try:
            res = strpdate2num(datefmt)(x)
        except:
            print("Cannot parse date ('"+x+"')")
            exit(1)
    return res

# parse data file
t,a,b,c = np.loadtxt(
    datafile, delimiter=';',
    converters={0:parsedate},
    unpack=True)

fig = figure()
ax = fig.add_axes((0.1,0.1,0.7,0.85))
# limit y axis to 0
ax.set_ylim(0);

# colors
colors=['b','g','r']
fill=[(0.5,0.5,1), (0.5,1,0.5), (1,0.5,0.5)]

# plot
for x in [c,b,a]:
    ax.plot_date(t, x, '-', lw=2, color=colors.pop())
    ax.fill_between(t, x, color=fill.pop())

# legend
ax.legend(['max','avg','min'], loc=(1.03,0.4), frameon=False)

fig.autofmt_xdate()
show()

This parses the lines from "cpu.dat" file. Date is parsed by parsedate function.

Matplotlib should find the best format for the x axis.

Edit: Added legend and fill_between (maybe there is better way to do this).

Answer 2

Don't mind my "I am middle of the work, but fast test:".

Would it possible to to mix plot_date with fill/fill_between?

Answer 3

How to handle data when you have gaps? Should I scan data and make dumpy values? Let's say I am missing 10 hours of log and I would like plot go be at 0 at that moment, not to show some ghost values. Maybe matplotlib has something for it?