Python: Reliably monitor current CPU usage

Question

I'd like to monitor the current system-wide CPU usage on a Mac with Python.

I've written some code that starts 'ps', and adds up all the values from the '%cpu' column.

def psColumn(colName):
"""Get a column of ps output as a list"""
ps = subprocess.Popen(["ps", "-A", "-o", colName], stdout=subprocess.PIPE)
(stdout, stderr) = ps.communicate()
column = stdout.split("\n")[1:]
column = [token.strip() for token in column if token != '']
return column

def read(self):
values = map(float, psColumn("%cpu"))
return sum(values)

However, I always get high readings from 50% - 80%, probably caused by the measuring program itself. This CPU usage peak does not register on my MenuMeters or other system monitoring programs. How can I get readings which are more like what MenuMeters would display? (I want to detect critical situations in which some program is hogging the CPU.)

p.s. I have tried psutil, but

psutil.cpu_percent()

always returns 100%, so either it's useless for me or I am using it wrongly.

Answer 1

For detecting critical situations when some program is hogging the CPU perhaps looking at load average is better? Take a look at "uptime" command.

Load average number tells you how many processes on average are using or waiting on CPU for execution. If it is close to or over 1.0, it means the system is constantly busy with something. If load average is constantly raising, it means that system cannot keep up with the demand and tasks start to pile up. Monitoring load average instead of CPU utilization for system "health" has two advantages:

• Load averages given by system are already averaged. They don't fluctuate that much so you don't get that problem with parsing "ps" output.
• Some app may be thrashing disk and rendering system unresponsive. In this case CPU utilization might be low, but load average would still be high indicating a problem.

Also monitoring free RAM and swap would be a good idea.

Answer 2

I was working on basically the same thing just a couple of weeks ago and I too was having trouble with psutil.cpu_percent().

Instead I used psutil.cpu_times() which gives cpu time used for user, system, idle and other things depending on your OS. I dont know if this is a good way or even an accurate way of doing things.

import psutil as ps

class cpu_percent:
    '''Keep track of cpu usage.'''

    def __init__(self):
        self.last = ps.cpu_times()

    def update(self):
        '''CPU usage is specific CPU time passed divided by total CPU time passed.'''

        last = self.last
        current = ps.cpu_times()

        total_time_passed = sum([current.__dict__.get(key, 0) - last.__dict__.get(key, 0) for key in current.attrs])

        #only keeping track of system and user time
        sys_time = current.system - last.system
        usr_time = current.user - last.user

        self.last = current

        if total_time_passed > 0:
            sys_percent = 100 * sys_time / total_time_passed
            usr_percent = 100 * usr_time / total_time_passed
            return sys_percent + usr_percent
        else:
            return 0

Answer 3

I also had to come up with a method for this once, here is what I ended up with:

import time

INTERVAL = 1

def getTimeList():
    statFile = file("/proc/stat", "r")
    timeList = statFile.readline().split(" ")[2:6]
    statFile.close()
    for i in range(len(timeList))  :
        timeList[i] = int(timeList[i])
    return timeList

def deltaTime(interval):
    x = getTimeList()
    time.sleep(interval)
    y = getTimeList()
    for i in range(len(x)):
        y[i] -= x[i]
    return y

def getCpuLoad():
    dt = deltaTime(INTERVAL)
    cpuPct = 100 - (dt[len(dt) - 1] * 100.00 / sum(dt))
    return cpuPct

Answer 4

>>> import psutil, time
>>> print psutil.cpu_times()
softirq=50.87; iowait=39.63; system=1130.67; idle=164171.41; user=965.15; irq=7.08; nice=0.0
>>>
>>> while 1:
...     print round(psutil.cpu_percent(), 1)
...     time.sleep(1)
...
5.4
3.2
7.3
7.1
2.5