Writing a kernel mode profiler for processes in python.

Question

I would like seek some guidance in writing a "process profiler" which runs in kernel mode. I am asking for a kernel mode profiler is because I run loads of applications and I do not want my profiler to be swapped out.

When I said "process profiler" I mean to something that would monitor resource usage by the process. including usage of threads and their statistics.

And I wish to write this in python. Point me to some modules or helpful resource.

Please provide me guidance/suggestion for doing it.

Thanks,

Edit::: Would like to add that currently my interest isto write only for linux. however after i built it i will have to support windows.

Answer 1

It's going to be very difficult to do the process monitoring part in Python, since the python interpreter doesn't run in the kernel.

I suspect there are two easy approaches to this:

1. use the /proc filesystem if you have one (you don't mention your OS)
2. Use dtrace if you have dtrace (again, without the OS, who knows.)

---

Okay, following up after the edit.

First, there's no way you're going to be able to write code that runs in the kernel, in python, and is portable between Linux and Windows. Or at least if you were to, it would be a hack that would live in glory forever.

That said, though, if your purpose is to process Python, there are a lot of Python tools available to get information from the Python interpreter at run time.

If instead your desire is to get process information from other processes in general, you're going to need to examine the options available to you in the various OS APIs. Linux has a /proc filesystem; that's a useful start. I suspect Windows has similar APIs, but I don't know them.

If you have to write kernel code, you'll almost certainly need to write it in C or C++.

Answer 2

don't try and get python running in kernel space!

You would be much better using an existing tool and getting it to spit out XML that can be sucked into Python. I wouldn't want to port the Python interpreter to kernel-mode (it sounds grim writing it).

The /proc option does sound good.

some code code that reads proc information to determine memory usage and such. Should get you going:

http://www.pixelbeat.org/scripts/ps_mem.py reads memory information of processes using Python through /proc/smaps like charlie suggested.

Answer 3

have you looked at PSI? (http://www.psychofx.com/psi/)

"PSI is a Python module providing direct access to real-time system and process information. PSI is a Python C extension, providing the most efficient access to system information directly from system calls."

it might give you what you are looking for. .... or at least a starting point.

Answer 4

Some of your comments on other answers suggest that you are a relatively inexperienced programmer. Therefore I would strongly suggest that you stay away from kernel programming, as it is very hard even for experienced programmers.

Why would you want to write something that

• is a very complex system (just look at existing profiling infrastructures and how complex they are)
• can not be done in python (I don't know any kernel that would allow execution of python in kernel mode)
• already exists (oprofile on Linux)