Detect number of IDLE processors ruby

Question

Hello, I work on shared linux machines with between 4 and 24 cores. To make best use of them, I use the following code to detect the number of processors from my ruby scripts:

return `cat /proc/cpuinfo | grep processor | wc -l`.to_i

(perhaps there is a pure-ruby way of doing this?)

But sometimes a colleague is using six or eight of the 24 cores. (as seen via top). How can I get an estimate of the number of currently unused processors that I can use without making anyone upset?

Thanks!

Answer 1

You can use the data in the /proc filesystem to get CPU affinity info for running processes. The following should give you the number of CPUs currently in use (Note: I don't have a Linux or Ruby box handy so this code is untested, but you can get the idea):

def processors_in_use
    procs=[]
    Dir.glob("/proc/*/stat") {|filename|
        next if File.directory?(filename)
        this_proc=[]
        File.open(filename) {|file| this_proc = file.gets.split.values_at(2,38)}
        procs << this_proc[1].to_i if this_proc[0]=="R"
    }
    procs.uniq.length
end

def num_processors
    IO.readlines("/proc/cpuinfo").delete_if{|x| x.index("processor")==nil}.length
end

def num_free_processors
    num_processors - processors_in_use
end

def estimate_free_cpus(count, waittime)
    results=[]
    count.times {
        results << num_free_processors
        sleep(waittime)
    }
    sum=0
    results.each {|x| sum += x}
    (sum.to_f / results.length).round
end

Edit: I verified that the above code works (I was using Ruby 1.9)

Answer 2

inspired by bta's reply, this is what i'm using:

private
def YWSystemTools.numberOfActiveProcessors # internal
    processorForProcs = []
    processFiles      = Dir.glob("/proc/*/stat")
    raise IOError, 'Cannot find /proc/*/stat files. Are you sure this is a linux machine?' if processFiles.empty?

    processFiles.each do |filename|
        next if  File.directory?(filename)  # because /proc/net/stat is a directory
        next if !File.exists?(filename) # may have disappeared in the meantime
        this_proc = []
        File.open(filename) {  |file| this_proc = file.gets.split.values_at(2,38) }
        processorForProcs << this_proc[1].to_i if this_proc[0]=="R"
    end
    processorsInUse = processorForProcs.uniq
    return(processorsInUse.length)
end
public

def YWSystemTools.numberOfAvailableProcessors
    numberOfAttempts = 5
    $log.info("Will determine number of available processors. Wait #{numberOfAttempts.to_s} seconds.")
    #we estimate 5 times because of local fluctuations in procesor use. Keep minimum.
    estimationsOfNumberOfActiveProcessors = []
    numberOfAttempts.times do 
        estimationsOfNumberOfActiveProcessors << YWSystemTools.numberOfActiveProcessors
        sleep(1)
    end
    numberOfActiveProcessors  = estimationsOfNumberOfActiveProcessors.min
    numberOfTotalProcessors   = number_of_processors()

    raise IOError, '!! # active Processors > # processors' if numberOfActiveProcessors > numberOfTotalProcessors

    numberOfAvailableProcessors = numberOfTotalProcessors - numberOfActiveProcessors
    $log.info("#{numberOfAvailableProcessors} out of #{numberOfTotalProcessors} are available!")        
    return(numberOfAvailableProcessors)
end