How do manage the limit of executions to be done by hour (Max: 1000 requests per hour) without a database

Question

I am currently developing a script in PHP to fetch webpages. The fact is that by doing so, I occasionally do too much requests to a particular website. In order to control any overflow, I would like to keep trace of how many requests have been done in the last hour or so for each domain. It doesn't need to be perfect, just a good estimate.

I doesn't have access to a database, except sqlite2. I would really like something really simple because there will typically be a lot of updates, which is kind of heavy for a sqlite database. If no one has a magical solution, I'll go for sqlite, but I was curious what you can come up with

Thank you very much

Answer 1

I found an answer to my dilemma, so I thought I'd share:

Assume $file and $max are set and wrap with a bit of error handling. Basically, it uses the fact that a timestamp has 10 bytes (+1 with newline). Every now and then, something calls clean() that eliminates timestamps older than 3600 seconds

function check() {
    if (file_exists($file))
        return (filesize($file) / 11) < $max;
    else
        return true;
}
function writeLog() {
    file_put_contents($file, time() . "\n", FILE_APPEND);
}
function clean() {
    $f = fopen($file, 'r');
    while ($time = fgets($f)) {
        if ($now - $time < 3600) break;
    }
    while (!feof($f))
        $time .= stream_get_contents($f);
    fclose($f);
    file_put_contents($file, $time);
}

cheers