how can i 'flush' my cURL session's memory without opening a new session?

Question

I am writing a scraper in PHP using cURL but am experiencing some memory issues.

These memory issues arise since i am using a single cURL session to log in to a website, and then to scrape many pages.

Is there a way to 'flush' the cURL session's memory without having to close the session, open another one and re-log in?

Perhaps I could terminate the cURL session but use the cookies I had saved previously?

Thanks :)

Answer 1

You can terminate and reopen a cURL session without loosing the cookies. Just add those options when initializing the session.

$fn = "cookies.txt";  // Make sure this file is read- and writable
curl_setopt($ch, CURLOPT_COOKIEJAR, $fn);
curl_setopt($ch, CURLOPT_COOKIEFILE, $fn);

I never tried if it really works with session cookies too, but for many things it worked well enough.