how to loop a loop in php?

Question

is there a way to loop a loop in php?

if i got a function for example, could i loop this function endless times?

in php6 there is goto. but how could u do this in php5?

function getLinks($link) {
    // step 1: if $link got links, add them in a array
    // step 2: iterate array of links
    // step 3: save current link
    // step 4: check if current link got links, if it has, run $this->getLinks($link)
}

in step 4 i need to use the function getLinks. And as you can see i want to make a crawler that crawls every link and saves them.

has someone done anything similar, tried to loop/crawl all levels of links?

i think i need the Goto function here but is there a way to accomplish this in php 5?

i have no idea how i could do this...thanks in advance!

Answer 1

Use a while() loop instead, and break to get out.

while(1)
{
  ....
};

Answer 2

Yes, this is possible. If you are calling the same method inside of the method, you are using recursion. One of the big things to remember about recursive loops is to ensure you add a base case to stop the loop, otherwise you'll loop continuously and that's not good.

I would suggest not going the recursive route, its a lot of overhead and should be your last resort. Using a while loop should achieve similar results.

Answer 3

Its called a recursive function (just call the function again in the body of the function). However, if you're building a crawler, I'd caution you to be well behaved. You can check out http://www.robotstxt.org/. Things like the amount of time between requests on the same domain can be limited by the owners robots.txt file. If you don't behave well, you can get banned by many sys admins.

Answer 4

You could do what you wrote in your comments:

function getLinks($link) {
    step 1: Parse the page and get the list of links ==> $listLinks
    loop - step 2: iterate over $listLinks (currLink = $currLink)
        step 3: $globalListOfLinks[count($globalListOfLinks)] = $currLink
        step 4: $this->getLinks($link);
    end loop;
}

Of course you have to look after the number of recursions, you cannot recurse too far, so, you could also have a depth variable and pass it along with the $link. If you are > some depth, return;

Answer 5

I don't think you need a goto to perform this task. To be pedant you don't need a goto. Ever! Böhm-Jacopini :)

Maybe with a bit of code I would be more precise but, assuming that $links is an url and that you have a function called harvestPage that get the url of a page and return an array with all the links (urls) it contains you don't have to resort to recursion neither. A pipe (FIFO) will be sufficent:

function getLinks($link) {
 $linksToCrawl = harvestPage($links)
  while (count($linksToCrawl)) {
    $currentLink = array_shift($linksToCrawl);
    save($CurrentLink);
    $linksToCrawl = array_merge($linksToCrawl, harvestPage($currentLink));
  }
}

Remember you should check for circular references too (and maybe the robot.txt too :) ).

Answer 6

Something like...

function stuff($link,$links=Array()){
  if (($contents=@file_get_contents($link))!==false){
    preg_match_all('@https?://[^"\'}{\[\]<>\s]{3,255}@i',$contents,$linkDump);
    $links=array_merge($links,$linkDump[0]);
  }
  stuff(array_shift($links),$links);
}
stuff('http://www.foxnews.com/'); //packet them all you want while testing ;)

probably better to use curl class and parse with callbacks until there's no more links, rather than recurse. You need to filter out css,js,pics, etc and make sure you don't do the same links repeatedly. file_get_contents is stupid but good 'cause lets me write it short hand and still functional.