Scan xml for links using ajax, apply links to another ajax call to scrape page and return data.

Question

So, I've tried to look around before posting, but I can't seem to find an answer. My dilemma:

I have an XML file that houses url links to various pages(all similar, diff products).

By using jQuery and AJAX, I am able to pull the links from the XML file.

I then want to be able to pass those links, in order, to another AJAX call that will use a proxy server to connect to another site and scrape it for data. In this case, it's a specific class. I then need to be able to collect the data within that class and apply it to a span in my site, in order.

HTML markup that will be waiting for the scraped data

<div class="panel">
   <span class="price"></span>
</div>
<div class="panel">
   <span class="price"></span>
</div>
<div class="panel">
   <span class="price"></span>
</div>

JS executing all of this:

//Pull Content Necessary Links From XML File
$.ajax({
    type: "GET",
    url: "/Test/addTocart/realExample/Media/xml/items.xml",
    dataType: "xml",
    success: function(xml) {
        $(xml).find('target').each(function() {

            //Scan Through XML to Find Each Link
            link = $(this).find('link').text();

            function fetchPage(url) {
                $.ajax({
                    type: "GET",
                    url: "http://test/crossdomainproxy.aspx?u=" + link,
                    error: function(request, status) {
                        alert('Error fetching ' + url);
                    },
                    success: function(data) {
                        parse(data);
                    }
                });
            }

            function parse(data) {
                alert($(data).find(".threeThin .price-side").text());
            }

            fetchPage(1);           

        });
    }
});

So essentially, if the other pages markup looks like this:

Site 1

<span class="price-side">$23.33</span>

Site 2

<span class="price-side">$22.33</span>

Site 3

<span class="price-side">$245.33</span>

My markup should look like this:

<div class="panel">
   <span class="price">$23.33</span>
</div>
<div class="panel">
   <span class="price">$22.33</span>
</div>
<div class="panel">
   <span class="price">$245.33</span>
</div>

So I guess my question would be, how do I specify them to load in my markup in order? Do I loop through links in xml, then throw them into ajax --> proxy and then loop through the results to spit out to each span?

Any help would be greatly appreciated! Thanks!

Answer 1

If I were you, I wouldn't put an ajax request inside a loop. If you want to perform multiple requests to a proxy, you should put that logic on your server and pass an array of links to that server script with just one http request from the browser. Once you have the data back, you could even use ASP to return an array of objects with the id and price for each item. Then, in javascript, you can loop through that data and wrap it in whatever html you like using Javascript's append() or html() (or whatever DOM manipulation you want to use). Please let me know if I'm missing an important piece of your question.