groovy multithreading

Question

Hi, I'm newbie to groovy/grails.

How to implement thread for this code . Had 2500 urls and this was taking hours of time for checking each url.

so i decided to implement multi-thread for this :

Here is my sample code :

def urls = [
  "http://www.wordpress.com",
  "http://67.192.103.225/QRA.Public/" ,
  "http://www.subaru.com",
  "http://baldwinfilter.com/products/start.html"
]

def up = urls.collect { ur ->
    try {
        def url = new URL(ur)
        def connection = url.openConnection()
        if (connection.responseCode == 200) {
            return true
        } else {
            return false
        }
    } catch (Exception e) {
        return false
    } 
}

For this code i need to implement multi-threading .
Could any one please suggest me the code.

thanks in advance,
sri.

Answer 1

See the Groovy docs for an example how to use an ExecutorService to do what you want.

Answer 2

I would take a look at the Groovy Parallel Systems library. In particular I think that the Parallel collections section would be useful.

Looking at the docs, I believe that collectParallel is a direct drop-in replacement for collect (bearing in mind the obvious caveats about side-effects). The following works fine for me:

def urls = [
  "http://www.wordpress.com",
  "http://www.subaru.com",
  "http://baldwinfilter.com/products/start.html"
]
Parallelizer.doParallel {
    def up = urls.collectParallel { ur ->
        try {
            def url = new URL(ur)
            def connection = url.openConnection()
            if (connection.responseCode == 200) {
                return true
            } else {
                return false
            }
        } catch (Exception e) {
            return false
        }
    }
    println up
}

Answer 3

You can use this to check the URL in a separate thread.

class URLReader implements Runnable
{
    def valid
    def url

    URLReader( url ) {
        this.url = url
    }

    void run() {
        try {
            def connection = url.toURL().openConnection()
            valid = ( connection.responseCode == 200 ) as Boolean
        } catch ( Exception e ) {
            println e.message
            valid = Boolean.FALSE
        }
    }
}
def reader = new URLReader( "http://www.google.com" )
new Thread( reader ).start()
while ( reader.valid == null )
{
    Thread.sleep( 500 )
}
println "valid: ${reader.valid}"

Notes: The valid attribute will be either null, Boolean.TRUE or Boolean.FALSE. You'll need to wait for a while to give all the threads a chance to open the connection. Depending on the number of URLs you're checking you will eventually hit a limit of the number of threads / connections you can realistically handle, so should check URLs in batches of the appropriate size.

Answer 4

This is how i implemented :

class ValidateLinks extends Thread{
def valid
def url

ValidateLinks( url ) {
    this.url = url
}

void run() {
    try {
        def connection = url.toURL().openConnection()
    connection.setConnectTimeout(5000)
        valid = ( connection.responseCode == 200 ) as Boolean
    } catch ( Exception e ) {
        println url + "-" + e.message
        valid = Boolean.FALSE
    }
}

}

def threads = [];
urls.each { ur ->
def reader = new ValidateLinks(ur.site_url)
reader.start()
threads.add(reader);
}

while (threads.size() > 0) {

        for(int i =0; i < threads.size();i++) {
                def tr = threads.get(i);
                if (!tr.isAlive()) {
                        println "URL : " + tr.url  + "Valid " + tr.valid
                        threads.remove(i);
                        i--;

                }
        }
}