parallel-processing

Massively Parallel algorithm to propagate pixels

I'm designing a CUDA app to process some video. The algorithm I'm using calls for filling in blank pixels in a way that's not unlike Conway's game of life: if the pixels around another pixels are all filled and all of similar values, the specific pixel gets filled in with the surrounding value. This iterates until all the number of pixel...

Python threading unexpectedly slower

I have decided to learn how multi-threading is done in Python, and I did a comparison to see what kind of performance gain I would get on a dual-core CPU. I found that my simple multi-threaded code actually runs slower than the sequential equivalent, and I cant figure out why. The test I contrived was to generate a large list of random ...

Perl Parallel::ForkManager with DBI database handlers

Hi all, im interested in performing multiple database actions in parallel. I have played with Perl Parallel::ForkManager but not used it with any databases yet. I have read that database connectivity is not supported very well with this. Does anyone have experience with this? As an example i would probably be spawning a system call(whic...

How to run multiple macro instances using Remote Dekstop Connection and Amazon AWS EC2

I am trying to to run multiple Remote Desktop connection instances with Amazon's AWS EC2 and keep a macro file running on each instance via Macro Express (installed separately on each node), but each time I minimize an instance, the the mouse movement of that macro freezes and the process is suspended. Since these macros that I am runni...

Counting Unique Users using Mapreduce for Java Appengine

I'm trying to count the number of unique users per day on my java appengine app. I have decided to use the mapreduce framework (mapreduce.appspot.com) for java appengine to do this calculation offline. I've managed to create a map reduce job that goes through all of my entities which represent a single users session event. I can use a si...

Multiple levels of parallelism using OpenMP - Possible? Smart? Practical?

I am currently working on a C++ sparse matrix/math/iterative solver library, for a simulation tool I manage. I would have preferred to use an existing package, however, after extensive investigation, none were found that were appropriate for our simulator (we looked at flens, it++, PetSC, eigen, and several others). The good news is my...

MATLAB parfor is slower than for -- what is wrong?

Hi, the code I'm dealing with has loops like the following: bistar = zeros(numdims,numcases); parfor hh=1:nt bistar = bistar + A(:,:,hh)*data(:,:,hh+1)' ; end for small nt (10). After timing it, it is actually 100 times slower than using the regular loop!!! I know that parfor can do parallel sums, so I'm not sure why t...

What is meant by diameter of a network?

The diagram shown on this link of the "A graph with 6 vertices and 7 edges where the vertex no 6 on the far-left is a leaf vertex or a pendant vertex." has DIAMETER 4? right or wrong? Definitions are The diameter of a graph is the maximum eccentricity of any vertex in the graph. That is, it is the greatest distance between any...

Any C++ libraries to run a single program on multiple PC's (i.e. "use grid computing to run my app")

I'm after a method of converting a single program to run on multiple computers on a network (think "grid computing"). I'm using MSVC 2007 and C++ (non-.NET). The program I've written is ideally suited for parallel programming (its doing analysis of scientific data), so the more computers the better. ...

Parallelism in PL/SQL

hi how can i run one query in pl-sql- in parallel? i need all the flow... ...

How does one write cross-platform parallel programs in C?

I am working on a programming language. Currently it compiles to C. I would like to be able to include parallel programming facilities natively in my language so as to take advantage of multiple cores. Is there a way to write parallel C programs which is cross-platform? I would prefer to stick to straight C so as to maximize the number o...

throttling when using Parallel.For

When using a single threadded loop I was easily able to limit my messages sent per second by putting the thread to sleep (ie Thread.Sleep(1000/MessagesPerSecond)), easy enough... but now that I have expanded into parallel threads this no longer works properly. Does anyone have a suggestion how to throttle messages sent when using Parall...

throttle parallel request to remote api

I'm working on an ASP.NET MVC application that uses the Google Maps Geocoding API. In a single batch there may be upto 1000 queries to submit to the Geocoding API, so I'm trying to use a parallel processing approach to imporove performance. The method responsible for starting a process for each core is: public void GeoCode(Queue<Job> ...

Quickly load 350M numbers into a double[] array in C#

I am going to store 350M pre-calculated double numbers in a binary file, and load them into memory as my dll starts up. Is there any built in way to load it up in parallel, or should I split the data into multiple files myself and take care of multiple threads myself too? Answering the comments: I will be running this dll on powerful en...

Distributed, Parallel testing with Rails

I've been recently playing around with distributing our testing load across multiple machines (and multiple cores) over the network. I've tried out: SpecJour http://github.com/sandro/specjour This worked pretty well and was really easy to set up. Reduced out testing time by about 30% across 4 machines (8 cores) However, I really li...

Is there a threadsafe and generic IList<T> in c#?

Is List<T> or HashSet<T> or anything else built in threadsafe for addition only? My question is similar to http://stackoverflow.com/questions/1278010/threadsafe-and-generic-arraylist-in-c but I'm only looking for safety to cover adding to this list threaded, not removal or reading from it. ...

What's an event-loop and how is it different than using other models?

I have been looking into Node.JS and all the documentation and blogs talk about how it uses an event-loop rather than a per-request model. I am having some confusion understanding the difference. I feel like I am 80% there understanding it but not fully getting it yet. ...

Is there a Scheme implementation that parallelizes?

Is there a R5RS-or-higher Scheme implementation that does parallelization? For example, if I say to do: (map (lambda (x) (pure-functional-stuff x)) '(1 3 5 7 11 13)) it will process 1, 3, 5, and 7 simultaneously if the machine can do it? That's supposed to be one of the big advantages of functional programming, but I c...

What are canonical examples of parallel computation?

I am writing a paper to test a new application that will demonstrate the benefits of parallelized computation (compared to the traditional serialized version of this application). I want to use the canonical examples for parallel computation in my paper. My first example is the parallel computation of pi. I would ideally like an exa...

Need tutorial for implementing my own Krylov method that is not provided with PETSc.

Do anyone know a good and basic tutorial to how to implement my own Krylov method that is not provided with PETSc in PETSc. It should teach how to execute them as well... ...