Initially, when I made the question I attributed the problem to wildcards, a problem, but a bigger problem apparently looming. If I understand the error right, GNU make has no make-subprocessing so have to switch to some derivative of Boor's make. I may be understanding it wrong but I try to make this problem easier to understand. Makef...
My MysQL database contains multiple MyISAM tables, with each table containing millions of rows. There is a heavy insert load on the database, so I cannot issue SELECTs on that live database. Instead, I create a replica of the database for queries and conduct analysis on that. For the analysis, I need to issue multiple parallel queries....
I am writing a small C application that use some threads for processing data. I want to be able to know the number of processors on a certain machine, without using system() & in combination to a small script. The only way i can think of is to parse /proc/cpuinfo. Any other useful suggestions ? ...
Hi guys, I'm posting this question to ask for advice on how to optimize the use of multiple processors from R on a Windows XP machine. At the moment I'm creating 4 scripts (each script with e.g. for (i in 1:100) and (i in 101:200), etc) which I run in 4 different R sessions at the same time. This seems to use all the available cpu. I ...
I'm hoping to get some advice on the use of thread managment and hopefully the task parallel library, because I'm not sure I've been going down the correct route. Probably best is that I give an outline of what I'm trying to do. Given a Problem I need to generate a Solution using a heuristic based algorithm. I start of by calculating a ...
I'm interested in optimizing the hashing of some large files (optimizing wall clock time). The I/O has been optimized well enough already and the I/O device (local SSD) is only tapped at about 25% of capacity, while one of the CPU cores is completely maxed-out. I have more cores available, and in the future will likely have even more...
hello. I am curious as to how F# performance compares to C++ performance? I asked a similar question with regards to Java, and the impression I got was that Java is not suitable for heavy numbercrunching. I have read that F# is supposed to be more scalable and more performant, but how is this real-world performance compares to C++? s...
I am having some issues involving Parallel for loops and adding to a List. The problem is, the same code may generate different output at different times. I have set up some test code below. In this code, I create a List of 10,000 int values. 1/10th of the values will be 0, 1/10th of the values will be 1, all the way up to 1/10th of the ...
Hey all- I'm using ParallelPython to develop a performance-critical script. I'd like to share one value between the 8 processes running on the system. Please excuse the trivial example but this illustrates my question. def findMin(listOfElements): for el in listOfElements: if el < min: min = el import pp min...
I'm just looking in to the new .NET 4.0 features. With that, I'm attempting a simple calculation using Parallel.For and a normal for(x;x;x) loop. However, I'm getting different results about 50% of the time. long sum = 0; Parallel.For(1, 10000, y => { sum += y; } ); Console.WriteLine(sum.ToString()); sum = 0; for...
In .NET 4 and multicore environment, does the linq to sql datacontext object take advantage of the new parallels if we use DataLoadOptions.LoadWith? EDIT I know linq to sql does not parallelize ordinary queries. What I want to know is when we specify DataLoadOption.LoadWith, does it use parallelization to perform the match between each...
I have a C shell script that does something like this: #!/bin/csh gcc example.c -o ex gcc combine.c -o combine ex file1 r1 <-- 1 ex file2 r2 <-- 2 ex file3 r3 <-- 3 #... many more like the above combine r1 r2 r3 final \rm r1 r2 r3 Is there some way I can make lines 1, 2 and 3 run in parallel instead of one after the anothe...
Quoting http://www.mongodb.org/display/DOCS/MapReduce#MapReduce-Parallelism As of right now, MapReduce jobs on a single mongod process are single threaded. This is due to a design limitation in current JavaScript engines. We are looking into alternatives to solve this issue, but for now if you want to parallelize your M...
Hi there, I'm having a problem where I have a queue set up in shared mode and multiple consumers bound to it. The issue is that it appears that rabbitmq is serializing the messages, that is, only one consumer at a time is able to run. I need this to be parallel, however, I can't seem to figure out how. Each consumer is running in its...
I have a shell script that transfers a build.xml file to a remote unix machine (devrsp02) and executes the ANT task wldeploy on that machine (devrsp02). Now, this wldeploy task takes around 15 minutes to complete and while this is running, the last line at the unix console is - "task {some digit} initialized". Once this task is co...
I'm running R 2.11 64-bit on a WinXP64 machine with 8 processors. With R 2.10.1 the following code spawned 6 R processes for parallel processing: require(foreach) require(doSNOW) cl = makeCluster(6, type='SOCK') registerDoSNOW(cl) bl2 = foreach(i=icount(length(unqmrno))) %dopar% { (Some code here) } stopCluster(cl) When I run the...
Hi, Lets say I have two shared variables - a and b - that are related to each other. When multiple applications share these shared variables, access to them needs to be an atomic operation, otherwise the relation may break. So to ensure mutual exclusion, I'll put their modification under a critical section protected by lock. critical_c...
There are certain algorithms whose running time can decrease significantly when one divides up a task and gets each part done in parallel. One of these algorithms is merge sort, where a list is divided into infinitesimally smaller parts and then recombined in a sorted order. I decided to do an experiment to test whether or not I could I ...
I have an inline assembler loop that cumulatively adds elements from an int32 data array with MMX instructions. In particular, it uses the fact that the MMX registers can accommodate 16 int32s to calculate 16 different cumulative sums in parallel. I would now like to convert this piece of code to MMX intrinsics but I am afraid that I wi...
Here is problem. long sum = 0; Parallel.For(1, 10000, y => { sum1 += y;} ); Solution is .. Parallel.For<int>(0, result.Count, () => 0, (i, loop, subtotal) => { subtotal += result[i]; return subtotal; }, (x) => Interlocked.Add(ref sum, x) ); if there are two parameters in this code. For example long sum1 = 0; long sum2 = 0;...