I'm looking for a way to easily load test and benchmark some of our SQL (using ADO.NET, nothing fancy using LINQ or PLINQ) that has to be performant when running under high parallel load. I've thought of using the new parallel extensions CTP and specifically Parallel.For / Parallel.ForEach to simply run the SQL over 10k iterations or so...
We have a stored procedure that runs nightly that in turn kicks off a number of other procedures. Some of those procedures could logically be run in parallel with some of the others. How can I indicate to SQL Server whether a procedure should be run in parallel or serial ie: kicked off of asynchronously or blocking? What would be...
As someone in the world of HPC who came from the world of enterprise web development, I'm always curious to see how developers back in the "real world" are taking advantage of parallel computing. This is much more relevant now that all chips are going multicore, and it'll be even more relevant when there are thousands of cores on a chip...
How can i connect two or more machines to form a network grid and how can i distribute work load to the two machines? What operating systems do i need to run on the machines, and what application should i use to manage the load balancing? NB: I read somewhere that google uses cheap machines to perform this fete, how do they connect tw...
I'm spending a bit of time with Erlang, and I'm wanting to apply TDD to code I'm writing. While EUnit in the standard lib provides a nice traditional unit testing framework for testing regular style code, there doesn't seem to be anything to help specifically with testing concurrent code, which is used a LOT in Erlang. Note that we're ...
I'd like to have the full history of a large text field edited by users, stored using Django. I've seen the projects: Django Full History (Google Code) Django ModelHistory, and Django FullHistory I've a special use-case that probably falls outside the scope of what these projects provide. Further, I'm wary of how well documented, te...
I have a list/queue of 200 commands that I need to run in a shell on a Linux server. I only want to have a maximum of 10 processes running (from the queue) at once. Some processes will take a few seconds to complete, other processes will take much longer. When a process finishes I want the next command to be "popped" from the queue a...
The team I work with attended the SuperComputing Convention last November, but it was geared far more towards hardware then application and algorithms. We would like to find a convention for us to attend that caters towards the algorithmic and implementation side of things. We have found this convention, but I haven't seen any press on ...
Does anyone know of any good websites on the upcoming "Grand Central" technology from Apple (in Snow Leopard) and how it compares to the Parallel Extensions (and the native parallel computing stuff) coming from Microsoft? ...
This was a question in one of my CS textbooks. I am at a loss. I don't see why it necessarily would lead to parallel computing. Anyone wanna point me in the right direction? ...
I currently have an application which can contain 100s of user defined formulae. Currently, I use reverse polish notation to perform the calculations (pushing values and variables on to a stack, then popping them off the stack and evaluating). What would be the best way to start parallelizing this process? Should I be looking at a functi...
I am quite excited by the possibility of using languages which have parallelism / concurrency built in, such as stackless python and erlang, and have a firm belief that we'll all have to move in that direction before too long - or will want to because it will be a good/easy way to get to scalability and performance. However, I am so u...
For relatively coarse-grained parallel processing (data transfers can be done in 100 Mbit Ethernet network), which middleware that is available for Windows and Linux do you consider best? CORBA? MPI1? MPI2? XML-RPC/SOA/WSDL? Custom network protocols? Java RPC? Python RPC? Others? ...
I need to parallelize a simple password cracker, for using it on a n-processor system. My idea is to create n threads and to feed them more and more job as they finish. What is the best way to know when a thread has finished? A mutex? Isn't expensive checking this mutex constantly while other threads are running? ...
I have D2 program that, in its current form, is single threaded, and calls the same pure function about 10 to 100 times in an inner loop for each iteration of the outer loop of this program. There is no data dependency among the calls, i.e. no call uses the result from any other call. Overall, this function is called millions of times,...
I've been pondering this question for a while: Can you build a faster fundamental datastructure (i.e. linked list, hashtable, set, skiplist, bloom filter, red-black tree, etc.) on a multicore machine, by taking advantage of the fact that you've got more than one CPU? I did some preliminary experimenting with pthreads, and found that pt...
Is F# better than C# in scenarios where you need complete parallelism in parts of an application? My main language is C# and I am writing an app where the core functionality built on top of the app (C#) has to be and planned to be very parallel. Am I better of implementing these classes in F#? ...
I know C# is getting a lot of parallel programming support, but AFAIK there is still no constructs for side-effects verification, right? I assume it's more tricky now that C# is already laid out. But are there plans to get this in? Or is F# the only .NET language that has constructs for side-effects verification? ...
Erlang is known for being able to support MANY lightweight processes; it can do this because these are not processes in the traditional sense, or even threads like in P-threads, but threads entirely in user space. This is well and good (fantastic actually). But how then are Erlang threads executed in parallel in a multicore/multiproces...
If you plan to write a very computationally intensive parallel application, what guidelines would you use to design your objects (whether classes or structs, or anything else) to maximize your potential of getting the most out of the parallelism. I am thinking of an application that say interprets/compiles a tree-like graph of objects t...