Hello, I'm working on some code for a loosely coupled cluster. To achieve optimal performance during jobs, I have the cluster remap its data each time a child enters or exits. This will eventually be made optional, but for now it performs its data balancing by default. My balancing is basically just making sure that each child never has...
I'm working on a loosely coupled cluster for some data processing. The network code and processing code is in place, but we are evaluating different methodologies in our approach. Right now, as we should be, we are I/O bound on performance issues, and we're trying to decrease that bottleneck. Obviously, faster switches like Infiniband wo...
I've used clusters in industrial and academic settings, but they were owned by the organizations I worked for. I've never purchased time from a company that sells cluster/cloud/grid time. Suppose I want to run some compute intensive program like as a large simulation or a data mining application. Or maybe I want to optimize a complete b...
Is it possible for F# and MPI to run off of Mono on a linux computational cluster? Anyone able to attest to it's performance/reliability? ...
We know that a single application instance can use multiple cores and even multiple physical processors. With cloud and cluster computing, or other special scenario, I don't know if a single stance can run across multiple computers, or across multiple OS instances. This is important for me because, besides being considered as bad progra...
I understand the differences between a multithreaded program and a program relying on inter-machine communication. My problem is that I have a nice multithreaded program written in 'C' that works and runs really well on an 8-core machine. There is now opportunity to port this program to a cluster to gain access to more cores. Is it wo...
I've been thinking about getting a little bit greener with my computers and using some lower power, mini-itx boards in my next computer. Some can generate under 10 watts and are pretty inexpensive. So I though, if one is such low cost and low power, why not try to make a cluster out of them? However, I'm not really sure what I would nee...
Hi all, can you suggest me some good literature or course or manual to set up a small size (20-25 machines quad-core) computer cluster? Ideally, from zero to having a fully-functional multiuser queuing system. Thanks a lot :-) ...
Dear Overflowers: At work we perform demanding numerical computations. We have a network of several Linux boxes with different processing capabilities. At any given time, there can be anywhere from zero to dozens of people connected to a given box. I created a script to measure the MFLOPS (Million of Floating Point Operations per Sec...
I'm trying to use the gsh command on ubuntu to do some embarrassingly parallel stuff. I've made an /etc/ghosts file containing all the IP addresses for the hosts, and each host has an account that I created just for "cluster" purposes. However, when I try a command such as, $ gsh -l cluster ok 'echo $host' I get in return two errors...
Does anyone have any experience running BLAST with XGrid? Googling reveals a tool called 'Xgrid BLAST' existed but not where to get. ...
I have a library, written in C#, containing one method: Response CalculateSomething(Request); The execution time of this method is relatively large, and there are a lot of responses that should be processed. I want to use a "cluster", spread this DLL to different machines (nodes) in this "cluster" and write some controller that will d...
I need to write a scientific application in C++ doing a lot of computations and using a lot of memory. I have part of the job but due to high requirements in terms of resources I was thinking to start moving to OpenMPI. Before doing that I have a simple curiosity: If I understood the principle of OpenMPI correctly it is the developer t...
is it bad to output many files to the same directory in unix/linux? I run thousands of jobs on a cluster and each outputs a file, to one directory. The upper bound here is around ~50,000 files. Can IO be limited in speed in light of this? If so, does the problem go away with a nested directory structure? Thanks. ...
I've never used Message Passing Interface (MPI), but I've heard its name thrown about, most recently with Windows HPC Server. I had a quick look on amazon to see if there were any books on it, but they're all dated around 7 or more years ago. Is MPI still a valid technology choice for new applications, or has it been largely superceded b...
I am doing a project in the university which requires running of multiple instances (1000s) of a program I've written (in C++), which runs for quite a while (say 2 hours). The program is very self contained - it does not require input files, and the only dependency I think is boost. I'm currently using the university-owned cluster of co...
What are the best IDE's / IDE plugins / Tools, etc for programming with CUDA / MPI etc? I've been working in these frameworks for a short while but feel like the IDE could be doing more heavy lifting in terms of scaling and job processing interactions. (I usually use Eclipse or Netbeans, and usually in C/C++ with occasional Java, and ...
I am particularly interested how one can deal with a huge amount of information for a commercial service like Google Search or Google Maps. We all know they use (or "did" at least) a kind of Linux clusters, but how exactly are they organized? What kind of hardware do they use, what file systems, networking, what problems are the most fre...
I am doing some expensive caluations right now. It is one programm, which I run several instances of at the same time. I am running them under linux on a machine with 4 cpus with 6 cores each. The cpus are Intel Xeon X5660, which support hyper thearting. (That's some insane hardware, huh?) Right now I am running 24 processes at once. Wou...
I'm looking to implement 'branch and bound' over a cluster (like Amazon's say), as I want it to be horizontally scalable, not limited to a single CPU. There's a paper "Task Pool Teams: A Hybrid Programming Environment for Irregular Algorithms on SMP Clusters" by Judith Hippold and Gudula Runger. It's basically a bottom-up, task-stealing ...