hpc

Log files in massively distributed systems

I do a lot of work in the grid and HPC space and one of the biggest challenges we have with a system distributed across hundreds (or in some case thousands) of servers is analysing the log files. Currently log files are written locally to the disk on each blade but we could also consider publishing logging information using for exampl...

Have you successfully used a GPGPU?

I am interested to know whether anyone has written an application that takes advantage of a GPGPU by using, for example, nVidia CUDA. If so, what issues did you find and what performance gains did you achieve compared with a standard CPU? ...

What are the advantages and disadvantages of GPGPU (general-purpose GPU) development?

I am wondering what is the key thing that helps you in GPGPU development and of course what is the constraints that you find unacceptable. Comes to mind for me: Key advantage: the raw power of these things Key constraint: the memory model What's your view? ...

Can I emulate MS Compute Cluster Server on my dev machine?

I have a project for a client that will consist of managing jobs on a MS Compute Cluster. I will be developing the application outside of their network, and would like a way to develop/debug my app without the need to be on their network. I am developing the app in C#, and all I have so far is the Microsoft Compute Cluster Pack SDK. ...

Rent A Cluster

I am wondering if there are clusters available to rent. Scenario: We have a program that will take what we estimate a week to run(after optimization) on a given file. Quite possibly, longer. Unfortunately, we also need to do approximately 300+ different files, resulting in approximately 300 weeks of compute time(roundable to 6 wallcloc...

Does Anyone Have a real-world MPI.NET application they are willing to share?

I'm an experienced .NET developer trying to learn HPC programming and am starting with MPI.NET (closest to my existing knowledge set). I've seen basic samples and gone through the tutorials provided by ISU (thanks guys) but, in the mode of Scott Hanselman's "be a better programmer by reading someone else's code", I'm looking for a real-w...

CUDA, OpenCL, PGI, etc.... but what happened to GLSL and Cg?

CUDA, OpenCL, and the GPU options offered by the Portland Group are intriguing... Results are impresive (125-times speedup for some groups). It sounds like the next wave of GPGPU tools are poised to dominate the scientific computing world. However, I recall the same fanfare when GLSL and Cg were announced. What ever happened to GLSL a...

Running a Data Synapse Win32 DLL Grid Node on a Windows 64 Machine?

We are developing a Data Synapse calc node process in C# that requires functionality in a Win32 DLL. We have no problems building this. The question is it has to run on a Windows 64 bit system, with Data Synapse 5.1 Grid APIs for Windows 64. While Windows 64 will run most processes transparently using WOW we don't know that Data Synapse...

Automated Testing of Binaries from SVN on HPC

I would like to setup some automated testing of test cases upon Fortran binaries within an SVN Repository, on demand, or with a Cron job. To complicate the issue slightly, the test cases would be run on a computational cluster, so that each test case would need to generate a PBS script. (Linux Environment) There are a lot of web testin...

Assignment Algorithm

Hi, I need to assign N entities (each with possible parents and possible children) to M computation nodes while satisfying the following optimization conditions: Children of an entity want to be assigned to the same computation node (to maximize data locality among siblings) The distribution of entities should be as even as possible (...

C# Property System

Update Sorry. I didn't mean the whole reflection library was off limits. I just meant the insanely slow *.Invoke() stuff. Hi, I need to implement a property system in C# that allows both normal property access [property_attribute()] return_type Property { get; set; } and access by string SetProperty(string name, object value); obje...

Intel MKL vs. AMD Math Core Library

Does anybody have experience programming for both the Intel Math Kernel Library and the AMD Math Core Library? I'm building a personal computer for high performance statistical computations and am debating on the components to buy. An appeal of the AMD Math Core library is that it is free, but I am in academia so the MKL is not that ex...

Is the PVM (parallel virtual machine) library widely used in HPC?

Has everyone migrated to MPI (message passing interface) or is PVM still widely used in supercomputers and HPC? ...

MPI , Sungrid vs JPPF ?

Greetings , I have a little experience with SungridEngine and MPI (using OpenMPI). Whats the different between these frameworks/API and JPPF ? ...

PVM terminates after Adding Host

On Ubuntu 9.10 using PVM 3.4.5-12 (the PVM package when you use apt-get) The program terminates after adding a host. laptop> pvm pvm> add bowtie-slave add bowtie-slave terminated laptop> Current Configuration only $PVM_RSH = bin/usr/ssh I can ssh perfectly fine into the slave without a password, and run commands on it. Any ideas? T...

SunGridEngine, Condor, Torque as Resource Managers for PVM

Anyone have any idea which Resource manager is good for PVM? Or should I not have used PVM and instead relied on MPI (or any version of it, such as MPICH-2 [are there any other ones that are better?]). Main reason for using PVM was because the person before me who started this project assumed the use of PVM. However, now that this projec...

Condor job using DAG with some jobs needing to run the same host

I have a computation task which is split in several individual program executions, with dependencies. I'm using Condor 7 as task scheduler (with the Vanilla Universe, due do constraints on the programs beyond my reach, so no checkpointing is involved), so DAG looks like a natural solution. However some of the programs need to run on the ...

MPICH vs OpenMPI

Can someone elaborate the differences between the OpenMPI and MPICH implementations of MPI ? Which of the two is a better implementation ? ...

Can the STREAM and GUPS (single CPU) benchmark use non-local memory in NUMA machine

Hello I want to run some tests from HPCC, STREAM and GUPS. They will test memory bandwidth, latency, and throughput (in term of random accesses). Can I start Single CPU test STREAM or Single CPU GUPS on NUMA node with memory interleaving enabled? (Is it allowed by the rules of HPCC - High Performance Computing Challenge?) Usage of no...

C++ programming for clusters and HPC

I need to write a scientific application in C++ doing a lot of computations and using a lot of memory. I have part of the job but due to high requirements in terms of resources I was thinking to start moving to OpenMPI. Before doing that I have a simple curiosity: If I understood the principle of OpenMPI correctly it is the developer t...