Hi there, For my bachelors thesis, I am implementing a distributed version of an algorithm for factoring large integers (finding the prime factorisation). This has applications in e.g. security of the RSA cryptosystem. My vision is, that clients (linux or windows) will download an application and compute some numbers (these are indepen...
I have an architectural question on handling message ordering. For purposes of this question, the transport is irrelevant, so I'm not going to specify one. Say we have three systems, a website, a CRM and an ERP. For this example, the ERP will be the "master" system in terms of data ownership. The website and the CRM can both send a new ...
Hi, I have a nested loop that I'm using foreach, DoSNOW, and a SNOW socket cluster to solve for. How should I go about profiling the code to make sure I'm not doing something grossly inefficient. Also is there anyway to measure the data flows going between the master and nodes in a Snow cluster? Thanks, James ...
From hard experience I've found it useful to occasionally save the state of my long computations to disk to start them up later if something fails. Can I do this in a distributed computation package in R (like SNOW or multicore)? It does not seem clear how this could be done since the master is collecting things from the slaves in a non...
hi('/') i am writing an application that need to be distributed on different nodes. what is the recommended framework. ...
My goal is to create a Distributed computing program that launches a server and client at the same time. I need it to be able to install on a couple of machines and have all the machines communicating with each other, i.e. Master node and 5 slave nodes all from one application. My problem is that I cannot properly use unicastRef, I'm th...
Im interested in programming a Project which distributes a certain computation on large files throughout several computers. The need for distributed computing arises from the crashy and unstable nature of the software I'm using to do the actual computing - so it might crash on some computers but others will surely do the job. The ideas I...
What is Terracotta? What services does it offer? What problems does it solve? What other products solve problems similar to those that Terracotta solves? ...
I'm interested in learning techniques for distributed computing. As a Java developer, I'm probably willing to start with Hadoop. Could you please recommend some books/tutorials/articles to begin with? ...
I'm stuck with some problem where I have to design a leader election algorithm for an oriented hypercube. This should be done by using a tournament with a number of rounds equal to the dimension D of the hypercube. In each stage d, with 1 <= d < D two candidate leaders of neighbouring d-dimensional hypercubes should compete to become the...
Scenario Does anyone have any good examples of peer-to-peer (p2p) networking in C++ using Winsock? It's a requirement I have for a client who specifically needs to use this technology (god knows why). I need to determine whether this is feasible. Any help would be greatly appreciated. EDIT And I would like to avoid using libraries so...
Google has described a novel framework for distributed processing on Massive Graphs. http://portal.acm.org/citation.cfm?id=1582716.1582723 I wanted to know if similar to Hadoop (Map-Reduce) are there any open source implementations of this framework? I am actually in process of writing a Pseudo distributed one using python and multip...
I made a website that does absolutely nothing, and I've proven to myself that people like to stay there - I've already logged 11+ hours worth of cumulative time on the page. My question is whether it would be possible (or practical) to use the website as a distributed computing site. My first impulse was to find out if there were any J...
Hi, There are many job descriptions having a phrase similar to "building distributed internet-scale systems" in the list of required skills. Although I do have a basic sense of what it probably involves (using technologies like Hadoop, key-value stores, message queues, focusing on scalability, availability etc), could you describe how...
We aim to implement a distributed system on a cluster, which will perform resource-consuming image-based computing with heavy storage I/O, having following characteristics: There is a dedicated manager computer node and up to 100 compute nodes. The cluster must be easily expandable. It is built around job-task concept. A job may have o...
I'm looking for a framework (open source) for distributed computing for .Net / Mono that is not simply task-based but supports persistence of distributed tasks. The project at hand is a complex system simulation which needs to be distributed into smaller independent "subsimulations". These subsimulations will keep running for a long tim...
I know the theritical definition for distributed computing and interproces communication. But in real time I was not able to come to conclusion that when we go for distributed or interprocess. Tell me some scenario where we can go for distributed computing or interprocess communication by example. ...
I've seen a few questions here on S.O where people recommend "checking out Qizmt", but has anyone actually deployed a Qizmt instance and used it? Do you have any reviews/tips/tricks/gotchas? How does it compare to other distributed computing implementations? Ease-of-use, performance, etc. are all valid review points. I assume that if...
Hi, I would like to implement a GA-based feature selection program for an SGI UV system (cores=512, shared memory=4TB) which operates as thus: (Feature selection is the process by which the smallest subset of original features is found which makes discrimination between the output classes better than using the original set of features-or...
I am curious about the contraints and tradeoffs for generating unique sequence numbers in a distributed and concurrent environment. Imagine this: I have a system where all it does is give back an unique sequence number every time you ask it. Here is an ideal spec for such a system (constraints): Stay up under high-load. Allow as many...