The cucumber test first makes an entry in the database and posts a form to a second server. This second server does some processing in background and then hits the first app (where the test is being run) with some data that the cucumber test needs to know about. I've tried running the main server via script/server and script/server -e ...
Hi, I have a Terabyte of data, maybe more, which I'd like to index and search with Lucene. I'd like to be able to split the index out to different machines, similar to what Solr does (if I understand Solr correctly). Are there any existing tools to do this on the Windows platform? Thanks! Edit: I'm not very keen on running Java Luce...
I've heard these two terms quite many times, but always confused. My guess: for distributed servers,each server has its own unique data to serve for clustered servers,each server is supposed to have the same data given enough time to synchronize Is that the difference? ...
hi. creating a distributed crawling python app. it consists of a master server, and associated client apps that will run on client servers. the purpose of the client app is to run across a targeted site, to extract specific data. the clients need to go "deep" within the site, behind multiple levels of forms, so each client is specifical...
Hi.. Maybe not the correct place to post. But, I'm going to try anyway! I've got a couple of test python parsing scripts that I created. They work enough for me to test what I'm working on. However, I recently came across the python framework, Scrapy, which is used for web scraping. My app runs in a distributed process, across a test...
I'm working on application for tracking vehicles. There will be about 10k or more vehicles. Each will be sending ~250bytes in each minute. Data contains gps location and everything from CAN Bus (every data that we can read from vehicle computer and dashboard). Data are sent by GSM/GPRS (using UDP protocol). Estimated rows with this data ...
I am writing a driver to act as a wrapper around two separate MySQL connections (to distributed databases). Basically, the goal is to enable interaction with my driver for all applications instead of requiring the application to sort out which database holds the desired data. Most of the code for this is in place, but I'm having a prob...
I am looking for guidance on how to best think about designing a high-level application protocol to sync metadata between end-user devices and a server. My goal: the user can interact with the application data on any device, or on the web. The purpose of this protocol is to communicate changes made on one endpoint to other endpoints thr...
I have a C shell script that calls two C programs - one after the another with some file handling before, in-between and afterwards. Now, as such I have three different files - one C shell script and 2 .c files. I need to give this script to other users. The problem is that I have to distribute three files - which the users m...
What resources would you recommend I look into to get started on implementing a basic distributed filesystem? Are there any open source distributed filesystems that are simple enough to be a good starting point? ...
Hi guys, I'm doing a system for a hospital in my country as the final year project of my degree, my supervisor specially asked me to use php and mysql for this. i don't have any experience with distributed systems and php programming, can any one help me out to build my base and improove my knowledge stating some sites, books to refer t...
Hi, There is a technical requirement to scale a new system easily. This new system consists of three tiered applications (as a batch processors). Each tier will contains at least 2 servers with the same application resides on each server. So, when one of the tier reaches peak performance, we could extend the scalability easily by addi...
We're considering a switch from SVN to a distributed VCS at my workplace. I'm familiar with all the reasons for wanting to use a DVCS for day-to-day development: local version control, easier branching and merging, etc., but I haven't seen that much that's compelling in terms of managing software releases. Here's our release process: ...
There seems to a quite a few grid computing frameworks out there, but which ones are actually being used to any great extent by the investment banks for purposes of low latency distributing calculation? I'd be interested to hear answers covering both windows,Linux and cross platform. Also, what RPC mechanisms seem to be favoured most? I...
I'm stuck with some problem where I have to design a leader election algorithm for an oriented hypercube. This should be done by using a tournament with a number of rounds equal to the dimension D of the hypercube. In each stage d, with 1 <= d < D two candidate leaders of neighbouring d-dimensional hypercubes should compete to become the...
Hi, I have access to a grid (running condor) that would (potentially) allow to very substantially reduce how long by nltk based nlp tasks take. unfortunately, i dont have root access on the cluster so cannot install new packages, only run whatever is available on the linux boxes. python is of course available, but nltk isnt - i was won...
Hi there. I require a simple work queue type system for asynchronous task management. I have looked at both beanstalkd and gearman. However, both these seem to assume that the client and the queue server are on the same network, and therefore that there will always be a reliable network between them. I need one that can support the c...
I am interested in a system that allows for distributed development with an authentication piece. What do I mean by that? Ok so lets take SVN, SVN keeps track of revisions and doesn't care who submits, as long as you have the right to submit you can submit, really, to any part in the repository. Where does my system come into play? Bei...
I've never used Message Passing Interface (MPI), but I've heard its name thrown about, most recently with Windows HPC Server. I had a quick look on amazon to see if there were any books on it, but they're all dated around 7 or more years ago. Is MPI still a valid technology choice for new applications, or has it been largely superceded b...
We are currently starting to build a client-server system which will see 10 or more Apple iPad client devices communicating to a central Windows server over a wireless LAN. We wanted to some existing plumbing (.NET remoting/WCF/web services/etc) that would allow us to implement a reliable, secure solution without having to start at a lo...