I'm scraping data from the web, and I have several processes of my scraper running in parallel. I want the output of each of these processes to end up in the same file. As long as lines of text remain intact, I the order of the lines does not matter. In UNIX, can I just pipe the output of each process to the same file using the >> ope...
I have a database containing a single huge table. At the moment a query can take anything from 10 to 20 minutes and I need that to go down to 10 seconds. I have spent months trying different products like GridSQL. GridSQL works fine, but is using its own parser which does not have all the needed features. I have also optimized my databas...
In C++, I've written a mathematical program (for diffusion limited aggregation) where each new point calculated is dependent on all of the preceding points. Is it possible to have such a program work in a parallel or distributed manner to increase computing speed? If so, what type of modifications to the code would I need to look into? ...
I am working on a community detection algorithm for analyzing social network data from Facebook. The first task, detecting all cliques in the graph, can be done efficiently in parallel, and leaves me with an output like this: 17118 17136 17392 17064 17093 17376 17118 17136 17356 17318 12345 17118 17136 17356 17283 17007 17059 17116 E...
Not sure if this is possible in one Makefile alone, but I was hoping to write a Makefile in a way such that trying to build any target in the file auto-magically detects the number of processors on the current system and builds the target in parallel for the number of processors. Something like the below "pseudo-code" examples, but much...
I have a queue structure that is being used by several pthreads. The threads are supposed to dequeue from the queue if it's not empty and then do their business. I initially had this set up as a while loop where the threads checked whether the queue was empty using a mutex_lock. Unfortunately this slowed my program down to a crawl. I t...
Im very much new to map reduce and i completed hadoop wordcount example. In that example it produces unsorted file (with key value) of word counts. So is it possible to make it sorted according to the most number of word occurrences by combining another map reduce task to the earlier one. Thanks in Advance ...
I have a WCF service (the same one) running on multiple servers, and I'd like to call all instances in parallel from a single client. I'm using ChannelFactory and the interface (contract) to call the service. Each service has a local <endpoint> client defined in the .config file. What I'm trying to do is build some kind of generic frame...
Is there a good algorithm to calculate the cartesian product of three seqs concurrently in Clojure? I'm working on a small hobby project in Clojure, mainly as a means to learn the language, and its concurrency features. In my project, I need to calculate the cartesian product of three seqs (and do something with the results). I found t...
I'm currently doing a project where I have to interact with a circuit I made through the parallel port of a computer. However, my computer doesn't have a parallel port so I borrowed a Parallel to USB adapter cable. The cable didn't come with any drivers, but it's recognized by the device manager as a "USB Printing Support" controller, un...
Hello all, I have a high-end architecture which receives many requests every second (in fact, it can receive many requests every millisecond). The architecture is designed so that some controls rely on a certain unique id assigned to each request. To create such UID we use a DB2 Sequence. Right now I already understand that this appro...
I have a list of starting data. I want to apply a function to the starting data that creates a few pieces of new data for each element in the starting data. Some pieces of the new data are the same and I want to remove them. The sequential version is essentially: def create_new_data_for(datum): """make a list of new data from some...
ok, so I'm doing ProjectEuler Problem #14, and I'm fiddling around with optimizations in order to feel f# out. in the following code: let evenrule n = n / 2L let oddrule n = 3L * n + 1L let applyRule n = if n % 2L = 0L then evenrule n else oddrule n let runRules n = let rec loop a final = if a = 1L then final ...
Hello, startTrains() -> TotalDist = 100, Trains = [trainA,trainB ], PID = spawn(fun() -> train(1,length(Trains)) end), [ PID ! {self(),TrainData,TotalDist} || TrainData <- Trains], receive {_From, Mesg} -> error_logger:info_msg("~n Mesg ~p ~n",[Mesg]) after 10500 -> refresh end. so, I created Two Processes nam...
I am writing a multi-threaded program using OpenMP in C++. At one point my program forks into many threads, each of which need to add "jobs" to some container that keeps track of all added jobs. Each job can just be a pointer to some object. Basically, I just need the add pointers to some container from several threads at the same tim...
I have a large number of sets of integers, which I have, in turn, put into a vector of pointers. I need to be able to update these sets of integers in parallel without causing a race condition. More specifically. I am using OpenMP's "parallel for" construct. For dealing with shared resources, OpenMP offers a handy "atomic directive," w...
I'd like to start using the Task Parallel Library, as this is the recommended framework going forward for performing asynchronous operations. One thing I haven't been able to find is any means of forcible Abort, such as what Thread.Abort provides. My particular concern is that I schedule tasks running code that I don't wish to complete...
I just wrote my first OpenMP program that parallelizes a simple for loop. I ran the code on my dual core machine and saw some speed up when going from 1 thread to 2 threads. However, I ran the same code on a school linux server and saw no speed-up. After trying different things, I finally realized that removing some useless printf statem...
how to do thread in matlab? i want to run one function on two variables simulatniosly how to do it? ...
Hello Is there an converter from fortran 90 downto fortran 77 ? I have a fortran77 only compiler and want to run NAS Parallel Benchmark (NPB for short) on it. But NPB uses some features of F90, like do enddo, smth else. All features are rather simple. Is there A way to translate NPB to F77 strict language? Tags: fortran parallel conv...