I have a "manager" process on a node, and several worker processes. The manager is the actual server who holds all of the connections to the clients. The manager accepts all incoming packets and puts them into a queue, and then the worker processes pull the packets out of the queue, process them, and generate a result. They send the resu...
I have a python application that grabs a collection of data and for each piece of data in that collection it performs a task. The task takes some time to complete as there is a delay involved. Because of this delay, I don't want each piece of data to perform the task subsequently, I want them to all happen in parallel. Should I be using ...
We have been told that Google Chrome runs each tab in a separate process. Therefore a crash in one tab would not cause problems in the other tabs. AFAIK, multi-processes are mostly used in programs without a GUI. I have never read any technique that could embed multiple GUI processes into a single one. How does Chrome do that? I am a...
I'm making a python script that needs to do 3 things simultaneously. What is a good way to achieve this as do to what i've heard about the GIL i'm not so lean into using threads anymore. 2 of the things that the script needs to do will be heavily active, they will have lots of work to do and then i need to have the third thing reporting ...
I set up a process that read a queue for incoming urls to download but when urllib2 open a connection the system hangs. import urllib2, multiprocessing from threading import Thread from Queue import Queue from multiprocessing import Queue as ProcessQueue, Process def download(url): """Download a page from an url. url [str]: url...
My application hangs when trying to open a concurrent data store (CDB) database for reading: #0 0x0000003ad860b309 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007ffff7ce67de in __db_pthread_mutex_lock (env=0x610960, mutex=100) at /home/steve/ldm/package/src/Berkeley-DB/dist/../mutex/mut_pthread.c:318...
Hi everyone, We have an application that parts of it are in .net, c++ usermode and C++ drivers. The application is divided into several executables that run on demand and communication with each other using LPC(the processes run in different sessions(winlogon)). Currently We have a home written logging service to which .net and c++ use...
I have programmed a simple app that every X minutes checks if an image has changed in several websites and downloads it. It's very simple: downloads image header, make some CRC checks, downloads the file, stores in a MySQL database some data about each image and process next item... This process takes about 1 minute to complete. The pr...
Few years ago, under Windows environment, I did some testing, by letting multiple instance of CPU computation intensive + Memory access intensive + I/O access intensive application run. I develop 2 versions. One is running under multi-process, another is running under multi-thread. I found the performance is much better for multi-proces...
In the code below, I am considering using mutli-threading or multi-process for fetching from url. I think pools would be ideal, Can anyone help suggest solution.. Idea: pool thread/process, collect data... my preference is process over thread, but not sure. import urllib URL = "http://download.finance.yahoo.com/d/quotes.csv?s=%s&am...
I am writing multi process fibonacci number calculator , I have a file that keeps track of the fibonacci numbers , first process open the file and write first fibonacci numbers (0 and 1 ), then do fork and its child process read the last two numbers add them up and write the next into file and close the file and fork again this process c...
Hi, I have one multprocess demo here, and I met some problems with it. Researched for a night, I cannot resolve the reason. Any one can help me? I want to have one parent process acts as producer, when there are tasks come, the parent can fork some children to consume these tasks. The parent monitors the child, if any one exits with exc...
Hi, Im writing a program that manage muti-processes. This is what I have done and its working great! but now, I want to send messages from the child processes to the parent process and viceversa (from the parent to the childs), do you know the best way? Do you know if what I have done is the proper way for what I want (send messages, or ...