python

Intelligent screen scraping using different proxies and user-agents randomly?

I want to download few HTML pages from http://abc.com/view_page.aspx?ID= The ID is from an array of different numbers. I would be interested in visiting multiple instances of this URL and saving the file as [ID].HTML using different proxy IP/ports. I want to use different user-agents and I want to randomize the wait times before each...

Python date string to date object

How do I convert a string to a date object in python? The string would be: "24052010" (corresponding to the format: "%d%m%Y") I don't want a datetime.datetime object, but rather a datetime.date I suspect that I'm asking a trivial question but I searched and couldn't find it neither on stackoverflow nor on google. ...

regular expression search in python

Hello all, I am trying to parse some data and just started reading up on regular Expressions so I am pretty new to it. This is the code I have so far String = "MEASUREMENT 3835 303 Oxygen: 235.78 Saturation: 90.51 Temperature: 24.41 DPhase: 33.07 BPhase: 29.56 RPhase: 0.00 BAmp: 368.57 BPot: ...

Problem upgrading YouTube session token with the YouTube Python API

I am having trouble upgrading my session token in google app engine if my user is not logged into my application using the google accounts user api, if the user is currently logged in then it functions perfectly. If not then i am getting this error: Traceback (most recent call last): File "/base/python_runtime/python_lib/versions/1/...

How to add http headers in suds 0.3.6?

Hi everyone, I have an application in python 2.5 which sends data through suds 0.3.6. The problem is that the data contains non-ascii characters, so I need the following header to exist in the soap message: Content-Type="text/html; charset="utf-8" and the header that exists in the SOAP message is just: Content-Type="text/html" I kn...

OperationalError "unable to open database file" processing query results with SQLAlchemy and SQLite3

I'm running into this little problem that I hope is just a dumb user error. It looks like some sort of a size limit with a query to a SQLite database. I managed to reproduce the issue with an in-memory DB and a simple script shown below. I can make it work by either reducing the number of records in the DB; or by reducing the size of eac...

How do I retrieve program output in Python?

I'm not a Perl user, but from this question deduced that it's exceedingly easy to retrieve the standard output of a program executed through a Perl script using something akin to: $version = `java -version`; How would I go about getting the same end result in Python? Does the above line retrieve standard error (equivalent to C++ s...

Pausing a process?

Is there a way to pause a process (running from an executable) so that it stops the cpu load while it's paused, and waits till it's unpaused to go on with its work? Possibly in python, or in some way accessible by python. ...

twisted reactor stops too early

I'm doing a batch script to connect to a tcp server and then exiting. My problem is that I can't stop the reactor, for example: cmd = raw_input("Command: ") # custom factory, the protocol just send a line reactor.connectTCP(HOST,PORT, CommandClientFactory(cmd) d = defer.Deferred() d.addCallback(lambda x: reactor.stop()) reactor.c...

read subprocess stdout line by line

My python script uses subprocess to call a linux utility that is very noisy. I want to store all of the output to a log file, but only show some of it to the user. I thought the following would work, but the output does show up in my application until the utility has produced a significant amount of output. #fake_utility.py, just gene...

How to create a non-persistent Elixir/SQLAlchemy object?

Hi, because of legacy data which is not available in the database but some external files, I want to create a SQLAlchemy object which contains data read from the external files, but isn't written to the database if I execute session.flush() My code looks like this: try: return session.query(Phone).populate_existing().filter(Phone.ma...

Is possible to auto-import a module from a diferent subfolder in other subfolder?

I have a kind of plugin system, with this layout: Python SDK Plugins Plugin1 Plugin2 All 3 have a __init__.py file. I wonder if is possible to be able to do import SDK from any plugin (as if SDK was in the site-packages folder). I'm in a situation where need to deploy, update, delete, add or change SDK files or any of the plug...

How to "signal" interested child processes (without signals)?

I'm trying to find a good and simple method to signal child processes (created through SocketServer with ForkingMixIn) from the parent process. While Unix signals could be used, I want to avoid them since only children who are interested should receive the signal, and it would be overkill and complicated to require some kind of registra...

How to interface Ruby with Mercurial (Python)?

I'm hoping to integrate Mercurial into my Rails site. I want to run basic commands like status, summary, log, and maybe even clone. I'd like to interface directly with the Mercurial libraries and avoid doing any command line parsing via regex. Mercurial is written in Python; I am using Ruby. So, I need Mercurial Ruby bindings. It seems,...

How can I do DNS lookups in Python, including referring to /etc/hosts?

dnspython (http:/www.dnspython.org) will do my DNS lookups very nicely, but it entirely ignores the contents of /etc/hosts. Is there a python library call which will do the right thing? ie check first in /etc/hosts, and only fallback to dns lookups otherwise? ...

How to compile Python scripts for use in FORTRAN?

Hello, Although I found many answers and discussions about this question, I am unable to find a solution particular to my situation. Here it is: I have a main program written in FORTRAN. I have been given a set of python scripts that are very useful. My goal is to access these python scripts from my main FORTRAN program. Currently, I s...

GTK Progressbar pulsing python

How can I get a Progressbar to "pulse" while another function is run? ...

Routing Skype call to another Voip company

Hello, As my project to do over this summer I would like to create a program that answers a Skype call using the Skype API and allows a user to connect to another VOIP provider (through SIP) and make calls by dialling through the client callers Skype application. I understand that the Skype API allows me to answer and receive keypad in...

Equivalent to F#’s Seq.scan() method in Python?

is there a function like a F#'s Seq.scan() in python? i want to do some cumsum() or cumproduct() kind of things without looping. ...

Scipy-Cluster installation for python 2.6

Hi, Has anyone used scipy-cluster for python? I am trying to compile its source code with python 2.6 but I get some irrelevant errors. has someone had the same problem? ...