python

Optimizing BeautifulSoup (Python) code

I have code that uses the BeautifulSoup library for parsing, but it is very slow. The code is written in such a way that threads cannot be used. Can anyone help me with this? I am using BeautifulSoup for parsing and than save into a DB. If I comment out the save statement, it still takes a long time, so there is no problem with the dat...

When to use buildout:eggs and when to install via zc.recipe.egg ?

There seem to be more than one way to install eggs into a buildout. Way 1: [buildout] ... eggs = eggname othereggname ... Way 2: [buildout] ... parts = eggs [eggs] recipe = zc.recipe.egg eggs = eggname = othereggname Both ways work. ( variation on way 2 would be to install each requirement as a separate part. ) ...

Handling urllib2's timeout? - Python

Hi folks, I'm using the timeout parameter within the urllib2's urlopen. urllib2.urlopen('http://www.example.org', timeout=1) How do I tell Python that if the timeout expires a custom error should be raised? Any ideas? ...

Log Unittest output to a text file

Hello, I am trying to log the output of tests to a text file. I am using the unittest module and want to log results into a text file instead of the screen. I have some script here to explain what has been tryied so far. This is the test script. import unittest, sys class TestOne(unittest.TestCase): def setUp(self): self....

Why doesn't Python's `re.split()` split on zero-length matches?

One particular quirk of the (otherwise quite powerful) re module in Python is that re.split() will never split a string on a zero-length match, for example if I want to split a string along word boundaries: >>> re.split(r"\s+|\b", "Split along words, preserve punctuation!") ['Split', 'along', 'words,', 'preserve', 'punctuation!'] ins...

python dict function on enumerate object

Hi, If I have an enumerate object x, why does doing the following: dict(x) clear all the items in the enumerate sequence? ...

User Crontab + Python + Random wallpapers = Not working?

I have a python script that correctly sets the desktop wallpaper via gconf to a random picture in a given folder. I then have the following entry in my crontab * * * * * python /home/bolster/bin/change-background.py And syslog correctly reports execution Apr 26 14:11:01 bolster-desktop CRON[9751]: (bolster) CMD (python /home...

Python package name conventions

Is there a package naming convention for Python like Java's com.company.actualpackage? Most of the time I see simple, potentially colliding package names like "web". If there is no such convention, is there a reason for it? What do you think of using the Java naming convention in the Python world? ...

python decorator to modify variable in current scope

Goal: Make a decorator which can modify the scope that it is used in. If it worked: class Blah(): # or perhaps class Blah(ParentClassWhichMakesThisPossible) def one(self): pass @decorated def two(self): pass >>> Blah.decorated ["two"] Why? I essentially want to write classes which can maintain specifi...

py2exe/pyinstaller and DispatchWithEvents

I have a program that uses the win32com library to control iTunes, but have been having some issues getting it to compile into an executable. The problem seems to revolve around using DispatchWithEvents instead of Dispatch. I've created a very simple program to illustrate my problem: import win32com.client win32com.client.gencache.is_re...

How to auto-restart a python script on fail?

This post describes how to keep a child process alive in a BASH script: http://stackoverflow.com/questions/696839/how-do-i-write-a-bash-script-to-restart-a-process-if-it-dies This worked great for calling another BASH script. However, I tried executing something similar where the child process is a Python script, daemon.py which crea...

Instance variables vs. class variables in Python

I have Python classes, of which I need only one instance at runtime, so it would be sufficient to have the attributes only once per class and not per instance. If there would be more than one instance (what won't happen), all instance should have the same configuration. I wonder which of the following options would be better or more "idi...

python 3: ipaddr/netaddr modules

I have got to be doing something wrong here... I am at present trying to validate whether an ip is within a specific subnet utilizing a builtin module. I am using activepython: ActivePython 3.1.2.3 (ActiveState Software Inc.) based on Python 3.1.2 (r312:79147, Mar 22 2010, 12:20:29) [MSC v.1500 32 bit (Intel)] on win32 which has th...

Why would it be necessary to subclass from object in Python?

I've been using Python for quite a while now, and I'm still unsure as to why you would subclass from object. What is the difference between this: class MyClass(): pass And this: class MyClass(object): pass As far as I understand, object is the base class for all classes and the subclassing is implied. Do you get anything fr...

CherryPy and RESTful web api

What's the best approach of creating a RESTful web api in CherryPy? I've been looking around for a few days now and nothing seems great. For Django it seems that are lots of tools to do this, but not for CherryPy or I am not aware of them. Later edit: How should I use Cherrypy to transform a request like /getOrders?account=X&type=Y into...

How to plot non-numeric data in Matplotlib

I wish to plot the time variation of my y-axis variable using Matplotlib. This is no problem for continuously discrete data, however how should this be tackled for non-continuous data. I.e. if I wanted to visualise the times at which my car was stationary on the way to work the x-axis would be time and the y-axis would be comprised of ...

Search engine recommendation for 100 sites of about 4000 pages

I am looking for a search engine that can regularly (daily-ish) scan about 100 pages for changes and index an associated site if changes since the last scan are found. It should be able to handle about 100 sites, each averaging 4000 pages of about 5k average size, each on a different server (but only the one centralized search engine). E...

Passing list and dictionary type parameter with Python

When I run this code def func(x, y, *w, **z): print x print y if w: print w if z: print z else: print "None" func(10,20, 1,2,3,{'k':'a'}) I get the result as follows. 10 20 (1, 2, 3, {'k': 'a'}) None But, I expected as follows, I mean the list parameters (1,2,3) matching *w, and dictionary matching **...

How to get path of stroke using opencv ?

I want to get path stroke from image using opencv. I know how to get contours but I need path of stroke (path that runs through the center of the stroke). Is it possible to get this in opencv ? ...

pyinotify file deletion user

I'm trying to use pyinotify to alert me whenever files are deleted, but I want to know what user deleted the files. Is there a way to find this information? ...