I'm creating and processing a very large data set, with about 34 million data points, and I'm currently storing them in python dictionaries in memory (about 22,500 dictionaries, with 15 dictionaries in each of 1588 class instances). While I'm able to manage this all in memory, I'm using up all of my RAM and most of my swap. I need to b...
I'm looking for a library that can help me build a good XMLRPC server in Python that could run on Windows. The SimpleXMLRPCServer class looks fine but I don't know if it will suit all my needs, since I'd like to be able to connect from multiple clients at the same time. I found this on GitHub, but I don't know if it'll work. Any suggest...
We store objects in XML. Sometimes we update the base objects, then we have to save more data in our files to represent the extra attributes of our objects. How to organize/implement a system to ensure backwards compatibility with old versions of our files? The complicated part comes when looking at several versions at once. Version...
I need to start some threads in a python program. The threads perform a background task which might take a long time, so I don't want to block the main thread waiting on the task to happen. Python provides the ability to 'reap' threads using Thread.join() and Thread.isAlive(). But I don't actually care about finding out when the thread ...
does it take the filename of the .py and then what? ...
I'm trying to write a python script which backs up the database every midnight. The code i am using is below: from subprocess import call call (["mysqldump", "-u", "root", "-p*****", "normalisation", ">", "date_here.sql"]) The first problem i came across is that mysql thinks the ">" is a table when it is not, the query works fine whe...
I'm looking for a way to run tests on command-line utilities written in bash, or any other language. I'd like to find a testing framework that would have statements like setup: command = 'do_awesome_thing' filename = 'testfile' args = ['--with', 'extra_win', '--file', filename] run_command command args test_output_was_...
Is there some kind of enumeration library or method I should use or should I write from scratch with recursion? I'm parsing a JSON tree into an object tree as it happens, and I'd like to replace some nodes with other kinds of objects. E.g: db = {'bigBang' : {'stars': {'planets': {}, 'is_list':true ...
hi all I'm trying to extract the "META" description from a webpage using libxml for python. When it encounters UTF chars it seems to choke and display garbage chars. However when getting the data via a regex I get the unicode chars just fine. Am I doing something wrong with libxml? thanks ''' test encoding issues with utf8 ''' from lx...
Im wrtiting a script which saves the current date and time as a filename but I get an error stating "TypeError: not all arguments converted during string formatting" I am new to Python andmay of missed something obvious. Code below: from subprocess import Popen import datetime today = datetime.date.today() today = str(today) print to...
I have a PyMongo newbie question: If collection is a PyMongo Collection and I use it to save an object with obj = {'foo': 'bar'} collection.insert(obj) then MongoDB automatically generates an _id field for obj; once can confirm this with print obj which yields something like {'foo': 'bar', '_id': ObjectId('4c2fea1d289c7d837e000000...
What is the default working directory for my project? I have several projects under my workspace, and a couple of run configurations. I use os.getcwd() and the directory goes to other project's folder, after deleting all run configurations, the directory goes to eclipse's install folder. How to make the default working directory goes to ...
I recently jumped on a project using Pylons. I'm not familiar with either Python or Pylons, but I haven't had very much trouble getting the hang of things. Pylon projects seem to cache templates indefinitely by default and I can't figure out a way to clear the cached templates (stored by default in /data/templates) except by manually de...
What's the most succinct way of saying, in Python, "Give me dict['foo'] if it exists, and if not, give me this other value bar"? If I were using an object rather than a dictionary, I'd use getattr: getattr(obj, 'foo', bar) but this raises a key error if I try using a dictionary instead (a distinction I find unfortunate coming from Jav...
I know that I can write: foo = 'bar' def update_foo(): global foo foo = 'baz' But do I really need two lines of code there? Python, alas, won't allow me to say global foo = 'baz' I could also mash the two lines together with the unfortunately repetitive global foo; foo = 'baz' Any other shortcuts? I'm on Python 2.6.5, but I'...
I'm processing a list of thousands of domain names from a DNSBL through dig, creating a CSV of URLs and IPs. This is a very time-consuming process that can take several hours. My server's DNSBL updates every fifteen minutes. Is there a way I can increase throughput in my Python script to keep pace with the server's updates? Edit: the sc...
I have read in the documentation that there are 4 or 5 ways in which i can python for web pages. Like With CGI Mod_python : mod_python does have some problems. Unlike the PHP interpreter, the Python interpreter uses caching when executing files, so changes to a file will require the web server to be restarted FastCGI and SCGI mod_wsgi...
This looks like code in the C language, but I am not completely sure ... # define v putchar # define print(x) main(){v(4+v(v(52)-4));return 0;}/* #>+++++++4+[>++++++<-]>++++.----.++++.*/ print(202*2);exit(); #define/*>.@*/exit() ...
I have chosen Python as a langauge to build my ecommerce webiste. The site will contains 1)Logins 2)registration 3)SHop Cart 4)Payment gateway 5)Admin can edit some content pages I have started learning basic python. But i want to build website and i have to start with one framework the web users can vary fromm 100's to 1000's SO ...
In REST based architectures what's the difference between a resource and a method. Is there one? ...