python

Speeding up parsing of HUGE lists of dictionaries - Python

Hi folks, I'm using MongoDB an nosql database. Basically as a result of a query I have a list of dicts which themselves contains lists of dictionaries... which I need to work with. Unfortunately dealing with all this data within Python can be brought to a crawl when the data is too much. I have never had to deal with this problem, a...

a StringIO like class, that extends django.core.files.File

class MyModel(models.Model) image = models.FileField(upload_to="blagh blagh...") #more spam... I have a file in memory and I want to save it via Django FileField save method, like this: photo.image.save(name, buffer) # second arg should be django File I've tried to use StringIO, but it doesn't extend django.core.files.File and thu...

AttributeError: Problem in python

Here is a brief summary of my aims. I have a list of data in the data text file that are basically names or identifiers. The list of names is all on one line and seperated by a space. I want to make each data a seperate line. These data are identifiers. If for instance one name from the original data text file in also present in the big ...

Python: How to override data attributes in method calls?

My question is how to use data attributes in a method but allow them to be overridden individually when calling the method. This example demonstrates how I tried to do it: class Class: def __init__(self): self.red = 1 self.blue = 2 self.yellow = 3 def calculate(self, red=self.red, blue=self.blue, yello...

Append elements with list comprehension

Having a list like this: ['foo','spam','bar'] is it possible, using list comprehension, to obtain this list as result? ['foo','ok.foo', 'spam', 'ok.spam', 'bar', 'ok.bar'] ...

Django: How to find out whether RawQuetySet has rows ? There's no count() method

Hello, I think the title speaks for itself. I have a complex query with a subquery, but sometimes it returns no values, which is absolutely normal. But I can not prevent the ValueError message, cuz I am not able to find out whether RawQuerySet is empty or not. The RQS object is always present, but if I try to access it's first row resul...

IronPython: Is there an alternative to significant whitespace?

For rapidly changing business rules, I'm storing IronPython fragments in XML files. So far this has been working out well, but I'm starting to get to the point where I need more that just one-line expressions. The problem is that XML and significant whilespace don't play well together. Before I abandon it for another language, I would l...

What is the proper method of printing Python Exceptions?

except ImportError as xcpt: print "Import Error: " + xcpt.message Gets you a deprecation warning in 2.6 because message is going away. Stackoverflow How should you be dealing with ImportError? (Note, this is a built-in exception, not one of my making....) ...

How would an irc bot written in tcl stack up against a python/node.js clone?

I believe eggdrop is the most active/popular bot and it's written in tcl ( and according to wiki the core is C but I haven't confirmed that ). I'm wondering if there would be any performance benefit of recoding it's functionality in node.js or Python, in addition to making it more accessible since Python and JS are arguably more popular...

How can I search a text in some website (not from the source) with Python?

I want to make an "autofill" that works like this: The site contains the Name and down the Name has the field to write it. So, I want to make a program the search the "Name" in the website, and before that, click down the "Name" and write the name (easy part)... How can I search for it? I can't use the site forms, because, everysite ...

Unit testing: More Actions than Expected after calling addAction Method

Here is my class: class ManagementReview(object): """Class describing ManagementReview Object. """ # Class attributes id = 0 Title = 'New Management Review Object' fiscal_year = '' region = '' review_date = '' date_completed = '' prepared_by = '' __goals = [] # List of <ManagementReviewGoals...

Strip the last character sent by JavaScript through websockets to Python

I'm currently trying out websockets, creating a client in JavaScript and a server in Python. I'm stuck on a simple problem, though: when I send something from the client to the server it always contains a special ending character, but I don't know how to remove it. I've tried data[:-1] thinking that would get rid of it, but it didn't. ...

Django aggregate and grouping : code cleanliness problem ...

Hello ! Here is a small problem I am having. 3 very simple models : >>> class Instrument(models.Model): ... name = models.CharField(max_length=100) ... >>> class Musician(models.Model): ... instrument = models.ForeignKey(Instrument) ... >>> class Song(models.Model): ... author = models.ForeignKey(Musician) I would l...

Django: limiting model data

Hi there! I'm searching in a way to limit the queryset which I can get through a model. Suppose I have the following models (with dependencies): Company |- Section | |- Employee | |- Task | `- more models... |- Customer | |- Contract | |- Accounts | `- other great models ... `- some more models... It should be n...

import matplotlib._png fails

Hi, I have a problem getting matplotlib 1.0.0 work in Python 2.5.2 (Ubuntu 10.04) See attached console output. Any ideas appreciated. Thanks Python 2.5.2 (r252:60911, Jul 23 2010, 23:04:44) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> import scipy >>> import matplotli...

Python for-loop counter error

Hello! I am attempting to script a short code to figure out the number of days it takes to reach a given principal in the bank due to daily interest. Using my code below does not yield any errors when run in IDLE, but the counter returns 0. Any ideas what I missed? def main(): # irrelevant code elided by msw, Bal, Int and Tar are nu...

How to break a large CSV data file into individual data files?

I have a CSV file the first row of which contains the variables names and the rest of the rows contains the data. What's a good way to break it up into files each containing just one variable in Python? Is this solution going to be robust? E.g. what if the input file is 100G in size? I am trying to perform a divide conquer strategy but i...

Python: Unpacking an inner nested tuple/list while still getting its index number

I am familiar with using enumerate(): >>> seq_flat = ('A', 'B', 'C') >>> for num, entry in enumerate(seq_flat): print num, entry 0 A 1 B 2 C I want to be able to do the same for a nested list: >>> seq_nested = (('A', 'Apple'), ('B', 'Boat'), ('C', 'Cat')) I can unpack it with: >>> for letter, word in seq_nested: pr...

Resuming a large file write in Python

I have a big file transfer (say 4gb or so) and rather than using shutil, I'm just opening and writing it the normal file way so I can include a progress percentage as it moves along. It then occurred to me to try to attempt to resume the file write, if for some reason it borked out during the process. I haven't had any luck though. I p...

certain utf characters do not show up on browsers and fails python script

Hi All, I generated a SQL script from a C# application on Windows 7. The name entries have utf8 characters. It works find on Windows machine where I use a python script to populate the db. Now the same script fails on Linux platform complaining about those special characters. Similar things happened when I generated XML file containing...