I have the following dictionary ->
key : (time,edge_list)
Now I want to increment all time values by 1. How do I do that?
dict_list = dict(key:(time+1,edge_list) for key:(time,edge_list) in dict_list)
views:
98answers:
2
+6
A:
dict((key, (time + 1, edge_list)) for (key, (time, edge_list)) in somedict.iteritems())
Ignacio Vazquez-Abrams
2010-02-18 08:12:31
I'd just use "items" rather than "iteritems" - there's no need for the iterator overhead in this case.
Steve314
2010-02-18 08:16:50
@Steve314: Except that you already have iterator overhead from the genex anyway.
Ignacio Vazquez-Abrams
2010-02-18 08:18:33
Theres always overhead - but I don't buy that if you have one overhead you may as well add another. Of course there may be something I don't know about the Python interpreter that genuinely means there's no difference here.
Steve314
2010-02-18 08:34:36
items() creates a new list, which seems like a lot more overhead to me than creating an iterator over an existing structure.
Paul McGuire
2010-02-18 08:55:12
@Paul I agree - I've never heard of iterators causing overhead compared to copying an entire container. In Python3.x, dict.items returns an iterator anyway.
James Hopkin
2010-02-18 10:55:36
Python3 doesn't have `dict.iteritems` so most times `dict.items` is going to be more portable. Of course it is still worth using iteritems for Python2 in cases where there is a large difference in performance.
gnibbler
2010-02-18 11:00:07
@gnibbler, Writing code that works with with Python 2 and 3 is silly, especially for such trivial things. `2to3` will easily change your `iteritems` call into its close Py3k equivalent `items`.
Mike Graham
2010-02-18 16:16:38
@Steve314, When you iterate over something in Python, Python calls `iter(your_object)` to get an iterator object. Using `iteritems` just gets one directly rather than making a list and using its.
Mike Graham
2010-02-18 16:21:03
@Mike - thinking about it, you're probably right - it makes no difference since there's an iterator either way.
Steve314
2010-02-19 04:02:23
@Paul - a non-iterator solution is often faster because it doesn't have to repeatedly do context-switches. Basically, using a list trades a small amount of very efficiently used memory (accessed linearly, in a cache friendly way) for the CPU cycles from the overhead of creating, maintaining and then cleaning up a coroutine (a kind of non-multithreading thread). And when you create that coroutine, it's context data structure (stack etc) can easily be bigger than the list would have been anyway.
Steve314
2010-02-19 04:15:17
@Steve314 - are you saying 'iterator' but thinking 'generator'? An iterator is an object with a next() method, a generator is a callable that yields values when called successively until it returns. From the caller's standpoint, they can look very similar, but I've never heard 'iterator' and 'coroutine' in the same sentence. Here is a good reference discussing the two: http://heather.cs.ucdavis.edu/~matloff/Python/PyIterGen.pdf
Paul McGuire
2010-02-19 04:42:39
@Paul - "iterator", "generator" and "coroutine" are terms defined in computer science generally, not just in Python. The Python-specific details are beside the point - the principles are the same. The "context" isn't a native processor stack, register set etc in any case since Python isn't a native compiler, but whether its a method with yields (that Python converts into an object with a "next" method) or purely an object with a "next" method, there's still creation, context-switch and cleanup overhead, and the abstraction is still a co-routine.
Steve314
2010-02-19 08:12:17
@Paul (cont.) - The same applies to a C++ iterator/cursor pattern implementing a traversal of a binary tree. The intent is the same as having a co-routine that does a simple recursive traversal. In fact, the class *is* an implementation of that co-routine, just as much as if written in a language that had "coroutine" declarations. It's the same abstraction with the same basic issues, hidden behind different language.
Steve314
2010-02-19 08:16:57
@Paul (cont.) - of course from the mention of a "stack" in my original comment, which is not needed for iteritems over a dictionary, clearly I was slightly confused - but I think I managed a pretty good save in the circumstances, eh ;-)
Steve314
2010-02-19 08:30:05
@Steve314 - I'm all for keeping in mind that our beloved Python features also have meaning beyond just our own Python corner of the world. But your *original* comment, recommending using items() instead of iteritems() in the name of avoiding the overhead of an iterator seems to me to run counter to the conventional Python wisdom as I understand it. You are the first person I have heard suggest that building a list for iterating over a dict has a performance/overhead reduction benefit over making an iterator, under *any* circumstances. Aside from the generic concepts of "iterator", et al.,...
Paul McGuire
2010-02-19 13:58:02
@Steve314 (cont.) ...your *original* comment was a recommendation to avoid iterator overhead by creating a list, which sounded to me like a language-specific performance-enhancing technique. I'm certainly open to challenging the Generally Accepted Practice, but at the moment, in this language, your suggestion runs counter to it. I'm sure that some timing evidence illustrating your position would be of general interest.
Paul McGuire
2010-02-19 13:58:41
@Steve314 (cont.) - I'm also aware that there are some circumstances where you *must* create a list copy instead of iterating over the original, but since your *original* comment was oriented to overhead avoidance, I'm assuming that we are talking about cases where either option is open to us.
Paul McGuire
2010-02-19 13:59:53
@Paul - I'm standing by the context-switch overheads. An iterator still has to save its state before returning a value, and restore that state in order to start working on deriving the next one. Whether that state is some special data structure for generators, or just member variables in a class instance, the total work to handle all items is still more than just doing the whole lot in a simple loop in one go.
Steve314
2010-02-19 14:22:02
Of course building a list has its own overheads. Soul-searching a bit, I guess the real reason for the objection is that I still think of lists as the default case and iterators/generators as the (potentially premature) optimisation. If you're going to make the effort to optimise, there has to be some substantial benefit to it. In this case, the "optimisation effort" is just 4 extra characters of typing, but it's still enough to trigger my kneejerk reaction.
Steve314
2010-02-19 14:30:11
+5
A:
>>> d={"key" : (100,"edge_list")}
>>> for i,(time,edge_list) in d.items():
... d[i] = time+1, edge_list
...
>>> d
{'key': (101, 'edge_list')}
gnibbler
2010-02-18 08:16:08