I've decided to learn Python 3. For those that have gone before, what did you find most useful along the way and wish you'd known about sooner?

+20  A: 
  1. I wished I didn't know Java.
  2. More functional programming. (see itertools module, list comprehension, map(), reduce() or filter())
I never wanted to know Java either. I was forced to in college. And since I have learned about it, I dislike it even more.
I learned Pascal, then C, then C++ (sort of), then Java. My understanding was that explicitly typing was the way to go. Until I met Python three years ago. At first it was annoying (I was getting some errors that you usually get at compile time in static languages), but after a while it was very clear that my productivity increased something like 5 fold. Now, the reversal happened: each time I program in Java I hate it for not being Python.
What's wrong with Java?
1. Lack of consistency (Arrays vs Collections?).2. Too many layers of (more than often useless) abstractions (check IO)3. Passing thrown exceptions (throws IOException)4. Lack of typedefs (is inheritance equivalent?)5. Confusing: PriorityBlockingQueue has: poll, take, remove (useless abstractions?).6. Lack of functors and operator redefinition.I can probably enumerate a few more.
PriorityBlockingQueue poll/take/remove do distinct things, and are all useful. Definitely not useless abstractions, and not confusing once you start using them.
Robert Munteanu
Knowing more languages is always better
+4  A: 

Most helpful: Dive Into Python. As a commenter points out, if you're learning Python 3, Dive Into Python 3 is more applicable.

Known about sooner: virtualenv.

Hank Gay
Hmm, he did say "Python 3", and I don't think virtualenv has been ported to Python 3 yet. Still +1, because it's so fantastic. Also, include the link for Dive into Python 3. http://www.diveintopython3.org/
It's not virtualenv, but setuptools that needs to work with python 3. Now that virtualenv can be made to use distribute in place of setupttools, it might work just fine.
+20  A: 

List comprehension (makes a list cleanly):

[x for x in y if x > z]

Generator expansion (same as list comprehension but doesn't evaluate until it is used):

(x for x in y if x > z)
Paul Tarjan
i'd +1 you; but first check those expressions, are you sure meant 'if y==z'?)
good point. fixed
Paul Tarjan

I really like list comprehension and all other semifunctional constructs. I wish I had known those when I was in my first Python project.

+2  A: 

List comprehensions, if you're coming to Python fresh (not from an earlier version).

+9  A: 

Don't have variable names that are types. For example, don't name a variable "file" or "dict"

+13  A: 

Learn how to use iPython It's got Tab completion. View all the elements in your namespace with 'whos'.

After you import a module, it's easy to view the code:

>>> import os
>>> os?? # this display the actual source of the method
>>> help() # Python's interactive help. Fantastic!

Most Python modules are well documented; in theory, you could learn iPython and the rest of what you'd need to know could be learned through the same tool.

iPython also has a debug mode, pdb(). Finally, you can even use iPython as a python enabled command line. The basic UNIX commands work as %magic methods. Any commands that aren't magic command can be executed:

>>> os.system('cp file1 file2')
That last command can be made even easier. In Ipython a leading exclamation point runs the following command in the shell. So it becomes "!cp file1 file2"
`files = !find path/ -type f` sets `files` to the list you'd expect.
+32  A: 

I learned Python back before the 1.5.2 release, so the things that were key for me back then may not be the key things today.

That being said, a crucial thing that took me a little bit to realize, but I now consider crucial: much functionality that other languages would make intrinsic is actually made available by the standard library and the built-ins.

The language itself is small and simple, but until you're familiar with the built-ins and the "core parts" of the standard library (e.g., nowadays, sys, itertools, collections, copy, ...), you'll be reinventing the wheel over and over. So, the more time you invest in getting familiar with those parts, the smoother your progress will be. Every time you have a task you want to do, that doesn't seem to be directly supported by the language, first ask yourself: what built-ins or modules in the standard library will make the task much simpler, or even do it all for me? Sometimes there won't be any, but more often than not you'll find excellent solutions by proceeding with this mindset.

Alex Martelli
collections and itertools are fantastic. I also wish more people knew about optparse. To quote Andrew Perry, "[It's amazing] how 'import optparse' can turn a hackish Python script into something resembling a 'real' piece of software." http://twitter.com/pansapiens/statuses/5438320772
Agreed. All the available functionality has the effect of making Python, for me, a very _large_ language. I don't suppose that I'm aware of even one-tenth of the stuff that it's good to know, and I've been messing around with it since pre-2.0. The trouble is that, unlike C, even if you did want to reinvent the wheel, you don't have easy access to the bits and bytes from within the core lanuage itself, unless I've been missing something (which is quite possible).
I started with python 1.5.2. The thing that helps a lot is to sit down once a day and pick one of the std library modules and learn it. You never get board and you always find something useful.
Steven Graham
Wow, I've just started learning Python, and haven't yet heard of optparse. That is very cool, and incredibly useful for writing scripts.
+3  A: 

Closures. Clean and concise, without having to resort to using a Strategy Pattern unlike languages such as Java

Julson Lim
I'd be interested to know your best recommended link for this. E.g. how about this: http://ynniv.com/blog/2007/08/closures-in-python.html
Craig McQueen
+19  A: 

Two brain-cramping things. One of which doesn't apply to Python 3.

a = 095

Doesn't work. Why? The leading zero is an octal literal. The 9 is not valid in an octal literal.

def foo( bar=[] ):
    bar.append( 1 )
    return bar

Doesn't work. Why? The mutable default object gets reused.

From personal experience, I definitely second the warning about mutable default argument values :-)
Jarret Hardie
FTR: It's the first one that doesn't apply to python 3. octals are represented as 0o95. Much better IMHO.
Regarding the second one: it wouldn't be a problem with a persistent (immutable) data structure.
+9  A: 

Decorators. Writing your own is not something you might want to do right away, but knowing that @staticmethod and @classmethod are available from the beginning (and the difference between what they do) is a real plus.

David Berger

What I really liked: List comprehensions, closures (and high-order functions), tuples, lambda functions, painless bignumbers.

What I wish I had known about sooner: The fact that using Python idioms in code (e.g. list comprehensions instead of loops over lists) was faster.

+3  A: 

If you learn from a good book, it will not only teach you the language, it will teach you the common idioms. The idioms are valuable.

For example, here is the standard idiom for initializing a class instance with a list:

class Foo(object):
    def __init__(self, lst=None):
        if lst is None:
            self.lst = []
            self.lst = lst

If you learn this as an idiom from a book, you don't have to learn the hard way why this is the standard idiom. @S.Lott already explained this one: if you try to make the default initializer be an empty list, the empty list gets evaluated just once (at compile time) and every default-initialized instance of your class gets the same list instance, which was not what was intended here.

Some idioms protect you from non-intended effects; some help you get best performance out of the language; and some are just small points of style, which help other Python fans understand your code better.

I learned out of the book Learning Python and it introduced me to some of the idioms.

Here's a web page devoted to idioms: http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html

P.S. Python code that follows the best-practice Python idioms often is called "Pythonic" code.

I think your `__init__` method's body could be written more idiomatically as `self.lst = lst or []`.
Will McCutchen
Or, depending on your intent, self.lst = list(lst) [and keep the empty array as the default].
@Will McCutchen, why do you think your way is more "idiomatic"? Only the `is` operator can tell exactly whether the default argument was invoked. The way I wrote it is universal: it works for classes that store an object for you, as well as classes there you are just providing a list of values. If you are just providing a list of values, your way works; but if the class is storing an instance, your way discards some provided instances and replaces them with an empty list. If I write `class MyClass(list):` and pass an empty instance of that, I don't want it discarded and replaced.
@chrispy: Your suggestion would work, but I have never seen it done that way anywhere ever. I think it is much more confusing than the standard idiom of using `None`. And see my answer to @Will McCutchen about classes used for storing instances; your answer would also do violence to the stored things. For example, if the user passed in an iterator, yours wouldn't store it; it would exhaust it, and store a list copy of it. My personal preference is to learn an idiom that always works, then always use the idiom, rather than doing the same thing different ways depending on circumstances.
@steveha I'm getting a strong sense of deja-vu here ;)
Heh. :-) Well, at least I'm consistent.
+1  A: 

I wish I knew well a functional language. After playing a bit with Clojure, I realized that lots of Python's functional ideas are borrowed from Lisp or other functional langs

+3  A: 

I implemented plenty of recursive directory walks by hand before I learned about os.walk()

Dan Monego

That multi-core was the future. Still love Python. It's writes a fair bit of my code for me.

-1 Like multicore matters when you've already accepted a hundred-fold slowdown by your choice of language.
100x? You sure about that?
CPython can be as bad as 100x slower than compiled C code, but it's not always that bad for any problem. If you write good Python that lets the built-in features do the heavy lifting, it can be reasonably fast. And I'm looking forward to Unladen Swallow: http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
That the `multiprocessing` library lets you take advantage of multicore hardware.
The GIL still remains a problem for Python. Using that package requires more than just having 2.6. From the Python Website:WarningSome of this package’s functionality requires a functioning shared semaphore implementation on the host operating system. Without one, the multiprocessing.synchronize module will be disabled, and attempts to import it will result in an ImportError.

I wish I'd known right off the bat how to code idiomatically in Python. You can pick up any language you like and start coding in it like it's C, Java, etc. but ideally you'll learn to code in "the spirit" of the language. Python is particularly relevant, as I think it has a definite style of its own.

While I found it a little later in my Python career than I would have liked, this excellent article wraps up many Python idioms and the little tricks that make it special. Several of the things people have mentioned in their answers so far are contained within: Code Like a Pythonista: Idiomatic Python.


Matt Baker
Exact Duplicate.
+6  A: 
  • What enumerate is for.
  • That seq = seq.append(item) and seq = seq.sort() both set seq to None.
  • Using set to remove duplicates.
  • Pretty much everything in the itertools and collections modules.
  • How the * and ** prefixes for function arguments work.
  • How default arguments to functions work internally (i.e. what f.func_defaults is).
  • How (why, really) to design functions so that they are useful in conjunction with map and zip.
  • The role of __dict__ in classes.
  • What import actually does.
Robert Rossney
I'm stunned at #2. :(
Michael Foukarakis
@Michael Foukarakis, don't be stunned. Python is adhering to the principle of command/query separation. If you get a value back, you can be confident you didn't mutate the object; if you mutate the object, you get back `None`. http://en.wikipedia.org/wiki/Command-query_separation
@steveha: Wow, thanks for this very helpful insight! I wonder why the Python Tutorial doesn't point that out. Or maybe it does, and I haven't found it...
Tim Pietzcker
@steveha, @Tim Pietzcker:With some exceptions, like `dict.pop(key)` (mutates and returns).
+6  A: 
  • using help() in the shell on any object, class or path
  • you can run import code; code.interact(local=locals()) anywhere in your code and it will start a python shell at that exact point
  • you can run python -i yourscript.py to start a shell at the end of yourscript.py
Nice! Is there a way to drop into ipython instead?
+3  A: 

One of the coolest things I learned about recently was the commands module:

>>> import commands
>>> commands.getoutput('uptime')
'18:24  up 10:22, 7 users, load averages: 0.37 0.45 0.41'

It's like os.popen or os.system but without all of the DeprecationWarnings.

And let's not forget PDB (Python Debugger):

% python -m pdb poop.py
+2  A: 

Dropping into interactive mode in IPython

from IPython.Shell import IPShellEmbed
ipshell = IPShellEmbed()
+1  A: 

When I started with python, started out with main methods from the examples. This was because I didn't know better, after that I found this on how to create a better main method.

+2  A: 

That a tuple of a single item must end with a comma, or it won't be interpreted as a tuple.

pprint() is very handy (yes, 2 p's)

reload() is useful when you're re-testing a module while making lots of rapid changes to a dependent module.

And learn as many common "idioms" as you can, otherwise you'll bang your head looking for a better way to do something, when the idiom really is regarded as the best way (e.g. ugly expressions like ' '.join(), or the answer to why there is no isInt(string) function.... the answer is you can just wrap the usage of a "possible" integer with a try: and then catch the exception if it's not a valid int. The solution works well, but it sounds like a terrible answer when you first encounter it, so you can waste a lot of time convincing yourself it really is a good approach.

Those are some things that wasted several hours of my time to determine that my first draft of some code which felt wrong, really was acceptable.

Readings from python.org:

http://wiki.python.org/moin/BeginnerErrorsWithPythonProgramming http://wiki.python.org/moin/PythonWarts

+1  A: 

Sequential imports overwrite:

If you import two files like this:

from foo import *
from bar import *

If both foo.py and bar.py have a function named fubar(), having imported the files this way, when you call fubar, fubar as defined in bar.py will be executed. The best way to avoid this is to do this:

import foo
import bar

and then call foo.fubar or bar.fubar. This way, you ALWAYS know which file's definition of fubar will be executed.

+1  A: 

Pretty printing:

>>> print "%s world" %('hello')
hello world

%s for string

%d for integer

%f for float

%.xf for exactly x many decimal places of a float. If the float has lesser decimals that indicated, then 0s are added

This isn't python 3.
Oops! I forgot to check this against Python 3. I'm working with 2.6. Sorry those who were misled by this
This is really just string formatting. "pprint" is for pretty printing.
Roger Pate
When I was taught this, it was taught to me as "pretty printing". I agree with you that there is "pprint", but as for the nomenclature, this is how I was taught, which is why I called it pretty printing. Sorry if it is misleading
+2  A: 

Maybe a touch more advanced, but I wish I'd known that you don't use threads to take advantage of multiple cores in (C)python. You use the multiprocessing library.

+2  A: 

Tab completion and general readline support, including histories, even in the regular python shell.

$ cat ~/.pythonrc.py 
#!/usr/bin/env python

    import readline
except ImportError:
    print("Module readline not available.")
    import rlcompleter
    readline.parse_and_bind("tab: complete")

    import os
    histfile = os.path.join(os.environ["HOME"], ".pyhist")
    except IOError:
    import atexit
    atexit.register(readline.write_history_file, histfile)
    del os, histfile

and then add a line to your .bashrc

export PYTHONSTARTUP=~/.pythonrc.py

These two things lead to an exploratory programming style of "it looks like this library might do what I want", so then I fire up the python shell and then poke around using tab-completion and the help() command until I find what I need.

Generators and list comprehensions are more useful than you might think. Don't just ignore them.

I do not recommend including `del os`. Scripts that use that module will choke. http://igotgenes.blogspot.com/2009/01/tab-completion-and-history-in-python.html
That hasn't been my experience, but I can see why you might think that. I wonder why our experiences have been different?