There's not a lot of difference. If you start with Python 3 and later on have occasion to work on Python 2.x code the differences will be minor - an annoyance at worst. If you start with Python 2 you may find yourself having to port to Python 3 at some point. However, Python 2 will be around for quite a while yet.
Start with whatever is current. 3 might be a little bleeding edge for a few months, so it might be easier to start with 2.5 or 2.6 and move to 3.0 in a year or so. Note that I have had Python 2.2, 2.3 or 2.4 on machines long after later versions came out in order to deal with bugs or compatibility issues.
For example, MakePy would break on Excel in version 2.4 as it generated a file that exceeded a size limit on Python 2.4. I used 2.3 for a while until the bug was fixed.
I still have 2.4 on a machine now as I haven't had any need to upgrade it.