Why the "mutable default argument fix" syntax is so ugly, asks python newbie

Question

^{Now following my series of "python newbie questions" and based on another question.}

Prerogative

Go to http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#other-languages-have-variables and scroll down to "Default Parameter Values". There you can find the following:

def bad_append(new_item, a_list=[]):
    a_list.append(new_item)
    return a_list

def good_append(new_item, a_list=None):
    if a_list is None:
        a_list = []
    a_list.append(new_item)
    return a_list

There's even an "Important warning" on python.org with this very same example, tho not really saying it's "better".

One way to put it

So, question here is: why is the "good" syntax over a known issue ugly like that in a programming language that promotes "elegant syntax" and "easy-to-use"?

^edit:

Another way to put it

I'm not asking why or how it happens (thanks Mark for the link).

I'm asking why there's no simpler alternative built-in the language.

I think a better way would probably being able to do something in the def itself, in which the name argument would be attached to a "local", or "new" within the def, mutable object. Something like:

def better_append(new_item, a_list=immutable([])):
    a_list.append(new_item)
    return a_list

I'm sure someone can come with a better syntax, but I'm also guessing there must be a very good explanation to why this hasn't been done.

Answer 1

This is called the 'mutable defaults trap'. See: http://www.ferg.org/projects/python_gotchas.html#contents_item_6

Basically, a_list is initialized when the program is first interpreted, not each time you call the function (as you might expect from other languages). So you're not getting a new list each time you call the function, but you're reusing the same one.

I guess the answer to the question is that if you want to append something to a list, just do it, don't create a function to do it.

This:

>>> my_list = []
>>> my_list.append(1)

Is clearer and easier to read than:

>>> my_list = my_append(1)

In the practical case, if you needed this sort of behavior, you would probably create your own class which has methods to manage it's internal list.

Answer 2

Probably you should not define these two functions as good and bad. You can use the first one with list or dictionaries to implement in place modifications of the corresponding objects. This method can give you headaches if you do not know how mutable objects work but given you known what you are doing it is OK in my opinion.

So you have two different methods to pass parameters providing different behaviors. And this is good, I would not change it.

Answer 3

Default arguments are evaluated at the time the def statement is executed, which is the probably the most reasonable approach: it is often what is wanted. If it wasn't the case, it could cause confusing results when the environment changes a little.

Differentiating with a magic local method or something like that is far from ideal. Python tries to make things pretty plain and there is no obvious, clear replacement for the current boilerplate that doesn't resort to messing with the rather consistent semantics Python currently has.

Answer 4

What if you were not talking about lists, but about AwesomeSets, a class you just defined? Would you want to define ".local" in every class?

class Foo(object):
    def get(self):
        return Foo()
    local = property(get)

could possibly work, but would get old really quick, really soon. Pretty soon, the "if a is None: a = CorrectObject()" pattern becomes second nature, and you won't find it ugly -- you'll find it illuminating.

The problem is not one of syntax, but one of semantics -- the values of default parameters are evaluated at function definition time, not at function execution time.

Answer 5

The extremely specific use case of a function that lets you optionally pass a list to modify, but generates a new list unless you specifically do pass one in, is definitely not worth a special-case syntax. Seriously, if you're making a number of calls to this function, why ever would you want to special-case the first call in the series (by passing only one argument) to distinguish it from every other one (which will need two arguments to be able to keep enriching an existing list)?! E.g., consider something like (assuming of course that betterappend did something useful, because in the current example it would be crazy to call it in lieu of a direct .append!-):

def thecaller(n):
  if fee(0):
    newlist = betterappend(foo())
  else:
    newlist = betterappend(fie())
  for x in range(1, n):
    if fee(x):
      betterappend(foo(), newlist)
    else:
      betterappend(fie(), newlist)

this is simply insane, and should obviously be, instead,

def thecaller(n):
  newlist = []
  for x in range(n):
    if fee(x):
      betterappend(foo(), newlist)
    else:
      betterappend(fie(), newlist)

always using two arguments, avoiding repetition, and building much simpler logic.

Introducing special-case syntax encourages and supports the special-cased use case, and there's really not much sense in encouraging and supporting this extremely peculiar one -- the existing, perfectly regular syntax is just fine for the use case's extremely rare good uses;-).

Answer 6

I think you're confusing elegant syntax with syntactic sugar. The python syntax communicates both approaches clearly, it just happens that the correct approach appears less elegant (in terms of lines of syntax) than the incorrect approach. But since the incorrect approach, is well...incorrect, it's elegance is irrelevant. As to why something like you demonstrate in better_append is not implemented, I would guess that There should be one-- and preferably only one --obvious way to do it. trumps minor gains in elegance.

Answer 7

This is better than good_append(), IMO:

def ok_append(new_item, a_list=None):
    return a_list.append(new_item) if a_list else [ new_item ]

You could also be extra careful and check that a_list was a list...

Answer 8

I've edited this answer to include thoughts from the many comments posted in the question.

The example you give is flawed. It modifies the list that you pass it as a side effect. If that's how you intended the function to work, it wouldn't make sense to have a default argument. Nor would it make sense to return the updated list. Without a default argument, the problem goes away.

If the intent was to return a new list, you need to make a copy of the input list. Python prefers that things be explicit, so it's up to you to make the copy.

def better_append(new_item, a_list=[]): 
    new_list = list(a_list)
    new_list.append(new_item) 
    return new_list 

For something a little different, you can make a generator that can take a list or a generator as input:

def generator_append(new_item, a_list=[]):
    for x in a_list:
        yield x
    yield new_item

I think you're under the misconception that Python treats mutable and immutable default arguments differently; that's simply not true. Rather, the immutability of the argument makes you change your code in a subtle way to do the right thing automatically. Take your example and make it apply to a string rather than a list:

def string_append(new_item, a_string=''):
    a_string = a_string + new_item
    return a_string

This code doesn't change the passed string - it can't, because strings are immutable. It creates a new string, and assigns a_string to that new string. The default argument can be used over and over again because it doesn't change, you made a copy of it at the start.