What's a good way to keep track of class instance variables in Python?

Question

I'm a C++ programmer just starting to learn Python. I'd like to know how you keep track of instance variables in large Python classes. I'm used to having a .h file that gives me a neat list (complete with comments) of all the class' members. But since Python allows you to add new instance variables on the fly, how do you keep track of them all?

I'm picturing a scenario where I mistakenly add a new instance variable when I already had one - but it was 1000 lines away from where I was working. Are there standard practices for avoiding this?

Edit: It appears I created some confusion with the term "member variable." I really mean instance variable, and I've edited my question accordingly.

Answer 1

The easiest is to use an IDE. PyDev is a plugin for eclipse.

I'm not a full on expert in all ways pythonic, but in general I define my class members right under the class definition in python, so if I add members, they're all relative.

My personal opinion is that class members should be declared in one section, for this specific reason.

Local scoped variables, otoh, should be defined closest to when they are used (except in C--which I believe still requires variables to be declared at the beginning of a method).

Answer 2

First of all: class attributes, or instance attributes? Or both? =)

Usually you just add instance attributes in __init__, and class attributes in the class definition, often before method definitions... which should probably cover 90% of use cases.

If code adds attributes on the fly, it probably (hopefully :-) has good reasons for doing so... leveraging dynamic features, introspection, etc. Other than that, adding attributes this way is probably less common than you think.

Answer 3

It sounds like you're talking about instance variables and not class variables. Note that in the following code a is a class variable and b is an instance variable.

class foo:
  a = 0 #class variable

  def __init__(self):
    self.b = 0 #instance variable

Regarding the hypothetical where you create an unneeded instance variable because the other one was about one thousand lines away: The best solution is to not have classes that are one thousand lines long. If you can't avoid the length, then your class should have a well defined purpose and that will enable you to keep all of the complexities in your head at once.

Answer 4

Instance variables should be initialized in the class's __init__() method. (In general)

If that's not possible. You can use __dict__ to get a dictionary of all instance variables of an object during runtime. If you really need to track this in documentation add a list of instance variables you are using into the docstring of the class.

Answer 5

A documentation generation system such as Epydoc can be used as a reference for what instance/class variables an object has, and if you're worried about accidentally creating new variables via typos you can use PyChecker to check your code for this.

Answer 6

Consider using slots.

For example:

   class Foo:
     __slots__ = "a b c".split()
   x = Foo()
   x.a =1    # ok
   x.b =1    # ok
   x.c =1    # ok
   x.bb = 1  # will raise "AttributeError: Foo instance has no attribute 'bb'"

It is generally a concern in any dynamic programming language -- any language that does not require variable declaration -- that a typo in a variable name will create a new variable instead of raise an exception or cause a compile-time error. Slots helps with instance variables, but doesn't help you with, module-scope variables, globals, local variables, etc. There's no silver bullet for this; it's part of the trade-off of not having to declare variables.

Answer 7

I would say, the standard practice to avoid this is to not write classes where you can be 1000 lines away from anything!

Seriously, that's way too much for just about any useful class, especially in a language that is as expressive as Python. Using more of what the Standard Library offers and abstracting away code into separate modules should help keeping your LOC count down.

The largest classes in the standard library have well below 100 lines!

Answer 8

This is a common concern I hear from many programmers who come from a C, C++, or other statically typed language where variables are pre-declared. In fact it was one of the biggest concerns we heard when we were persuading programmers at our organization to abandon C for high-level programs and use Python instead.

In theory, yes you can add instance variables to an object at any time. Yes it can happen from typos, etc. In practice, it rarely results in a bug. When it does, the bugs are generally not hard to find.

As long as your classes are not bloated (1000 lines is pretty huge!) and you have ample unit tests, you should rarely run in to a real problem. In case you do, it's easy to drop to a Python console at almost any time and inspect things as much as you wish.

Answer 9

pylint can statically detect attributes that aren't detected in __init__, along with many other potential bugs.

I'd also recommend writing unit tests and running your code often to detect these types of "whoopsie" programming mistakes.

Answer 10

It seems to me that the main issue here is that you're thinking in terms of C++ when you're working in python.

Having a 1000 line class is not a very wise thing anyway in python, (I know it happens alot in C++ though),

Learn to exploit the dynamism that python gives you, for instance you can combine lists and dictionaries in very creative ways and save your self hundreds of useless lines of code.

For example, if you're mapping strings to functions (for dispatching), you can exploit the fact that functions are first class objects and have a dictionary that goes like:

d = {'command1' : func1, 'command2': func2, 'command3' : func3}
#then somewhere else use this list to dispatch
#given a string `str`
func = d[str]
func() #call the function!

Something like this in C++ would take up sooo many lines of code!