tags:

views:

1190

answers:

2

[please also read the edit below]

I'm experimenting with MATLAB OOP, as a start I mimicked my C++'s Logger classes and I'm putting all my string helper functions in a String class, thinking it would be great to be able to do things like a + b, a == b, a.find( b) instead of strcat( a b), strcmp( a, b), retrieve first element of strfind( a, b), etc.

I put the above things to use and immediately noticed a drastic slowdown. Am I doing it wrong (which is certainly possible as I have rather limited MATLAB experience), or does MATLAB's OOP just introduce a lot of overhead?

Here's the simple test I did for string, basically just appending a string and removing the appended part again:

classdef String < handle
  ....
  properties
    stringobj = '';
  end
  function o = plus( o, b )
    o.stringobj = [ o.stringobj b ];
  end
  function n = Length( o )
    n = length( o.stringobj );
  end
  function o = SetLength( o, n )
    o.stringobj = o.stringobj( 1 : n );
  end
end

function atest( a, b ) %plain functions
  n = length( a );
  a = [ a b ];
  a = a( 1 : n );

function btest( a, b ) %OOP
  n = a.Length();
  a = a + b;
  a.SetLength( n );

function RunProfilerLoop( nLoop, fun, varargin )
  profile on;
  for i = 1 : nLoop
    fun( varargin{ : } );
  end
  profile off;
  profile report;

a = 'test';
aString = String( 'test' );
RunProfilerLoop( 1000, @(x,y)atest(x,y), a, 'appendme' );
RunProfilerLoop( 1000, @(x,y)btest(x,y), aString, 'appendme' );

The results, total time in seconds, for 1000 iterations:

btest 0.550 (with String.SetLength 0.138, String.plus 0.065, String.Length 0.057)

atest 0.015

Results for the logger system are likewise: 0.1 seconds for 1000 calls to frpintf( 1, 'test\n'), 7 (!) seconds for 1000 calls to my system when using the String class internally (OK, it has a lot more logic in it, but to compare with C++: the overhead of my system that uses std::string( "blah") and std::cout at the output side vs plain std::cout << "blah" is on the order of 1 millisecond.)

[ Edit ]

Since MATLAB is interpreted, it has to look up the definition of a function/object at run time. So I was wondering that maybe much more overhead is involved in looking up class or package function vs functions that are in the path. I tried to test this, and it just gets stranger. To rule out the influence of classes/objects, I compared calling a function in the path vs a function in a package:

function n = atest( x, y )
  n = ctest( x, y ); % ctest is in matlab path

function n = btest( x, y )
  n = util.ctest( x, y ); % ctest is in +util directory, parent directory is in path

Results, gathered same way as above:

atest 0.004 sec, 0.001 sec in ctest

btest 0.060 sec, 0.014 sec in util.ctest

So, is all this overhead just coming from MATLAB spending time looking up definitions for it's OOP implementation, whereas this overhead is not there for functions that are directly in the path?

+1  A: 

The handle class has an additional overhead from tracking all of references to itself for cleanup purposes.

Try the same experiment without using the handle class and see what your results are.

MikeEL
exactly the same experiment with String, but now as a value class (on another machine though); atest: 0.009, btest: o.356. That is basically the same difference as with the handle, so I do not think tracking references is the key answer. It also does not explain the overhead in functions vs function in packages.
stijn
What version of matlab are you using?
MikeEL
7.8.0 aka R2009a
stijn
+14  A: 

I've been working with OO MATLAB for a while, and ended up looking at similar performance issues.

The short answer is: yes, MATLAB's OOP is kind of slow. There is substantial method call overhead, higher than mainstream OO languages, and there's not much you can do about it. Part of the reason may be that idiomatic MATLAB uses "vectorized" code to reduce the number of method calls, and per-call overhead is not a high priority.

I benchmarked the performance by writing do-nothing "nop" functions as the various types of functions and methods. Here are some typical results.

Computer: PCWIN   Release: 2008b 
Calling each function/method 100000 times
nop() function:                 0.17773 sec   1.78 usec per call
nop1-5() functions:             0.12928 sec   1.29 usec per call
nop() method:                   0.24633 sec   2.46 usec per call
nop1-5() methods:               0.23261 sec   2.33 usec per call
nop() private function:         0.13144 sec   1.31 usec per call
classdef nop(obj):              0.96431 sec   9.64 usec per call
classdef obj.nop():             1.73764 sec  17.38 usec per call
classdef private_nop(obj):      0.90689 sec   9.07 usec per call
classdef nop(obj) (m-file):     0.92009 sec   9.20 usec per call
classdef class.staticnop():     1.25775 sec  12.58 usec per call
Java nop():                     2.22286 sec  22.23 usec per call
Java static_nop():              0.80935 sec   8.09 usec per call
Java nop() from Java:           0.00013 sec   0.00 usec per call
MEX mexnop():                   0.10492 sec   1.05 usec per call
C nop():                        0.00001 sec   0.00 usec per call

Similar results on R2008a through R2009b. This is on Windows XP x64 running 32-bit MATLAB.

The "Java nop()" is a do-nothing Java method called from within an M-code loop, and includes the MATLAB-to-Java dispatch overhead with each call. "Java nop() from Java" is the same thing called in a Java for() loop and doesn't incur that boundary penalty. Take the Java and C timings with a grain of salt; a clever compiler could optimize the calls away completely.

The package scoping mechanism is new, introduce at about the same time as the classdef classes. Its behavior may be related.

A few tentative conclusions:

  • Methods are slower than functions.
  • New style (classdef) methods are slower than old style methods.
  • The new obj.nop() syntax is slower than the nop(obj) syntax, even for the same method on a classdef object.
  • Method call overhead is higher (about 2x) in 64-bit MATLAB on Windows. (Not shown.)
  • MATLAB method dispatch is slower than some other languages.

Saying why this is so would just be speculation on my part. The MATLAB engine's OO internals aren't public. It's not an interpreted vs compiled issue per se - MATLAB has a JIT - but MATLAB's looser typing and syntax may mean more work at run time. (E.g. you can't tell from syntax alone whether "f(x)" is a function call or an index into an array; it depends on the state of the workspace at run time.) It may be because MATLAB's class definitions are tied to filesystem state in a way that many other languages' are not.

So, what to do?

An idiomatic MATLAB approach to this is to "vectorize" your code by structuring your class definitions such that an object instance wraps an array; that is, each of its fields hold parallel arrays (called "planar" organization in the MATLAB documentation). Rather than having an array of objects, each with fields holding scalar values, define objects which are themselves arrays, and have the methods take arrays as inputs, and make vectorized calls on the fields and inputs. This reduces the number of method calls made, hopefully enough that the dispatch overhead is not a bottleneck.

Mimicking a C++ or Java class in MATLAB probably won't be optimal. Java/C++ classes are typically built such that objects are the smallest building blocks, as specific as you can (that is, lots of different classes), and you compose them in arrays, collection objects, etc, and iterate over them with loops. To make fast MATLAB classes, turn that approach inside out.

The point is to arrange your code to play to the strengths of the language - array handling, vectorized math - and avoid the weak spots.

EDITed to add:

Path Sensitivity

Function dispatch time is sensitive to directories added to the MATLAB path. My original benchmark was done with a couple dozen directories from our M-code libraries on the path. Here's a rerun benchmark done in a session with the default MATLAB path, which is closer to bare MATLAB behavior.

>> call_nops
Computer: PCWIN   Release: 2009b
Calling each function/method 100000 times
nop() function:                 0.02261 sec   0.23 usec per call
nop1-5() functions:             0.02182 sec   0.22 usec per call
nop() subfunction:              0.02244 sec   0.22 usec per call
@()[] anonymous function:       0.08461 sec   0.85 usec per call
nop(obj) method:                0.24664 sec   2.47 usec per call
nop1-5(obj) methods:            0.23469 sec   2.35 usec per call
nop() private function:         0.02197 sec   0.22 usec per call
classdef nop(obj):              0.90547 sec   9.05 usec per call
classdef obj.nop():             1.75522 sec  17.55 usec per call
classdef private_nop(obj):      0.84738 sec   8.47 usec per call
classdef nop(obj) (m-file):     0.90560 sec   9.06 usec per call
classdef class.staticnop():     1.16361 sec  11.64 usec per call
Java nop():                     2.43035 sec  24.30 usec per call
Java static_nop():              0.87682 sec   8.77 usec per call
Java nop() from Java:           0.00014 sec   0.00 usec per call
MEX mexnop():                   0.11409 sec   1.14 usec per call
C nop():                        0.00001 sec   0.00 usec per call

Drastic difference in function call time; little in method call time. Private functions also speed up.

This suggests managing your codebase size has performance impacts. I'm not sure exactly what caused it - adding many empty directories to the path, or a single directory containing all our M-code, did not cause the same slowdown. Don't know if calls to MATLAB supplied functions are affected.

It also suggests that minimizing the public methods on your classes and preferring private functions could speed things up.

Andrew Janke
thanks for your elaborative answer!
stijn