Running a MATLAB code fragment without namespace pollution

Question

Hi!

I'm writing a version of Python's doctest test-runner, for MATLAB (it partly works...). For this to work, I need to run the code in people's examples in their m-file help. I want variables to carry over from one line to the next, e.g.

% >> I = 5 + 33; % expect no output
% >> I
% 
% I =
% 
%     38
%

To run the tests, I have a loop over matches to the REGEX that searches for the tests. For each match, I evalc the example and make sure the result matches:

for I = 1:length(examples)
    try
        got = evalc(examples(I).source);
    catch exc
        got = ['??? ' exc.message];
    end

    % process the result...
end

The problem is that the example's definition of I has now clobbered the loop variable in my loop, since the assignments carry over from the eval into the outer scope. I looked around for something capable of creating a new scope/workspace, but evalin can only re-use the caller's workspace, which is even worse. I've also considered options with calling a sub-function or save/load, but not gotten anywhere, but maybe I just haven't thought hard enough.

So I guess I need to just name all my variables doctest__system__* and live with the namespace problems... unless you have another idea for a strategy to avoid variable name conflicts?

Answer 1

A very interesting project for sure.. I think the best option you have is to write a separate function to execute the tests, and use a unique prefix for all variable inside of this function to avoid name conflict. Here my attempt at this:

function [PREFIX_b varargout] = testContext(PREFIX_src, PREFIX_srcOutput)
    %# TESTCONTEXT   Executes the source code and tests for
    %#               equality against the expected output
    %#
    %#   Input:
    %#       PREFIX_src       - source to execute, cellarry of statements
    %#       PREFIX_srcOutput - output to expect, cellarray of output of each statement
    %#
    %#   Output:
    %#       PREFIX_b         - true/false for success/failure of test
    %#                          note that the output is strtrim()'ed then strcmp()'ed
    %#       varargout{1}     - variable names assigned in this confined context
    %#       varargout{2}     - variable values assigned
    %#
    %#   Example 1:
    %#       source = { 'I = 5+33;' 'I' };
    %#       output = { [], ['I =' char(10) '    38'] };
    %#       b = testContext(source, output);
    %#
    %#   Example 2:
    %#       source = { 'I = 5+33; J = 2;' 'K = 1;' 'disp(I+J+K)' };
    %#       output = { [], [], '41' };
    %#       [b varNames varValues] = testContext(source, output);
    %#
    %#   See also: eval evalc
    %#

    PREFIX_b = true;

    try
        %# for each statement
        for PREFIX_i=1:numel(PREFIX_src)
            %# evaluate
            PREFIX_output = evalc( PREFIX_src{PREFIX_i} );
            PREFIX_output = strtrim(PREFIX_output);            %# trim whitespaces
            %# compare output
            if ~isempty( PREFIX_srcOutput{PREFIX_i} )
                if ~strcmp(PREFIX_output,PREFIX_srcOutput{PREFIX_i})
                    PREFIX_b = false;
                    return
                end
            end
        end

        if nargout > 1
            %# list created variables in this context
            %#clear ans
            PREFIX_vars = whos('-regexp', '^(?!PREFIX_).*');   %# java regex negative lookahead
            varargout{1} = { PREFIX_vars.name };

            if nargout > 2
                %# return those variables
                varargout{2} = cell(1,numel(PREFIX_vars));
                for PREFIX_i=1:numel(PREFIX_vars)
                    [~,varargout{2}{PREFIX_i}] = evalc( PREFIX_vars(PREFIX_i).name );
                end
            end
        end

    catch ME
        warning(ME.identifier, ME.message)
        PREFIX_b = false;
        varargout{1} = {};
        varargout{2} = {};
    end
end

I assume you are able to parse the m-file to recover the examples to test, where you have each statement along with its expected output.

As an example, consider this simple test embedded in the header of a function:

I = 5 + 33;
J = 2*I;
disp(I+J)

Since only the last statement has an output, we test it as:

source = {'I = 5 + 33;' 'J = 2*I;' 'disp(I+J)'};
output = {[], [], '114'};
[b varNames varValues] = testContext(source, output)

the results:

b =
     1
varNames = 
    'I'    'J'
varValues = 
    [38]    [76]

It shows whether the test passed of failed. Optionally, the function returns a list of variables created in that context along with their values.