Why does PROC FCMP function always return 33 bytes and no more?

Question

I have the following function defined via PROC FCMP. The point of the code should be pretty obvious and relatively straightforward. I'm returning the value of an attribute from a line of XHTML. Here's the code:

proc fcmp outlib=library.funcs.crawl;
    function getAttr(htmline $, Attribute $) $;

       /*-- Find the position of the match --*/
    Pos = index( htmline , strip( Attribute )||"=" );

       /*-- Now do something about it --*/
       if pos > 0 then do;
          Value = scan( substr( htmline, Pos + length( Attribute ) + 2), 1, '"');
       end;
       else Value = "";
       return( Value);
    endsub;
run;

No matter what I do with length or attrib statement to try to explicitly declare the data type returned, it ALWAYS returns only a max of 33 bytes of the requested string, regardless of how long the actual return value is. This happens no matter which attribute I am searching for. The same code (hard-coded) into a data step returns the correct results so this is related to PROC FCMP.

Here is the datastep I'm using to test it (where PageSource.html is any html file that has xhtml compliant attributes -- fully quoted):

data TEST;
length href $200;
infile "F:\PageSource.html";

input;

htmline = _INFILE_;

href = getAttr( htmline, "href");
x = length(href);

run;

UPDATE: This seems to work properly after upgrading to SAS9.2 - Release 2

Answer 1

I think the problem (though I don't know why) is in the scan function - it seems to be truncating input from substr(). If you pull the substr function out of scan(), assign the result of the substr funtion to a new variable that you then pass to scan, it seems to work.

Here is what I ran:

proc fcmp outlib=work.funcs.crawl;
    function getAttr(htmline $, Attribute $) $;
    length y $200;
       /*-- Find the position of the match --*/
    Pos = index( htmline , strip( Attribute )||"=" );

       /*-- Now do something about it --*/
       if pos > 0 then do;
          y=substr( htmline, Pos + length( Attribute ) + 2);
          Value = scan( y, 1, '"');       
       end;
       else Value = "";
       return( Value);
    endsub;
run;

options cmplib=work.funcs;

data TEST;
length href $200;
infile "PageSource.html";

input;

htmline = _INFILE_;
href = getAttr( htmline, "href");
x = length(href);
run;

Answer 2

I ended up backing out of using FCMP defined data step functions. I don't think they're ready for primetime. Not only could I not solve the 33 byte return issue, but it started regularly crashing SAS.

So back to the good old (decades old) technology of macros. This works:

/*********************************/
/*= Macro to extract Attribute  =*/
/*= from XHTML string           =*/
/*********************************/
%macro getAttr( htmline, Attribute, NewVar );
   if index( &htmline , strip( &Attribute )||"=" ) > 0 then do;
      &NewVar = scan( substr( &htmline, index( &htmline , strip( &Attribute )||"=" ) + length( &Attribute ) + 2), 1, '"' );
   end;
%mend;

Answer 3

In this case, an input pointer control should be enough. hope this helps.

/* create a test input file */
data _null_;
  file "f:\pageSource.html";
  input;
  put _infile_;
cards4;
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="w3.org/StyleSheets/TR/W3C-REC.css"; type="text/css"?>
;;;;
run;

/* extract the href attribute value, if any.                          */
/* assuming that the value and the attribute name occurs in one line. */
/* and max length is 200 chars.                                       */
data one;
  infile "f:\pageSource.html" missover;
  input @("href=") href :$200.;
  href = scan(href, 1, '"'); /* unquote */
run;

/* check */
proc print data=one;
run;
/* on lst
Obs                  href
 1
 2     w3.org/StyleSheets/TR/W3C-REC.css
*/