views:

456

answers:

1

Hi,

in this article about boost spirit semantic actions it is mentioned that

There are actually 2 more arguments being passed: the parser context and a reference to a boolean ‘hit’ parameter. The parser context is meaningful only if the semantic action is attached somewhere to the right hand side of a rule. We will see more information about this shortly. The boolean value can be set to false inside the semantic action invalidates the match in retrospective, making the parser fail.

All fine, but i've been trying to find an example passing a function object as semantic action that uses the other parameters (parser context and hit boolean) but i haven't found any. I would love to see an example using regular functions or function objects, as i barely can grok the phoenix voodoo

+5  A: 

This a really good question (and also a can of worms) because it gets at the interface of qi and phoenix. I haven't seen an example either, so I'll extend the article a little in this direction.

As you say, functions for semantic actions can take up to three parameters

  1. Matched attribute - covered in the article
  2. Context - contains the qi-phoenix interface
  3. Match flag - manipulate the match state

Match flag

As the article states, the second parameter is not meaningful unless the expression is part of a rule, so lets start with the third. A placeholder for the second parameter is still needed though and for this use boost::fusion::unused_type. So a modified function from the article to use the third parameter is:

#include <boost/spirit/include/qi.hpp>
#include <string>
#include <iostream>

void f(int attribute, const boost::fusion::unused_type& it, bool& mFlag){
    //output parameters
    std::cout << "matched integer: '" << attribute << "'" << std::endl
              << "match flag: " << mFlag << std::endl;

    //fiddle with match flag
    mFlag = false;
}

namespace qi = boost::spirit::qi;

int main(void){
   std::string input("1234 6543");
   std::string::const_iterator begin = input.begin(), end = input.end();

   bool returnVal = qi::phrase_parse(begin, end, qi::int_[f], qi::space);

   std::cout << "return: " << returnVal << std::endl;
   return 0;
}

which outputs:

matched integer: '1234'
match flag: 1
return: 0

All this example does is switch the match to a non-match, which is reflected in the parser output. According to hkaiser, in boost 1.44 and up setting the match flag to false will cause the match to fail in the normal way. If alternatives are defined, the parser will backtrack and attempt to match them as one would expect. However, in boost<=1.43 a Spirit bug prevents backtracking, which causes strange behavior. To see this, add phoenix include boost/spirit/include/phoenix.hpp and change the expression to

qi::int_[f] | qi::digit[std::cout << qi::_1 << "\n"]

You'd expect that, when the qi::int parser fails, the alternative qi::digit to match the beginning of the input at "1", but the output is:

matched integer: '1234'
match flag: 1
6
return: 1

The 6 is the first digit of the second int in the input which indicates the alternative is taken using the skipper and without backtracking. Notice also that the match is considered succesful, based on the alternative.

Once boost 1.44 is out, the match flag will be useful for applying match criteria that might be otherwise difficult to express in a parser sequence. Note that the match flag can be manipulated in phoenix expressions using the _pass placeholder.

Conext parameter

The more interesting parameter is the second one, which contains the qi-phoenix interface, or in qi parlance, the context of the semantic action. To illustrate this, first examine a rule:

rule<Iterator, Attribute(Arg1,Arg2,...), qi::locals<Loc1,Loc2,...>, Skipper>

The context parameter embodies the Attribute, Arg1, ... ArgN, and qi::locals template paramters, wrapped in a boost::spirit::context template type. This attribute differs from the function parameter: the function parameter attribute is the parsed value, while this attribute is the value of the rule itself. A semantic action must map the former to the latter. Here's an example of a possible context type (phoenix expression equivalents indicated):

using namespace boost;
spirit::context<              //context template
    fusion::cons<             
        int&,                 //return int attribute (phoenix: _val)
        fusion::cons<
            char&,            //char argument1       (phoenix: _r1)
            fusion::cons<
                float&,       //float argument2      (phoenix: _r2) 
                fusion::nil   //end of cons list
            >,
        >,
    >,
    fusion::vector2<          //locals container
        char,                 //char local           (phoenix: _a)
        unsigned int          //unsigned int local   (phoenix: _b)
    > 
>

Note the return attribute and argument list take the form of a lisp-style list (a cons list). To access these variables within a function, access the attribute or locals members of the context struct template with fusion::at<>(). For example, for a context variable con

//assign return attribute
fusion::at_c<0>(con.attributes) = 1;

//get the second rule argument
float arg2 = fusion::at_c<2>(con.attributes);

//assign the first local
fusion::at_c<1>(con.locals) = 42;

To modify the article example to use the second argument, change the function definition and phrase_parse calls:

...
typedef 
    boost::spirit::context<
        boost::fusion::cons<int&, boost::fusion::nil>, 
        boost::fusion::vector0<> 
    > f_context;
void f(int attribute, const f_context& con, bool& mFlag){
   std::cout << "matched integer: '" << attribute << "'" << std::endl
             << "match flag: " << mFlag << std::endl;

   //assign output attribute from parsed value    
   boost::fusion::at_c<0>(con.attributes) = attribute;
}
...
int matchedInt;
qi::rule<std::string::const_iterator,int(void),ascii::space_type> 
    intRule = qi::int_[f];
qi::phrase_parse(begin, end, intRule, ascii::space, matchedInt);
std::cout << "matched: " << matchedInt << std::endl;
....

This is a very simple example that just maps the parsed value to the output attribute value, but extensions should be fairly apparent. Just make the context struct template parameters match the rule output, input, and local types. Note that this type of a direct match between parsed type/value to output type/value can be done automatically using auto rules, with a %= instead of a = when defining the rule:

qi::rule<std::string::const_iterator,int(void),ascii::space_type> 
    intRule %= qi::int_;

IMHO, writing a function for each action would be rather tedious, compared to the brief and readable phoenix expression equivalents. I sympathize with the voodoo viewpoint, but once you work with phoenix for a little while, the semantics and syntax aren't terribly difficult.

Edit: Accessing rule context w/ Phoenix

The context variable is only defined when the parser is part of a rule. Think of a parser as being any expression that consumes input, where a rule translates the parser values (qi::_1) into a rule value (qi::_val). The difference is often non-trivial, for example when qi::val has a Class type that needs to be constructed from POD parsed values. Below is a simple example.

Let's say part of our input is a sequence of three CSV integers (x1, x2, x3), and we only care out an arithmetic function of these three integers (f = x0 + (x1+x2)*x3 ), where x0 is a value obtained elsewhere. One option is to read in the integers and calculate the function, or alternatively use phoenix to do both.

For this example, use one rule with an output attribute (the function value), and input (x0), and a local (to pass information between individual parsers with the rule). Here's the full example.

#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <string>
#include <iostream>

namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;

int main(void){
   std::string input("1234, 6543, 42");
   std::string::const_iterator begin = input.begin(), end = input.end();

   qi::rule<
      std::string::const_iterator,
      int(int),                    //output (_val) and input (_r1)
      qi::locals<int>,             //local int (_a)
      ascii::space_type
   >
      intRule =
            qi::int_[qi::_a = qi::_1]             //local = x1
         >> ","
         >> qi::int_[qi::_a += qi::_1]            //local = x1 + x2
         >> ","
         >> qi::int_
            [
               qi::_val = qi::_a*qi::_1 + qi::_r1 //output = local*x3 + x0
            ];

   int ruleValue, x0 = 10;
   qi::phrase_parse(begin, end, intRule(x0), ascii::space, ruleValue);
   std::cout << "rule value: " << ruleValue << std::endl;
   return 0;
}

Alternatively, all the ints could be parsed as a vector, and the function evaluated with a single semantic action (the % below is the list operator and elements of the vector are accessed with phoenix::at):

namespace ph = boost::phoenix;
...
    qi::rule<
        std::string::const_iterator,
        int(int),
        ascii::space_type
    >
    intRule =
        (qi::int_ % ",")
        [
            qi::_val = (ph::at(qi::_1,0) + ph::at(qi::_1,1))
                      * ph::at(qi::_1,2) + qi::_r1
        ];
....

For the above, if the input is incorrect (two ints instead of three), bad thing could happen at run time, so it would be better to specify the number of parsed values explicitly, so parsing will fail for a bad input. The below uses _1, _2, and _3 to reference the first, second, and third match value:

(qi::int_ >> "," >> qi::int_ >> "," >> qi::int_)
[
    qi::_val = (qi::_1 + qi::_2) * qi::_3 + qi::_r1
];

This is a contrived example, but should give you the idea. I've found phoenix semantic actions really helpful in constructing complex objects directly from input; this is possible because you can call constructors and member functions within semantic actions.

academicRobot
Thanks for that excellent explanation. Do you mind me 'stealing' this in order to repost it on the Spirit website (certainly giving due credits)?The backtracking issue you're mentioning is a bug in Spirit. The correct behavior after failing the first alternative should be that the second alternative starts over at the same point in the input as the first alternative. I'll see what I can do to fix this.Further, you shouldn't use the phoenix placeholders in semantic actions. Please always use the corresponding Spirit placeholders, i.e. qi::_1.
hkaiser
Ok, the backtracking issue is fixed now and will be ok in the next release (Boost V1.44).
hkaiser
@hkaiser Glad you like it and please do reuse it if you want. I was going to ask about that backtracking issue on the mailing list, thanks for taking care of it. A question for you on the placeholders: both `phoenix::_1` and `qi::_1` are defined as `const phoenix::actor<argument<0> >`, is this subject to change?
academicRobot
Thanks! I'll write an article asap incorporating your stuff. You're right, `phoenix::_1` and `qi::_1` are defined as const `phoenix::actor<argument<0> >`, but they are based on different argument<> types. `phoenix::_1` is based on `phoenix::argument<0>`, while `qi::_1` is based on `spirit::argument<0>`. In your case it just turns out to be identical by coincidence.
hkaiser
@hkaiser Ok, I see, thanks. I'd been using them indiscriminately, think this explains some problems I had previously. Edited answer with correct placeholder.
academicRobot
thanks for that explanation! i see so basically this approach isn't exactly wrapped behind a nice interface, but how would you use phoenix when trying to access the right hand side of a rule? I'm not even clear that a rule is always a rhs/lhs thing. What about rules that are for example: rule1 << lit("(") << rule2 << lit(")") << lit("->") << rule3, what is the right hand side in that rule?
lurscher
@lurscher, The example you gave is a parser (I assume you meant `>>`, not `<<`), but not a rule. Therefore it has no context (context arg for semantic action is unused type). You assign it to a rule: `rule4 %= rule1 >> lit("(") >> rule2 >> lit(")") >> lit("->") >> rule3`. Then, rule4 has a context, also each component rule has its own context. I'll edit my answer to show the phoenix equivalents.
academicRobot
thanks, i posted this example http://stackoverflow.com/questions/3078162/parsing-a-grammar-with-boost-spirit i am trying to get working with the phoenix style
lurscher