Subject: Re: [proto] interface changes - msg#00003

Previous Message by Date: (click to view message preview)

Re: [proto] interface changes

Eric, I've kept all the quoting in place to keep the context for easier reading... > > I had a closer look at your stuff and have some comments/questions: > > > > 1. IIUC the proto::basic_expr<> is the central data container of > > proto. It's functionality very much resembles a fusion sequence (in > > fact a large part of proto is dedicated to make it compatible to > > fusion). Is there a reason why you didn't use a fusion > sequence for that in the first place? > > Yes, there's a good reason. I need static, aggregate > initialization. This: > > proto::meta::terminal<int>::type const i = {1}; > > is statically initialized. For DSELs, the primitives are > going to be declared as global constants, and it's crucial > that they do not have constructors. Otherwise, you'll get > order of initialization problems when other global objects > try to use them. > > Aggregate initialization is also extremely handy for nested > expressions. > Consider: > > // !(int >> int) > typedef proto::meta::unary_expr< > proto::logical_not_tag > ,proto::meta::binary_expr< > proto::right_shift_tag > ,proto::meta::terminal<int>::type > ,proto::meta::terminal<int>::type > >::type > >::type some_expr_type; > > Now you can say: > > some_expr_type expr = {{{1}, {2}}}; > > instead of writing some horrific constructor call. Can't do > that with any Fusion sequence I know of. Understood. Thanks for the explanation. But that means, that if we were going to use proto to implement spirit2, we were bound to this internal representation for parsers. This is perhaps not a big deal (I'm not sure, though - just have some bad gut feeling about that, can't verbalize). > > 2. How to define proto templated terminals? I.e. something like: > > > > namespace A { > > template <typename Char> > > struct char_ > > : proto::meta::terminal<Char> > > { > > ... > > }; > > } > > > > but this doesn't work... > > What you have above would work, as long as you use it correctly: > > A::char_<char>::type t = {'a'}; > A::char_<wchar_t>::type u = {L'a'}; > > meta::terminal, meta::unary_expr and meta::binary_expr are > metafunctions. They generate basic_expr expression types, > they are not > expression types themselves. That's why they're in the meta:: > namespace. > > If I read into your question a bit, it seems like you're asking for a > way to define your own expression types that inherit from proto > expression types. I want that myself, and I mocked up support > for that > last night. For instance, in xpressive, there is a mark_tag type that > you can use in regexes as: > > mark_tag number(1); // the first mark is called "number" > > // match a number, a space, and the same number again: > sregex rx = (number = +_d) >> ' ' >> number; > > So, what is mark_tag? The easy solution would be to make it a typedef > for some basic_expr type, but since basic_expr is an aggregate, you'd > need to initialize it as: > > mark_tag number = {{1}}; > > That's fine for library code but might be surprising to users. My > current thinking is that mark_tag will be something like this: > > struct mark_tag_ex : proto::extends< some-basic_expr-type > > { > mark_tag_ex(int backref) > { > some-basic_expr-type tmp = {{backref}}; > this->assign(tmp); > } > }; > > typedef proto::extends_tag<mark_tag_ex> mark_tag; > > Now users can use mark_tags in expressions, and it behaves > exactly like > some-basic_expr-type. A) Is this already available? B) Let me give you some context for my question. I was trying to implement the Spirit char_ parser. This thingy will be used in 3 different ways: as char_, char_(lit), char_(lit, lit). The first one is obvious to implement, but the others require to write the corresponding parse function as: template <typename Source, typename Char, typename Context> inline bool parse (Source& src, typename boost::result_of<terminals::char_(Char)>::type const&p, Context& ctx) { ... } Which is not possible. Even if I try to wrap this result_of in a separate structure: template <typename Char> struct char_lit : proto::meta::terminal<Char> { typedef typename boost::result_of<terminals::char_(Char)>::type type; }; then: template <typename Source, typename Char, typename Context> inline bool parse (Source& src, char_lit<Char> const& p, Context& ctx) { ... } doesn't work either for obvious reasons. The compiler is not able to deduce the correct templat eparameters when called with a 'boost::result_of<terminals::char_(Char)>::type'. Do I miss something? > > 3. Is it possible to use proto with several different > libraries at once? > > Consider: > > > > namespace A { typedef > proto::meta::terminal<some_tag>::type term1; } > > namespace B { typedef > proto::meta::terminal<some_other_tag>::type term2; > > } > > > > What happens here: > > > > term1 >> term2 > > > > ? > > You'll get: > > basic_expr< > right_shift_tag > ,mpl::vector2<ref<A::term1>, ref<B::term2> > > > Sure, thought so. > > Is there any way to add semantic concept checking to proto > to inhibit > > compilation in this case? > > Not yet. I'm open to suggestions on how to implement > something like that. No suggestions yet, sorry. > > 4. Is there any way to disable certain operators > implemented by proto for a > > concrete proto terminal? For instance what do I need to do > to disable the > > operator|() ? > > Not yet. It would be a nice addition, if I could figure out > how to add > it without bogging things down. The new proto is implemented > such that > the old enable_if hackery on the operator overloads is not needed. I > found that to be expensive at compile time. Ok. OTOH I find it very important to be able to inhibit certain operators, especially in a context where the user gets directly exposed to the proto terminal types (as char_ et.al.). Certainly we just could declare the corresponding functions without implementing it (or using some other nasty trick), but this somehow goes against the whole idea of having a underlying meta expression library handling all the related issues. > One of the gripes I've heard about xpressive is the long > compile times. > I'm trying very hard to bring that under control. Additional features > such as concept checking and operator selection must be added > carefully > so they don't make the whole thing slow. > > You might be disappointed that the new proto doesn't add any new > features. The difference is that we now have a solid foundation for > future work. Sure, understood. I'm not disappointed at all. All I was trying to do is to better understand, what proto can do for us in the context of Spirit2/Karma. Thanks! Regards Hartmut ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Next Message by Date: click to view message preview

Re: [proto] interface changes

Hartmut Kaiser wrote: > Eric, > > I've kept all the quoting in place to keep the context for easier reading... > >> My current thinking is that mark_tag will be something like this: >> >> struct mark_tag_ex : proto::extends< some-basic_expr-type > >> { >> mark_tag_ex(int backref) >> { >> some-basic_expr-type tmp = {{backref}}; >> this->assign(tmp); >> } >> }; >> >> typedef proto::extends_tag<mark_tag_ex> mark_tag; >> >> Now users can use mark_tags in expressions, and it behaves >> exactly like >> some-basic_expr-type. > > A) Is this already available? It's available now. -- Eric Niebler Boost Consulting www.boost-consulting.com ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Previous Message by Thread: click to view message preview

Re: [proto] interface changes

Eric, I've kept all the quoting in place to keep the context for easier reading... > > I had a closer look at your stuff and have some comments/questions: > > > > 1. IIUC the proto::basic_expr<> is the central data container of > > proto. It's functionality very much resembles a fusion sequence (in > > fact a large part of proto is dedicated to make it compatible to > > fusion). Is there a reason why you didn't use a fusion > sequence for that in the first place? > > Yes, there's a good reason. I need static, aggregate > initialization. This: > > proto::meta::terminal<int>::type const i = {1}; > > is statically initialized. For DSELs, the primitives are > going to be declared as global constants, and it's crucial > that they do not have constructors. Otherwise, you'll get > order of initialization problems when other global objects > try to use them. > > Aggregate initialization is also extremely handy for nested > expressions. > Consider: > > // !(int >> int) > typedef proto::meta::unary_expr< > proto::logical_not_tag > ,proto::meta::binary_expr< > proto::right_shift_tag > ,proto::meta::terminal<int>::type > ,proto::meta::terminal<int>::type > >::type > >::type some_expr_type; > > Now you can say: > > some_expr_type expr = {{{1}, {2}}}; > > instead of writing some horrific constructor call. Can't do > that with any Fusion sequence I know of. Understood. Thanks for the explanation. But that means, that if we were going to use proto to implement spirit2, we were bound to this internal representation for parsers. This is perhaps not a big deal (I'm not sure, though - just have some bad gut feeling about that, can't verbalize). > > 2. How to define proto templated terminals? I.e. something like: > > > > namespace A { > > template <typename Char> > > struct char_ > > : proto::meta::terminal<Char> > > { > > ... > > }; > > } > > > > but this doesn't work... > > What you have above would work, as long as you use it correctly: > > A::char_<char>::type t = {'a'}; > A::char_<wchar_t>::type u = {L'a'}; > > meta::terminal, meta::unary_expr and meta::binary_expr are > metafunctions. They generate basic_expr expression types, > they are not > expression types themselves. That's why they're in the meta:: > namespace. > > If I read into your question a bit, it seems like you're asking for a > way to define your own expression types that inherit from proto > expression types. I want that myself, and I mocked up support > for that > last night. For instance, in xpressive, there is a mark_tag type that > you can use in regexes as: > > mark_tag number(1); // the first mark is called "number" > > // match a number, a space, and the same number again: > sregex rx = (number = +_d) >> ' ' >> number; > > So, what is mark_tag? The easy solution would be to make it a typedef > for some basic_expr type, but since basic_expr is an aggregate, you'd > need to initialize it as: > > mark_tag number = {{1}}; > > That's fine for library code but might be surprising to users. My > current thinking is that mark_tag will be something like this: > > struct mark_tag_ex : proto::extends< some-basic_expr-type > > { > mark_tag_ex(int backref) > { > some-basic_expr-type tmp = {{backref}}; > this->assign(tmp); > } > }; > > typedef proto::extends_tag<mark_tag_ex> mark_tag; > > Now users can use mark_tags in expressions, and it behaves > exactly like > some-basic_expr-type. A) Is this already available? B) Let me give you some context for my question. I was trying to implement the Spirit char_ parser. This thingy will be used in 3 different ways: as char_, char_(lit), char_(lit, lit). The first one is obvious to implement, but the others require to write the corresponding parse function as: template <typename Source, typename Char, typename Context> inline bool parse (Source& src, typename boost::result_of<terminals::char_(Char)>::type const&p, Context& ctx) { ... } Which is not possible. Even if I try to wrap this result_of in a separate structure: template <typename Char> struct char_lit : proto::meta::terminal<Char> { typedef typename boost::result_of<terminals::char_(Char)>::type type; }; then: template <typename Source, typename Char, typename Context> inline bool parse (Source& src, char_lit<Char> const& p, Context& ctx) { ... } doesn't work either for obvious reasons. The compiler is not able to deduce the correct templat eparameters when called with a 'boost::result_of<terminals::char_(Char)>::type'. Do I miss something? > > 3. Is it possible to use proto with several different > libraries at once? > > Consider: > > > > namespace A { typedef > proto::meta::terminal<some_tag>::type term1; } > > namespace B { typedef > proto::meta::terminal<some_other_tag>::type term2; > > } > > > > What happens here: > > > > term1 >> term2 > > > > ? > > You'll get: > > basic_expr< > right_shift_tag > ,mpl::vector2<ref<A::term1>, ref<B::term2> > > > Sure, thought so. > > Is there any way to add semantic concept checking to proto > to inhibit > > compilation in this case? > > Not yet. I'm open to suggestions on how to implement > something like that. No suggestions yet, sorry. > > 4. Is there any way to disable certain operators > implemented by proto for a > > concrete proto terminal? For instance what do I need to do > to disable the > > operator|() ? > > Not yet. It would be a nice addition, if I could figure out > how to add > it without bogging things down. The new proto is implemented > such that > the old enable_if hackery on the operator overloads is not needed. I > found that to be expensive at compile time. Ok. OTOH I find it very important to be able to inhibit certain operators, especially in a context where the user gets directly exposed to the proto terminal types (as char_ et.al.). Certainly we just could declare the corresponding functions without implementing it (or using some other nasty trick), but this somehow goes against the whole idea of having a underlying meta expression library handling all the related issues. > One of the gripes I've heard about xpressive is the long > compile times. > I'm trying very hard to bring that under control. Additional features > such as concept checking and operator selection must be added > carefully > so they don't make the whole thing slow. > > You might be disappointed that the new proto doesn't add any new > features. The difference is that we now have a solid foundation for > future work. Sure, understood. I'm not disappointed at all. All I was trying to do is to better understand, what proto can do for us in the context of Spirit2/Karma. Thanks! Regards Hartmut ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Next Message by Thread: click to view message preview

Re: [proto] interface changes

Eric Niebler wrote: > > But that means, that if we were going to use proto to implement > > spirit2, we were bound to this internal representation for parsers. > > This is perhaps not a big deal (I'm not sure, though - just > have some > > bad gut feeling about that, can't verbalize). > > In proto/make_expr.hpp, there is a function called > unpack_expr(), which turns any Fusion sequence into a > basic_expr. That will certainly come in handy, and already > has, in fact. Good to know. Have not seen that, yet. > It would certainly be possible to further generalize proto to > work with models of some BasicExpr concept, with generic > accessors, so that Fusion sequences can be adapted to be > BasicExpr models non-intrusively, etc. > > That sounds great, but there are problems. The operator > overloads become more expensive -- you'll need to prune the > overload sets with enable_if. Not if we were going to use the same fusion sequence all over the place (somehow 'specialize' proto to use a given data structure). In this case there wasn't a need to overload set reduction. It boiled down essentially to the same scenarion you have today: just one internal representation. > But more crucially, it becomes extremely cumbersome to write > specialization and overloads for recognizing specific > sub-expressions. > In the toy spirit example, I do: > > typedef proto::meta::terminal<char>::type chlit_p; > > and then I define an overload of spirit2::parse() that > accepts a chlit_p. That works because there is really only > one way to have a character literal in proto: > > basic_expr<terminal_tag, mpl::vector1<char> > > > If lots of different types could model the BasicExpr concept, > then how do you write your spirit2::parse() overloads? > (You're already running into a similar problem below, but > with the container hard-coded as basic_expr, it's not so bad.) See above. > > B) Let me give you some context for my question. I was trying to > > implement the Spirit char_ parser. This thingy will be used in 3 > > different ways: as char_, char_(lit), char_(lit, lit). The > first one > > is obvious to implement, but the others require to write > the corresponding parse function as: > > > > > > template <typename Source, typename Char, typename Context> > > inline bool > > parse (Source& src, > > typename > boost::result_of<terminals::char_(Char)>::type const&p, > > Context& ctx) > > { ... } > > > > Which is not possible. Even if I try to wrap this result_of in a > > separate > > structure: > > > > template <typename Char> > > struct char_lit > > : proto::meta::terminal<Char> > > { > > typedef typename > > boost::result_of<terminals::char_(Char)>::type > > type; > > }; > > > > then: > > > > template <typename Source, typename Char, typename Context> > > inline bool > > parse (Source& src, char_lit<Char> const& p, Context& ctx) > > { ... } > > > > doesn't work either for obvious reasons. The compiler is > not able to > > deduce the correct templat eparameters when called with a > > 'boost::result_of<terminals::char_(Char)>::type'. Do I miss > something? > > Ah, I understand. Yes, this can be tricky. Today, what you > would have to do it rip apart the expression. To handle > char_(Char), you might write an overload like: > > // Note the 2 in the basic_expr type here: > template<typename Source, typename Args, typename Context> > bool parse( Source & src > , basic_expr< function_tag, Args, 2 > const & p > , Context & cxt) > { > // proto::left(p) is "char_" > // proto::right(p) is a terminal<Char>::type > } > > To handle char_(Char, Char) you would: > > // Note the 3 in the basic_expr type here: > template<typename Source, typename Args, typename Context> > bool parse( Source & src > , basic_expr< function_tag, Args, 3 > const & p > , Context & cxt) > { > // ... > } > > Hopefully you get the idea. If your grammar has multiple > expression types of the form basic_expr< function_tag, Args, > 2 >, you need to further deconstruct the expression and > dispatch to helpers. Ok, that makes sense. Should have thought this way from the beginning... > (I also need to add proto::arg_c<N>() for accessing the Nth > argument of a proto expression.) Yes, would be handy. > I admit this isn't great, and I'm interested in thoughts > about how to make this easier. But just for kicks, imagine > how much harder this problem would be if you couldn't count > on the container being a basic_expr<>. > > I could envision using something like MPL lambda expressions > and enable_if for this. Something like: > > template<typename Source, typename Expr, typename Context> > typename enable_if< > proto::matches< Expr, function<char_, terminal<_>, > terminal<_> > > > ,bool > >::type > parse( Source & src, Expr const &expr, Context & cxt ) > > > That'd be pretty slick, but probably very expensive. But > certainly the option should be there for those who don't care > about compile time. Wow, that's slick! Neat idea. Do you really think this is so expensive in terms of compile time? In the end MPL does use it as well... > > Ok. OTOH I find it very important to be able to inhibit > certain operators, > > especially in a context where the user gets directly > exposed to the proto > > terminal types (as char_ et.al.). > > Agree 100%. I wonder if my extends<> template could be > employed to make > this happen. Turns out this is currently my major stumblestone for/against using proto for Spirit/Karma. Hmmm, will have to think a bit more about this. > Understood. I could give you a list of reasons Spirit and > Karma would be > better of built on top of proto if you want. :-) I'm pretty sure, you could :-P > And proto > will become > more feature-rich over time. I'm very interested in your feedback. Sure! Thanks and regards Hartmut ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642