Boost karma: how does this implicit call to transform_attribute work? (or doesn't?)

Question

I have the following piece of code that seems to work fine (I based the semantic actions on reuse parsed variable with boost karma).

#include <iostream>
#include <iterator>
#include <memory>
#include <string>
#include <vector>

#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/sequence.hpp>
#include <boost/spirit/include/karma.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/spirit/include/phoenix_bind.hpp>
#include <boost/spirit/include/support_attributes.hpp>
#include <boost/spirit/include/support_adapt_adt_attributes.hpp>

using namespace boost::spirit;

struct DataElement
{
  DataElement(const std::string& s) : str_(s) {}

  const std::string& str() const { return str_; }
  std::string& str() { return str_; }
  std::string str_;
};
using Data = std::vector<std::shared_ptr<const DataElement>>;

namespace boost {
  namespace spirit {
    namespace traits {

      template<>
      struct transform_attribute<std::shared_ptr<const DataElement> const, const DataElement&, karma::domain>
      {
        using type = const DataElement&;
        static type pre(const std::shared_ptr<const DataElement>& val) { return *val; }
      };

    }
  }
}

BOOST_FUSION_ADAPT_ADT(
  DataElement,
  (std::string&, const std::string&, obj.str(), obj.str())
  );

template<typename Iterator>
struct TheGrammar: public karma::grammar<Iterator, Data()>
{
  TheGrammar(): karma::grammar<Iterator, Data()>(start)
  {
    start %= -(elt % karma::eol);
    elt %=
      karma::lit("'some prefix'")
      << karma::string [karma::_1 = boost::phoenix::at_c<0>(karma::_val)]
      << karma::lit("'some infix 1'")
      << karma::string [karma::_1 = boost::phoenix::at_c<0>(karma::_val)]
      << karma::lit("'some infix 2'")
      << karma::string [karma::_1 = boost::phoenix::at_c<0>(karma::_val)]
      << karma::lit("'some suffix'")
      ;
  }

  karma::rule<Iterator, Data()> start;
  karma::rule<Iterator, const DataElement&()> elt;
};

int main(void)
{
  Data vec = {
    std::make_shared<DataElement>("one"),
    std::make_shared<DataElement>("two"),
    std::make_shared<DataElement>("three"),
    std::make_shared<DataElement>("four"),
    std::make_shared<DataElement>("five"),
    std::make_shared<DataElement>("six"),
    std::make_shared<DataElement>("seven"),
    std::make_shared<DataElement>("eight"),
  };
  using iterator_type = std::ostream_iterator<char>;
  iterator_type out(std::cout);

  TheGrammar<iterator_type> grammar;
  return karma::generate(out, grammar, vec);
}

I would like to understand a couple of things:

1. Why don't I need to use karma::attr_cast anywhere? My start rule is a vector of std::shared_ptr whereas the elt rule works on the actual object const reference. I originally tried attr_cast but got nowhere, and sort of tried this version only halfheartedly just in case it worked, and it worked...
2. Why does it still compile if I comment out my custom transform_attribute altogether? Is there some default std::shared_ptr<T> -> T& transform_attribute provided? I couldn't find much, but maybe I'm not looking int the right place?
3. If I comment out my custom transform_attribute, as mentioned above, the code still compiled, but there's clearly some memory corruption at runtime. The karma::string generate garbage. In a way, I can understand that something funny must be happening since I don't even tell karma how to get from my shared_ptr to the objects. Is the fact that it compiles the actual error/bug?

Thanks a lot for your time and help!

Moffit

re. Manually attempting attr_cast, you may have run into an interesting bug that existed (at least in the Qi part of Spirit). You could try to add qi::copy() or boost::proto::deep_copy() around the parser sub-expression inside the attr_cast – Lenard

FWIW Here's where I found that bug with attr_cast: #19707754 – Lenard

Answer 1

1. each rule has an implicit attr_cast to the declared attribute type

2. Somehwere along the way Spirit's type compatibility rules go haywire. All I've seen is it has to do with the fact that the string type is a container. Somewhere along the way it "copy-constructs" a std::string that appears to have length 97332352. Unsurprisingly that is itself wrong and happens to trigger UB because the ranges that end up being passed to memset overlap:

   Source and destination overlap in memcpy(0x60c1040, 0x5cd2c90, 97332352)
      at 0x4C30573: memcpy@@GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      by 0x401B26: copy (char_traits.h:290)
      by 0x401B26: _S_copy (basic_string.h:299)
      by 0x401B26: _S_copy_chars (basic_string.h:342)
      by 0x401B26: void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) [clone .isra.53] (basic_string.tcc:229)
      by 0x402442: _M_construct_aux<char*> (basic_string.h:195)
      by 0x402442: _M_construct<char*> (basic_string.h:214)
      by 0x402442: basic_string (basic_string.h:401)
      by 0x402442: call<const boost::spirit::unused_type> (extract_from.hpp:172)
      by 0x402442: call<const boost::spirit::unused_type> (extract_from.hpp:184)
      by 0x402442: extract_from<std::__cxx11::basic_string<char>, boost::fusion::extension::adt_attribute_proxy<DataElement, 0, true>, const boost::spirit::unused_type> (extract_from.hpp:217)
      by 0x402442: extract_from<std::__cxx11::basic_string<char>, boost::fusion::extension::adt_attribute_proxy<DataElement, 0, true>, const boost::spirit::unused_type> (extract_from.hpp:237)
      by 0x402442: pre (attributes.hpp:23)
   

3. Yes, that's a QoI issue.

   The problem often is with c++'s implicit conversions. Pointer types have many unexpected conversions. Shared pointers do have their contextual conversion to bool.

More notes:

1. Your fusion adaptation seemed flawed: val was not being used in the setter

   BOOST_FUSION_ADAPT_ADT(DataElement, (std::string &, const std::string &, obj.str(), obj.str() = val))
   

2. You're doing many things I've learned to avoid.

   • I prefer to have semantic-action-less rules: Boost Spirit: "Semantic actions are evil"?
   • I don't do shared-pointers in Spirit grammars/generators How can I use polymorphic attributes with boost::spirit::qi parsers? (arguably, in a generator setting it's less of a problem!)
   • I don't do ADT adaptation (it's too easy to get bitten with UB) Error when adapting a class with BOOST_FUSION_ADAPT_ADT

Lenard

Thanks a lot! Indeed my adaptation was clunky, thanks for fixing. This is my first attempt at spirit, not too sure I see the point of karma so far, I think the reverse (i.e. parsing via qi) makes a lot more sense, but I rarely (never!) have to parse custom stuff, so I decided to give a go at a generator... – Moffit

I don't use Karma a lot. Most generators become interesting only when stateful (pretty-printing/formatting). I do use Qi a lot. I don't know what planet you live on when you don't have to parse text :) – Lenard