how to convert bash script to C++ using boost::iostreams

Question

I'm trying to convert the following bash code into C++ using boost::iostreams:

#!/usr/bin/bash
(
    gzip -cd file1.ext.gz
    cat file2.ext
) | grep '^regex' # or sed 's/search/replace/'

I can open a file and decompress it:

std::ifstream s("file.ext.gz", std::ios_base::in | std::ios_base::binary);
boost::iostreams::filtering_istreambuf in;
in.push(boost::iostreams::gzip_decompressor());
in.push(s);

Then open an uncompressed file:

std::ifstream s2("file.ext", std::ios_base::in | std::ios_base::binary);

Now I'm a bit stuck, so here are my questions:

1) What's the boost::iostreams solution to concat the two streams?

2) How to output the result through a regex filter to emulate grep/sed?

As a result I'd like to have a an istream that i can copy to cout:

boost::iostream::copy(result, std::cout);

UPDATE complete solution using Hamigaki's concatenate:

/*
 * convert the following bash script into C++
 *
 * #!/bin/bash
 * (
 *     gzip -cd file1.ext.gz
 *     cat file2.ext
 * ) | grep '^filter' | 'sed s/search/replace/g'
 *
 */

#include <iostream>
#include <boost/bind.hpp>
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/filter/regex.hpp>
#include <boost/iostreams/filter/grep.hpp>
#include <boost/iostreams/copy.hpp>

// http://hamigaki.sourceforge.jp/hamigaki/iostreams/concatenate.hpp
#include "concatenate.hpp"

namespace io = boost::iostreams;

int main(int argc, char const* argv[])
{
    io::file_source file1("file1.ext.gz");
    io::file_source file2("file2.ext");
    io::gzip_decompressor gzip;
    io::regex_filter sed(boost::regex("search"), "replace");
    io::grep_filter grep(boost::regex("^filter"));

    io::filtering_istreambuf in1(gzip | file1);
    io::filtering_istreambuf in2(file2);

    io::filtering_istreambuf combined(sed | grep | 
            hamigaki::iostreams::concatenate(
                boost::ref(in1),
                boost::ref(in2)
            )
        );

    io::copy(combined, std::cout);

    return 0;
}

Arv

As far as the shell side of things, do you know about zgrep (unless, of course, you need the file to be uncompressed)? Won't your script send the file to grep twice? – Minima

@Dennis: I think you misread the bash script. I have two files, one of them is compressed, and I want to concat them together and search(/replace) for a pattern in the result. – Arv

Sorry, I saw file.ext.gz and file.ext and took that to mean the second was the result of the first (I overlooked the -c). It would have been clearer if you had said file1.ext.gz and file2.ext. – Minima

@Dennis: I see. I edited the question accordingly. Sorry for the confusion. – Arv

Answer 1

1) I don't know if there's anything built into boost, but this class seems to be exactly what you want: http://hamigaki.sourceforge.jp/hamigaki/iostreams/concatenate.hpp

The catch here is that it expects CopyConstructible devices to concatenate and Chains seem to not be CopyConstructible. However, we can easily work around that using boost::ref. This code does (almost) what I understood you're asking:

int main(int argc, char const* argv[])
{
  boost::iostreams::filtering_istreambuf in;
  boost::regex regex("search");
  boost::iostreams::regex_filter rf(regex, "replace");
  in.push(rf);

  boost::iostreams::file_source file1(argv[1]);
  in.push(file1);

  boost::iostreams::file_source file2(argv[2]);
  boost::iostreams::copy(hamigaki::iostreams::concatenate(boost::ref(in), file2), std::cout);

  return 0;
}

I just used the regex filter instead of gzip, for testing.

2) boost::iostreams has a regex filter: http://www.boost.org/doc/libs/1_45_0/libs/iostreams/doc/classes/regex_filter.html

EDIT: You seem to have this working, now.

Stonge

I found both of these, but couldn't get any of them to work the way I want. If you can show how to solve my problem with these I'll accept it. – Arv

Forgot to say that boost::ref requires boost/bind.hpp – Stonge

Answer 2

1) Not available in boost

Hamigakis's concatenate sounds interesting, but I couldn't figure out how to use it to combine two boost::iostreams::chains. The code mentions it's meant for "concatenation of devices", so it might not be usable for chains. Please correct me if I'm wrong.

EDIT: updated my question with the complete solution.

2a) grep behavior (filter):

#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/filter/grep.hpp>

boost::iostreams::filtering_istreambuf in;
boost::regex regex("^search")
boost::iostreams::grep_filter grep(regex);
in.push(grep);

2b) sed behavior (search/replace):

#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/filter/regex.hpp>

struct formatter {
    std::string operator()(const boost::match_results<const char*>& match)
    {
        return str(boost::format("%s | %s") % match[2] % match[1]);
    }
};
boost::iostreams::filtering_istreambuf in;
boost::regex regex("^([a-z]+) ([0-9]+)");
boost::iostreams::regex_filter sed(regex, formatter());
in.push(sed);

Arv

@bta: No it is not. The answer is referring to my own question. – Arv