Getting text data from C++ using JNI through std::ostream into Java
Asked Answered
C

2

7

I have a class in C++ which takes an std::ostream as an argument in order to continuously output text (trace information). I need to get this text over to the Java side as efficiently as possible. What's the best way to do this? I was thinking of using a direct buffer, but another method would be to take all the function calls across to Java and do all the processing there, but it seems that I'd need a lot of JNI calls.

If an example could be shown of the exact implementation method, it would be very helpful, or if some code exists already to do this (perhaps part of another project). Another help would be to connect it up directly to a standard Java streaming construct, such that the entire implementation was completely transparent to the developer.

(Edit: I found Sharing output streams through a JNI interface which seems to be a duplicate, but not really of much help -- he didn't seem to find the answer he was looking for)

Coal answered 24/2, 2010 at 17:35 Comment(0)
K
5

The std::ostream class requires a std::streambuf object for its output. This is used by the fstream and stringstream classes, which use the features of ostream by providing a custom implementation of the streambuf class.

So you can write your own std::streambuf implementation with an overwritten overflow method, buffer the incomming chars in an internal stringbuffer. Every x calls or on eof/newline generate an java-string and call the print method of your java PrintStream.

An incomplete example class:

class JavaStreamBuff : std::streambuf
{
  std::stringstream buff;
  int size;
  jobject handle;
  JNIEnv* env

  //Ctor takes env pointer for the working thread and java.io.PrintStream
  JavaStreamBuff(JNIEnv* env, jobject jobject printStream, int buffsize = 50)
  {
     handle = env->NewGlobalRef(printStream);
     this->env = env;
     this->size = size;
  }
  //This method is the central output of the streambuf class, every charakter goes here
  int overflow(int in)
  {
    if(in == eof || buff.size() == size)
   {
     std::string blub = buff.str();

     jstring do = //magic here, convert form current locale unicode then to java string

     jMethodId id = env->(env->GetObjectClass(handle),"print","(java.lang.String)V");

     env->callVoidMethod(id,handle,do);

     buff.str("");
    }
    else
    {buff<<in;}
  }

  virtual ~JavaStreamBuff()
  {
     env->DeleteGlobalRef(handle);
  }
}

Missing:

  • Multithread support (the env pointer is only valid for the jvm thread)

  • Error handling (checking for java exceptions thrown)

  • Testing(written within the last 70 min)

  • Native java method to set the printstream.

On the java side you need a class to convert the PrintStream to a BufferedReader.

There have to be some bugs there, haven't spend enough time to work on them.
The class requires all access to be from the thread it was created in.

Hope this helps

Note
I got it to work with visual studio but I can't get it to work with g++, will try to debug that later.
Edit Seems that I should have looked for a more official tutorial on this bevore posting my answer, the MSDN page on this topic derives the stringbuffer in a different way.
Sorry for posting this without testing it better :-(.
A small correction to the code above in a more or less unrelated point: Just implement InputStream with a custom class and push byte[] arrays instead of Strings from c++.
The InputStream has a small interface and a BufferedReader should do most of the work.

Last update on this one, since im unable to get it to work on linux, even with the comments on the std::streambuf class stating that only overflow has to be overwritten.
This implementation pushes the raw strings into an inputstream, which can be read from by an other thread. Since I am too stupid to get the debugger working its untested, again.

//The c++ class
class JavaStreamBuf :public std::streambuf
{
  std::vector<char> buff;
  unsigned int size;
  jobject handle;
  JNIEnv* env;
public:
  //Ctor takes env pointer for the working thread and java.io.PrintStream
  JavaStreamBuf(JNIEnv* env, jobject  cppstream, unsigned int buffsize = 50)
  {
     handle = env->NewGlobalRef(cppstream);
     this->env = env;
     this->size = size;
     this->setbuf(0,0);
  }
  //This method is the central output of the streambuf class, every charakter goes here
  virtual int_type overflow(int_type in  = traits_type::eof()){
    if(in == std::ios::traits_type::eof() || buff.size() == size)
    {
        this->std::streambuf::overflow(in);
         if(in != EOF)
             buff.push_back(in);

         jbyteArray o = env->NewByteArray(buff.size());
         env->SetByteArrayRegion(o,0,buff.size(),(jbyte*)&buff[0]);
         jmethodID id = env->GetMethodID(env->GetObjectClass(handle),"push","([B)V");

         env->CallVoidMethod(handle,id,o);
         if(in == EOF)
             env->CallVoidMethod(handle,id,NULL);

         buff.clear();
    }
    else
    {
        buff.push_back(in);
    }

    return in;
  }

  virtual ~JavaStreamBuf()
  {
      overflow();
      env->DeleteGlobalRef(handle);
  }

//The java class
/**
 * 
 */
package jx;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InterruptedIOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

/**
 * @author josefx
 *
 */
public class CPPStream extends InputStream {

    List<Byte> data = new ArrayList<Byte>();
    int off = 0;
    private boolean endflag = false;
    public void push(byte[] d)
    {
        synchronized(data)
        {
            if(d == null)
            {
                this.endflag = true;
            }
            else
            {
                for(int i = 0; i < d.length;++i)
                {
                    data.add(d[i]);
                }
            }
        }
    }
    @Override
    public int read() throws IOException 
    {
        synchronized(data)
        {

            while(data.isEmpty()&&!endflag)
            {

                try {
                        data.wait();
                    } catch (InterruptedException e) {
                        throw new InterruptedIOException();
                    }
            }
        }
        if(endflag)return -1;
        else return data.remove(0);
    }
}

Sorry for wasting so much space^^(and time :-().

Kuban answered 3/3, 2010 at 18:44 Comment(7)
It's no problem about multithread support, I cracked that particular JNI nut ages ago :) Anyway, does this convert easily to an std::ostream ?Coal
std::ostream takes a streambuf as argument in its ctor. cplusplus.com/reference/iostream/ostream/ostreamKuban
An additional thought: putting the characters in a byte array and pushing them into a buffer for an InputStream could be less overhead to implement and the BufferedReader class would convert the input with the current localeKuban
Two obvious places to fix. :-) You need to call one of the *UTF* functions to convert from bytes to jstring. Also, the function signature should be using Ljava/lang/String;, not java.lang.String.Discontinue
@chris_Jester-Young true, I implemented the code without string and with single characters first, which required a call to java for each character. By now I believe its better to hand a byte[] array to java as extending InputStream requires only int read() instead of all the methods of BufferedInputStreamReader.Kuban
Cheers for linking me to the MSDN article :) The code is still useful, I have a basic skeleton of code to work with now.Coal
Just to say for anyone watching this, I managed to implement my own solution about a week ago which fully works. The implementation uses a line redirection buffer on the C++ side which overflows to a virtual function that takes in a line as string which you override as a subclass. So, I have a subclass with this method implemented that calls Java through JNI with each line. The Java InputStream (wrapped with BufferedReader) uses a CircularByteBuffer with infinite size to store data from C++. The buffers are linked using an integer ID (like a socket) which uses Map lookup on each side.Coal
P
0

It sounds as though the deliverable here is a subclass of ostream. The immediate question I'd want to be clear about is, will this class be responsible for buffering data until Java calls into it to retrieve, or is it expected to immediately (synchronously?) call via JNI to pass it on? That will be the strongest guide to how the code will shape up.

If you can reasonably expect the text to appear as a series of lines, I'd think about presenting them to Java in one line per call: this seems a fair compromise between the number of JNI calls and not unduly delaying the passing on of the text.

On the Java side I think you're looking at creating a Reader so that clients can pick up the text via a familiar interface, or perhaps a subclass of BufferedReader.

Paley answered 24/2, 2010 at 21:19 Comment(1)
It doesn't matter about passing it on immediately, it can wait until lines are complete and so on. I did try doing stuff with ostream, but have no idea where to proceed on this front. Really, I just want to get the text data into Java to process :)Coal

© 2022 - 2024 — McMap. All rights reserved.