"Endless" AudioInputStream from socket

Question

I have a problem with creation of AudioInputStream from Socket. Here are the important parts:

public class SoundStream extends Thread {
    private int port;
    private String IP;
    private Socket socket;

    private SoundObject soundObject;

    private OpenAL openAL;
    private Source source;

    private boolean run = true;

    public SoundStream(int port, String IP, SoundObject soundObject) {
        this.soundObject = soundObject;
        this.port = port;
        this.IP = IP;
    }

    public void run() {
        try {
            this.socket = new Socket(this.IP, this.port);
            this.openAL = new OpenAL();
        } catch (Exception e) {
            e.printStackTrace();
        }
        this.mainCycleMethod();
    }

    private void mainCycleMethod() {
        while (run) {
            this.soundObject.blockAndWait();
            switch (this.soundObject.getAndResetEvent()) {
                case 0:
                    this.run = false;
                    this.close();
                    break;
                case 1:
                    this.setPitch();
                    break;
                case 2:
                    this.closeSource();
                    this.play();
                    break;
                case 3:
                    this.pause(true);
                    break;
                case 4:
                    this.pause(false);
                    break;
            }
        }
    }

    private BufferedInputStream getInputStream() throws Exception {
        return new BufferedInputStream(socket.getInputStream());
    }

    private void setPitch() {
        if(this.source != null) {
            try {
                this.source.setPitch(this.soundObject.getPitch());
            } catch (ALException e) {
                e.printStackTrace();
            }
        }
    }

    private void play() {
        try {
            AudioInputStream audioInputStream = new AudioInputStream(this.getInputStream(), this.soundObject.getAudioFormat(), AudioSystem.NOT_SPECIFIED);
//            AudioInputStream audioInputStream_tmp = AudioSystem.getAudioInputStream(this.getInputStream());
//            AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(this.soundObject.getAudioFormat(), audioInputStream_tmp);
            this.source = openAL.createSource(audioInputStream);
            this.source.setGain(1f);
            this.source.play();
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }

    private void close() {
        this.closeSource();
        this.openAL.close();
        try {
            this.socket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private void closeSource() {
        if(this.source!=null) {
            this.source.close();
        }
    }

    private void pause(boolean pause) {
        if(this.source != null) {
            try {
                if (pause) {
                    this.source.pause();
                } else {
                    this.source.play();
                }
            } catch (ALException ex) {
                ex.printStackTrace();
            }
        }
    }
}


public class SoundObject extends AbstractEventObject {
    public AudioFormat getAudioFormat() {
        boolean signed = false;
        //true,false
        boolean bigEndian = false;
        //true,false
        return new AudioFormat(this.frequency, this.bits, this.channels, signed, bigEndian);
    }
.
.
.
.
}

This code throws UnsupportedAudioFileException at this line:

AudioInputStream audioInputStream_tmp = AudioSystem.getAudioInputStream(this.getInputStream());

However when I use this code:

AudioInputStream audioInputStream = new AudioInputStream(this.getInputStream(), this.soundObject.getAudioFormat(), 100000);

it plays the sound but only after it loads those 100000 sample frames to the audioinputstream. After it plays all the 100000 frames it finishes.

I guess that I would solve this issue if I could pass the AudioFormat directly as a parameter during the first AudioInputStream inicialization, but it doesn't seem to be possible. I'm receiving the audio format specifications from server.

I think that one possible solution would be to create a dataline which I can pass as a parametr to AudioInputStream constructor. However I'm not sure how to get the data from the socket directly to dataline. I know of a solution that uses infinite loop, in which it reads the data and writes them to the dataline. But it seems to be wasteful. Is there a more direct approach?

I hope it's possible to solve using java-openAL library, because I need to change the speed and I hope I won't have to do it myself.

Thanks

Circinate

As a first step you can try using AudioInputStream audioInputStream = new AudioInputStream(this.getInputStream(), this.soundObject.getAudioFormat(), AudioSystem.NOT_SPECIFIED); and see what happens. – Hamforrd

It doesn't solve my problem, because it gets blocked by openAL.createSource(audioInputStream) method. It probably waits for the whole InputStream to finish. Thanks – Powel

What is openAL? Can you show the full source code (a minimal reproducible example, probably)? – Hamforrd

I modified my post, now there is the whole code. openAL is just an initialization of the OpenAL class. @Hamforrd – Powel

And where can I found the source code for OpenAL class? – Hamforrd

It's an open-source library. It's github location is here. @Hamforrd – Powel

My two cents for this: Have you tried with implementing TargetDataLine? To me this seems like the more likely way to deal with "permanent" input. – Pattypatulous

I was thinking about this solution too, but I haven't found a way to initialize SourceDataLine yet. So it doesn't seem to be an "official way". However I'll definitely look into it further. @Pattypatulous – Powel

Hm - I'd say create a class that implements Target DataLine and see where that gets you. Source and Target are named strange... – Pattypatulous

The problem with openAL.createSource() is that it reads the entire stream contents into buffer. For streaming you'll need to read portions of the stream into individual buffers, queue them, wait for some of them to be processed, unqueue, repeat. Some useful links: How do I stream audio into OpenAL Sources?, Ogg/Vorbis Streaming. – Hamforrd

It makes sense. I'll try to add support to Java OpenAL. It's weird that there aren't any libraries solving this issue, it seems to be quite common problem. Thanks for the direction. @Hamforrd – Powel

Answer 1

I've finally solved the issue. As it turned out java-openAL has streaming support built in, but it wasn't in the documentation on GitHub so I didn't notice at first. There is a createOutputStream method in the Source class, which returns the OutputStream. You can write the bytes directly to the OutputStream.

Here is my code:

In this snippet I initialize OpenAL:

public void run() {
    try {
        this.socket = new Socket(this.IP, this.port);
        this.openAL = new OpenAL();
    } catch (Exception ex) {
        Log.severe(ex.toString());
    }
    this.mainCycleMethod();
}

Here is my play method which is called when the InputStream is available:

private void play() {
    try {
        this.source = openAL.createSource();
        this.outputWriter = new OutputWriter(this.socket.getInputStream(), this.source, this.soundObject.getAudioFormat());
        this.source.setGain(1f);
        this.outputWriter.start();
    } catch (Exception ex) {
        Log.severe(ex.toString());
    }
}

You have to use createSource method without parameters, it returns new instance of Source. Don't call the play method on source, it's handled by the SourceOutputStream class which instance is returned by the createOutputStream method. There is nothing wrong with calling the play method manually but I had a bad experience doing so when the buffers are empty. Basically it doesn't start playing later on when you begin streaming the data to OpenAL.

Here is my OutputWriter code which takes care of passing the bytes from InputStream to OutputStream:

package cz.speechtech.sound;

import org.urish.openal.ALException;
import org.urish.openal.Source;

import javax.sound.sampled.AudioFormat;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

/**
 * Created by honza on 16.12.15.
 */
public class OutputWriter extends Thread {
    private InputStream inputStream;
    private OutputStream outputStream;

    private int STREAMING_BUFFER_SIZE = 24000;
    private int NUMBER_OF_BUFFERS = 4;

    private boolean run = true;

    public OutputWriter(InputStream inputStream, Source source, AudioFormat audioFormat) {
        this.inputStream = inputStream;
        try {
            this.outputStream = source.createOutputStream(audioFormat, this.NUMBER_OF_BUFFERS, 1024);
        } catch (ALException e) {
            e.printStackTrace();
        }
    }

    public void run() {
        byte[] buffer = new byte[this.STREAMING_BUFFER_SIZE];
        int i;
        try {
            Thread.sleep(1000); // Might cause problems
            while (this.run) {
                i = this.inputStream.read(buffer);
                if (i == -1) break;
                outputStream.write(buffer, 0, i);
            }
        } catch (IOException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    public synchronized void stopRunning() {
        this.run = false;
        try {
            this.outputStream.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

Have a nice day.

Circinate