Java Record Mic to Byte Array and Play sound

Question

I want to make a live voice chatting program in Java, but I know nothing about recording/playing sound in Java, so with the help of Google, I think I have been able to record from my mic to a byte array with the following:

AudioFormat format = new AudioFormat(8000.0f, 16, 1, true, true);
TargetDataLine microphone;
try{
    microphone = AudioSystem.getTargetDataLine(format);

    DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
    microphone = (TargetDataLine)AudioSystem.getLine(info);
    microphone.open(format);

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    int numBytesRead;
    byte[] data = new byte[microphone.getBufferSize()/5];
    microphone.start();

    int bytesRead =0;

    try{
        while(bytesRead<100000){ //Just so I can test if recording my mic works...
            numBytesRead = microphone.read(data, 0, data.length);
            bytesRead = bytesRead + numBytesRead;
        //    System.out.println(bytesRead);
            out.write(data, 0, numBytesRead);
        }
    catch(Exception e){
        e.printStackTrace();
    }
    microphone.close();

catch(LineUnavailibleException e){
    e.printStackTrace();
}

So now, to my understanding, if I call out.toByteArray();, I should have gotten a byte array of the sound I just recorded from my microphone. (I got no errors running the above, but have no way to prove if it actually recorded because I do not wish to output it to a file and didn't do so)

Now, if the above is correct, then below is where I run into my problem: I want to now play the byte array I just created... (In my real program, I would've sent the bytes over to my "receiving program" through a Java socket which I have already been able to do, but right now I just want to make a small program that records the mic and plays it back). In order to play the sound information from the byte array I followed this: http://www.wikijava.org/wiki/Play_a_wave_sound_in_Java And came up with the following: (this is located right after microphone.close() from the above)

try{
    DataLine.Info info2 = DataLine.Info(SourceDataLine.class, format);
    SourceDataLine dataLine = (SourceDataLine)AudioSystem.getLine(info2);
    int bufferSize = 2200;
    soundLine.open(format, bufferSize);
    soundLine.start();
    AudioInputStream audioInputStream = null;

    InputStream input = new ByteArrayInputStream(out.toByteArray());
    audioInputStream = AudioSystem.getAudioInputStream(input);

    ...

The rest is pretty much copy pasted from the playSound.java from this link: http://www.wikijava.org/wiki/Play_a_wave_sound_in_Java

When I run the above code... The recording seems to work all right, but I get the following error:

javax.sound.sampled.UnsupportedAudioFileException: could not get audio input stream from input stream

for this line audioInputStream = AudioSystem.getAudioInputStream(input);

From my limited knowledge, I'm assuming it's because I somehow messed up the recording method, I need some sort of "Audio Format Header?" https://ccrma.stanford.edu/courses/422/projects/WaveFormat/ (I assumed I wouldn't need something like that because I never saved to a file and just kept it all as a byte array), or I just completely misunderstood how java's AudioInputStream reads and parses data...

This is my first time working with any sound related things in Java, so I apologize if I am completely misunderstanding and butchering this code (yeah I know the code looks pretty bad and unorganized but I just want to get it to work)... I tried multiple searches on Google/StackOverflow, and was able to find a very similar question:

java byte array play sound

but it was also unanswered (the only answer was to save it to a file, but what we both want is to stream it directly as a byte array without it ever becoming a file)

What I do know:

Audio can be recorded using a TargetDataLine, and recording the microphone, which can be outputted to a Byte Array using a ByteArrayOutputStream

Audio can be saved to a file and played by using a AudioInputStream to read the file and a SourceDataLine to play the data.

If I wanted to write a file, I could use AudioSystem.write(new AudioInputStream(microphone), AudioFileFormat.Type.WAVE, new File("recording.wav"); //I have tested this by replacing the while loop with this line and it recorded fine (except it would never stop so I had to manually terminate it), but I don't want that, because outputting to a file means it will be impossible to send it over a socket to another side in real time.

What I don't know / My Question:

How to record and stream audio recorded from a mic to another computer that can be played with as little delay as possible (pretty much like a voice chat similar to Skype) with Java.

Thanks in advance for any help or someone who can point me in the correct direction. Also if someone knows a simpler method then please tell me that as well.

Chilon

#14188167 – Valedictory

I actually saw that Question, but because the code they had was really small, I had no idea what technique they used to display. For example, I don't know what the line, or buffer are used for/what they are. Can you explain what line is/how to initiate, and what the ByteBuffer does? – Chilon

I can give a more detailed answer later but for now check out the java tutorial on audio I/O (there's multiple pages) docs.oracle.com/javase/tutorial/sound/accessing.html – Valedictory

I have already seen the tutorial but I only skimmed through it the first time, Ill be sure to check it out in detail ASAP, but would really want your answer too. Thanks. – Chilon

Answer 1

EDIT: Here's a slightly better version of the same idea as below that will playback directly as you record

AudioFormat format = new AudioFormat(8000.0f, 16, 1, true, true);
    TargetDataLine microphone;
    SourceDataLine speakers;
    try {
        microphone = AudioSystem.getTargetDataLine(format);

        DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
        microphone = (TargetDataLine) AudioSystem.getLine(info);
        microphone.open(format);

        ByteArrayOutputStream out = new ByteArrayOutputStream();
        int numBytesRead;
        int CHUNK_SIZE = 1024;
        byte[] data = new byte[microphone.getBufferSize() / 5];
        microphone.start();

        int bytesRead = 0;
        DataLine.Info dataLineInfo = new DataLine.Info(SourceDataLine.class, format);
        speakers = (SourceDataLine) AudioSystem.getLine(dataLineInfo);
        speakers.open(format);
        speakers.start();
        while (bytesRead < 100000) {
            numBytesRead = microphone.read(data, 0, CHUNK_SIZE);
            bytesRead += numBytesRead;
            // write the mic data to a stream for use later
            out.write(data, 0, numBytesRead); 
            // write mic data to stream for immediate playback
            speakers.write(data, 0, numBytesRead);
        }
        speakers.drain();
        speakers.close();
        microphone.close();
    } catch (LineUnavailableException e) {
        e.printStackTrace();
    } 

Bear with me because this is really rough, but it gets the recorded audio playing through speakers;

In order to make it sound better, you will need to add threads, and optimize the input/output streams.

http://www.developer.com/java/other/article.php/1579071/Java-Sound-Getting-Started-Part-2-Capture-Using-Specified-Mixer.htm

package audio;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.SourceDataLine;
import javax.sound.sampled.TargetDataLine;

public class AudioTest {

    public static void main(String[] args) {

        AudioFormat format = new AudioFormat(8000.0f, 16, 1, true, true);
        TargetDataLine microphone;
        AudioInputStream audioInputStream;
        SourceDataLine sourceDataLine;
        try {
            microphone = AudioSystem.getTargetDataLine(format);

            DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
            microphone = (TargetDataLine) AudioSystem.getLine(info);
            microphone.open(format);

            ByteArrayOutputStream out = new ByteArrayOutputStream();
            int numBytesRead;
            int CHUNK_SIZE = 1024;
            byte[] data = new byte[microphone.getBufferSize() / 5];
            microphone.start();

            int bytesRead = 0;

            try {
                while (bytesRead < 100000) { // Just so I can test if recording
                                                // my mic works...
                    numBytesRead = microphone.read(data, 0, CHUNK_SIZE);
                    bytesRead = bytesRead + numBytesRead;
                    System.out.println(bytesRead);
                    out.write(data, 0, numBytesRead);
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
            byte audioData[] = out.toByteArray();
            // Get an input stream on the byte array
            // containing the data
            InputStream byteArrayInputStream = new ByteArrayInputStream(
                    audioData);
            audioInputStream = new AudioInputStream(byteArrayInputStream,format, audioData.length / format.getFrameSize());
            DataLine.Info dataLineInfo = new DataLine.Info(SourceDataLine.class, format);
            sourceDataLine = (SourceDataLine) AudioSystem.getLine(dataLineInfo);
            sourceDataLine.open(format);
            sourceDataLine.start();
            int cnt = 0;
            byte tempBuffer[] = new byte[10000];
            try {
                while ((cnt = audioInputStream.read(tempBuffer, 0,tempBuffer.length)) != -1) {
                    if (cnt > 0) {
                        // Write data to the internal buffer of
                        // the data line where it will be
                        // delivered to the speaker.
                        sourceDataLine.write(tempBuffer, 0, cnt);
                    }// end if
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
            // Block and wait for internal buffer of the
            // data line to empty.
            sourceDataLine.drain();
            sourceDataLine.close();
            microphone.close();
        } catch (LineUnavailableException e) {
            e.printStackTrace();
        }
    }
}

Valedictory

note that this doesn't play any of the audio until AFTER it's all recorded – Valedictory

But using a thread I'm pretty sure I can get it to play while recording, Ill test your answer as soon as I get to a computer and report whether or not it works! – Chilon

you could also write directly to the sourceDataLine's input stream during the read loop – Valedictory

Thanks it worked perfectly (both examples), but I have one last question, there is a delay of about 0.5-1 sec between me talking and it playing back (the instant playback example, which is closer to what I want to make)... I know there has to be, but is there a way to minimalize that delay? – Chilon

There's always going to be a delay, but you can try making the chunk size smaller so that it reads less from the mic before outputting. There has to be something in the output buffer before it can be written – Valedictory

I have one last question, I got the code to work (sending the sound from my program's mic to the other program which played it to a speaker like how skype calls work) on my personal computers, but when I gave it to a friend I would receive data but nothing would be played. I also tried on my other computer and it worked fine, and even tried it with my other computer connected through mobile thethering to the internet to simulate connecting from outside, and that worked too, so I don't understand why my friend's would only send information over but nothing would play through my speakers. – Chilon

From my testing, I think most likely is that my friend's mic isn't being recorded for some reason... Even though the code works fine for my 2 computers, is it possible that the mic can't be picked up? I think my friend uses a USB headset, so could that be a problem? – Chilon

Update: I fixed it, the problem was it was recording from the wrong mic. Is there a way to determine what device I'm recording with, and/or display all possible devices and then choose which one to use? – Chilon

I think if you loop through the Mixer Info, you can find all the different available target sources; check out the answer here: #12863581 – Valedictory

Mind if I ask where the number 100000 comes from in the while loop? – Just

@arin 100000 was arbitrary. It was used in the example from the question, and it will limits the length of recording for testing purposes – Valedictory

@Valedictory Thanks. The answer was really helpful, although I had a different problem altogether. :) – Coypu

Aren't SourceDataLine and TargetDataLine named the wrong way around? – Cantabrigian

What's going on here with microphone being overridden in the code sample? – Cantabrigian