Streaming voice between Android Phones over WiFi
Asked Answered
P

3

31

I'm trying to stream audio from the mic from 1 Android to another over WiFi. After looking at some examples I made 2 applications with a single activity in each, 1 to capture and send audio and the other to receive.

I've used the Audiorecord and Audiotrack classes to capture and play. However, i just hear some crackling sound (which has now stopped after i made some changes though i reverted back)

The activity to send voice.

public class VoiceSenderActivity extends Activity {

private EditText target;
private TextView streamingLabel;
private Button startButton,stopButton;

public byte[] buffer;
public static DatagramSocket socket;
private int port=50005;         //which port??
AudioRecord recorder;

//Audio Configuration. 
private int sampleRate = 8000;      //How much will be ideal?
private int channelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;    
private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;       

private boolean status = true;




@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    target = (EditText) findViewById (R.id.target_IP);
    streamingLabel = (TextView) findViewById(R.id.streaming_label);
    startButton = (Button) findViewById (R.id.start_button);
    stopButton = (Button) findViewById (R.id.stop_button);

    streamingLabel.setText("Press Start! to begin");

    startButton.setOnClickListener (startListener);
    stopButton.setOnClickListener (stopListener);
}

private final OnClickListener stopListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = false;
                recorder.release();
                Log.d("VS","Recorder released");
    }

};

private final OnClickListener startListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = true;
                startStreaming();           
    }

};

public void startStreaming() {


    Thread streamThread = new Thread(new Runnable() {

        @Override
        public void run() {
            try {


                int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
                DatagramSocket socket = new DatagramSocket();
                Log.d("VS", "Socket Created");

                byte[] buffer = new byte[minBufSize];

                Log.d("VS","Buffer created of size " + minBufSize);
                DatagramPacket packet;

                final InetAddress destination = InetAddress.getByName(target.getText().toString());
                Log.d("VS", "Address retrieved");


                recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,sampleRate,channelConfig,audioFormat,minBufSize);
                Log.d("VS", "Recorder initialized");

                recorder.startRecording();


                while(status == true) {


                    //reading data from MIC into buffer
                    minBufSize = recorder.read(buffer, 0, buffer.length);

                    //putting buffer in the packet
                    packet = new DatagramPacket (buffer,buffer.length,destination,port);

                    socket.send(packet);


                }



            } catch(UnknownHostException e) {
                Log.e("VS", "UnknownHostException");
            } catch (IOException e) {
                Log.e("VS", "IOException");
            } 


        }

    });
    streamThread.start();
 }
 }

The activity to receive voice

public class VoiceReceiverActivity extends Activity {


private Button receiveButton,stopButton;

public static DatagramSocket socket;
private AudioTrack speaker;

//Audio Configuration. 
private int sampleRate = 8000;      //How much will be ideal?
private int channelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;    
private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;       

private boolean status = true;


@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    receiveButton = (Button) findViewById (R.id.receive_button);
    stopButton = (Button) findViewById (R.id.stop_button);
    findViewById(R.id.receive_label);

    receiveButton.setOnClickListener(receiveListener);
    stopButton.setOnClickListener(stopListener);

}


private final OnClickListener stopListener = new OnClickListener() {

    @Override
    public void onClick(View v) {
        status = false;
        speaker.release();
        Log.d("VR","Speaker released");

    }

};


private final OnClickListener receiveListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
        status = true;
        startReceiving();

    }

};

public void startReceiving() {

    Thread receiveThread = new Thread (new Runnable() {

        @Override
        public void run() {

            try {

                DatagramSocket socket = new DatagramSocket(50005);
                Log.d("VR", "Socket Created");


                byte[] buffer = new byte[256];


                //minimum buffer size. need to be careful. might cause problems. try setting manually if any problems faced
                int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);

                speaker = new AudioTrack(AudioManager.STREAM_MUSIC,sampleRate,channelConfig,audioFormat,minBufSize,AudioTrack.MODE_STREAM);

                speaker.play();

                while(status == true) {
                    try {


                        DatagramPacket packet = new DatagramPacket(buffer,buffer.length);
                        socket.receive(packet);
                        Log.d("VR", "Packet Received");

                        //reading content from packet
                        buffer=packet.getData();
                        Log.d("VR", "Packet data read into buffer");

                        //sending data to the Audiotrack obj i.e. speaker
                        speaker.write(buffer, 0, minBufSize);
                        Log.d("VR", "Writing buffer content to speaker");

                    } catch(IOException e) {
                        Log.e("VR","IOException");
                    }
                }


            } catch (SocketException e) {
                Log.e("VR", "SocketException");
            }


        }

    });
    receiveThread.start();
}

}

I used wireshark to check if the packets are being sent and i can see the packets. The source however, is the MAC address of the sending device and destination too something like a physical address. Not sure if this is relevant though.

So what's the problem with?

Pluvious answered 11/2, 2012 at 6:41 Comment(9)
There are at least three problems you have to deal with: delayed (or missing) data, overall data throughput, and the possibility of slightly mismatched sampling frequencies. Practical IP telephony has to have a means of dealing with all three. Mismatched clocks is surprisingly tricky - initially you can introduce a delay to give some buffering allowance, but if the sender is slower you will use up the buffer and the receiver will be starved for data; while if the sender is faster the buffer will eventually overflow with unplayed data.Improvised
I did manage to get this working actually. Didn't really have a problem of mismatched frequencies. Delay in data, yes. Had a sort of protocol of my own to match the receiver/sender clocks. At the end it did work but only with some lag (which increased with distance from the wireless router)Pluvious
Hey there, I implemented a test application for the code you have above, making all the necessary changes suggested below butI am still having a problem. I am getting comms between the two phones no problem, but I don't think the mic is recording properly as I am not hearing any sound on the other end. Do you perhaps have a link to a sample solution that I could take a look at?Tripterous
@Tripterous What do you mean by you're getting comms between the two phones? Do you mean you can hear some sound but its garbled? Try adjusting your sampling rate and buffer size and get the best fit.Pluvious
@Pluvious Thanks for replying, I fixed my problem, I ended up changing the sample rate from 8K to 44100k. Thanks for the help. Now I have on android phone sending the audio data to a small server on my laptop, do you know how I would then play this stream on the laptop? Should I output it to a mp3 file perhaps?Tripterous
@Pluvious did you managed to reduce the lag? I implemented you code using 44100 HZ and a buffer size of 4096 and it´s working fine, but it´s a little bit lag.Clarinda
@Pluvious In multiple places you've left comments saying you got it working. Please post your own answer and accept to help others know exactly what was wrong with the code in your question and how you addressed itYell
No sound on speaker even though data is being written to it.. what could be the issue?Paucity
@Pluvious This solution does not work! if you have a working code sample please provide it!Trainor
P
8

Hey there is an Open Source library called "Libstreaming" that is used for streaming voice/video over the network using WIFI. Just Have a look at it:

https://github.com/fyhertz/libstreaming

There are also some examples provided, kindly have a look at it:

https://github.com/fyhertz/libstreaming-examples

I have used the library to Stream RTSP Audio over the network,hope it may be useful.

Pechora answered 3/12, 2014 at 4:43 Comment(2)
Can u guide, how did you do only audio stream without streaming video ?Calvary
how to do we strea,m only the auidoZackzackariah
D
3

I'd try dividing the problem into Three parts.

Part 1

Ensure the Socket Connection is working fine by commenting everything related to audio

Part 2

Send simply just an arbitrary text message [Hello WiFi] from the sender, then receiving and printing it in the receiver side application.

Part 3

Whether the recorder is actually working? try to test your recording way in separate project to see if it is working properly or not.

Use this code to capture microphone and play it.

My experience

I once worked on a similar project and to test it, what I did was after recording I wrote the recorded audio data as a file on the sdcard

(it would be raw audio, so most music players won't be able to play it... mPlayer should play it I guess)

Dichy answered 16/2, 2012 at 6:30 Comment(4)
Ok I got it working. The voice breaks up too much and there's lag. Need to find out the correct sampling rate and buffer size for this. Would be great if any inputs on that. Anyway, thanks a lot. what you said helped.Pluvious
In your receiver activity, in startReceiving() method don't use 256 as the buffer size but use the minBufSize that you're getting in the next line. Other than that, might want to play a bit with different sampling rates but even 8k should be good.Dichy
Ok i got it working. Apparently the minBufSize was too much and hence the lag and breaking. Set the minBufSize to 256 and the while initializing the AudioRecord and AudioTrack objects set the buffer size to minBufSize*10.. Tried different combinations of sampling rates and this and got a satisfactory one now. Thanks a lot!Pluvious
Hi Alabhya, did you set the minBufSize to 256 in the both receiving and streaming methods?Volleyball
H
2

You need to carefully consider your use of UDP (DatagramSocket class) as your network protocol.

UDP is a lightweight protocol that does not guarantee to maintain the ordering of received packets. This may be part of the reason why the audio is garbled. A packet received out of order will result in a packets worth of audio being played out of order. At the boundary of these out-of-sequence packets you will hear clicks/pops where the audio sample is effectively corrupt. In addition to this, UDP packets aren't guaranteed to be delivered successfully. Any dropped packets will obviously add to any garbling or distortion that is heard.

TCP (Socket class) would be a better option for optimum audio quality. TCP is a more robust protocol that will maintain the order that packets are received. It also has in-built error checking and will resend any dropped packets. However, because of this attentional functionality, TCP has a higher network overhead.

I started this response by saying that you need to carefully consider which protocol you use. This is because there is a case for using either depending on what is important to you..

If you want ultra low latency playback but are happy to sacrifice the audio quality then UDP will work. However, it will take some experimentation to find the best buffer and sample size.

If you want the best possible audio re-production with zero distortion but are happy to introduce slightly more latency then TCP is the route to go.

I can't say how much more latency TCP would add. But it is possible that it could be implemented without impacting the user experience. The only way to find out is to try it and see.

Hambley answered 31/7, 2014 at 10:40 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.