VoIP RTP Streaming from/to server (in Java) to/from android

Question

My target is to have a push-to-talk chat app in GSM/UMTS/LTE networks; initially I wanted use multicast addresses and peer-to-peer without overload the server; unfortunatly, after deep investigation, I discovered that multicast is not allowed in GSM/UMTS/LTE networks, therefore I have to use the server in order to bounce the VoIP packets. I' don't like very much this solution because I have to overload the server, but I didn't find any better solution. If you have an alternative solution is very much apprieciated...

Therefore I have to send VoIP from an android client to a server (PC), and viceversa. The server is in Java, it has to receive the VoIP packets and then to send the VoIP packets to other N clients; the server is a bouncer of VoIP packets.

I developped the code, but it doesn't work; I don't have any error, simply I had very bad VoIP service: I loose a lot of pieces and what I hear is very much noised... Where is the error? I suppose It should be in the server code; the server simply get the packet and resend them, without know that they are VoIP on RTP.

Please find below

• the code I use to send the VoIP packets to server. It works because I don't have problems when I use it for individual call sending the VoIP packets directly from Android to Android; the code to receive in android the packets from server is very similar, so I don't recopy it. As you can see I use android.net.rtp .
• the code I use on the Java server to bounce the VoIP packets

Thank you in advance, Fausto

// ANDROID CODE USED TO SEND VOIP TO SERVER

//Attribute definition
private static final AudioCodec myAudioCodec_COSTANTE = AudioCodec.PCMU ; 
private static final int myAudioGroupTX_COSTANTE = AudioGroup.MODE_NORMAL ; 
private static final int myAudioGroupRX_COSTANTE = AudioGroup.MODE_NORMAL ;
private static final int myRtpStreamTX_COSTANTE = RtpStream.MODE_SEND_ONLY ;
private static final int myRtpStreamRX_COSTANTE = RtpStream.MODE_RECEIVE_ONLY ; 
private static final int myAudioManagerTX_COSTANTE = AudioManager.MODE_IN_COMMUNICATION;
private static final int myAudioManagerRX_COSTANTE = AudioManager.MODE_IN_COMMUNICATION; 

//Method called for VoIP trasmission
myAudioStream = new AudioStream(localClientIP);
myAudioGroup = new AudioGroup();
myAudioManager =  (AudioManager) myContext.getSystemService(Context.AUDIO_SERVICE);

myAudioGroup.setMode(myAudioGroupTX_COSTANTE);
myAudioStream.join(null); 
myAudioStream.setCodec(myAudioCodec_COSTANTE);
myAudioStream.setMode(myRtpStreamTX_COSTANTE);
myAudioStream.associate(ipaddress_Server, port_Server)    
myAudioStream.join(myAudioGroup); 
myAudioManager.setMode(myAudioManagerTX_COSTANTE);
myAudioManager.setSpeakerphoneOn(false);
myAudioManager.setMicrophoneMute(false);

//JAVA SERVER CODE USED TO RECEIVE VOIP FROM ANDROID AND TO RESEND IT TO SERVER

DatagramSocket datagramSocket_RX_VoIP= new DatagramSocket();
DatagramSocket datagramSocket_TX_VoIP= new DatagramSocket();
int unicast_port_TX_VoIP = 5000 ;
String unicast_ip_TX_VoIP = "192.168.0.3";

    Thread t = new Thread(new Runnable() {
    public void run() {
    try {
        DatagramPacket myPacket;
        while (true) {
            myPacket = ManagePacket.initializePacket(); //Function to prepare the packe ; the problem is not here!!!
            datagramSocket_RX_VoIP.receive(myPacket);

            InetAddress ppp =  InetAddress.getByName(unicast_ip_TX_VoIP);
            myPacket.setAddress(ppp);
            myPacket.setPort( unicast_port_TX_VoIP ) ;
            datagramSocket_TX_VoIP.send(myPacket);
        }

    }  catch (Exception ex) {
        log.debug("Exception: " + ex.getMessage(), ex);
    }
    }
    });
    t.start();                                        

Ingrain

Answer 1

You don't give enough detail about your application. With any UDP streaming application you need to address the following issues:

• Network Jitter and Buffering: When a packet arrives, you cannot play the audio immediately after you receive it because the next packet might be later than expected and you will have a gap in your audio playback. The variance in the arrival rate is called network jitter. You need to buffer some amount of data before you try to play back. Usually you use some sort of ring buffer.

• Packet loss: There will be packet losses with UDP. You need to "deal" with this. If you send 10 packets and packet #4 is missing, you can't play packet #3 then packet #5. It will sound bad. Ways to deal with this:

  • Loss Concealment: Try to minimize the bad effect off a packet lost. You can play silence (although this doesn't sound the best unless you fade it to silence). You can "estimate" the lost audio by generating a missing audio by examining the surrounding packets.
  • Forward Error Correction: Send packets more than once. There are numerous ways and schemes. The tradeoff is higher latency and more network utilization
• Out of order arrivals: Packets may arrive out of order. Use the RTP sequence numbers to deal with this.

Audio streaming is not a trivial task. You can't just open a socket, send data, and play it on the other end and expect it to work. Your biggest concern with UDP streaming is network jitter and packet loss. If you don't want to deal with losses and you can have some extra latency, use TCP but be sure to buffer up enough audio before you start playing.

Echinate

Firt of all thank you very much; i tried to accept your answer but I got that I need 15 reputation...by the way, some doubts: I send VoIP by Android.net.rtp package, that I suppose should already take care of everything (RTP, jitter, FEC, etc...); the problem is what I have to use in order to receive in the server those packets and to resend them to the other adroids clients (they will use again android.net.RTP package to receive). ps: I thougth to have added all details about my app, what is missing? – Ingrain

android.net.RTP only implements RTP which is an application protocol on top of UDP. That is, rtp is only a specification of the data. The quality of the delivery of the data is completely abstracted from the format (specification) of the data. The delivery speed, FEC, loss concealment, jitter, buffering, etc. is implemented completely outside the spec of RTP. Thus, you have decide how that should be implemented based on your application. There is no single best way to do buffering, FEC, etc. It depends on your application. – Echinate

@Echinate - Your post is immensely helpful. Just saw your answer after posting my question. link. Can you please do a quick check on my implementation if I can still handle network jitter and buffering? – Masry

When you say to buffer some amount of data, do you mean to decrease the receiving buffer size? – Distressful

You need to get some amount of data ( maybe 50ms, maybe 200ms, maybe 2 seconds) before you can start playing. If you don't receive some data first, you will run out of data due to jitter. – Echinate

I would like to mention that these problems are just really normal if your going with the UDP protocol. For example packet loss & out-of-order arrivals, in order to avoid these problems you should probably switch to another protocol like TCP or RUDP. And voip applications generally require high scalability therefor your better off not using a regular datagram server but switch to something like netty. But either way if you wanna receive all packets and in order then you will sacrifice some network latency. So by correcting one problem you can make the other worse. – Distressful

And why exactly would you need to handle packet loss and out-of-order arrivals? Do we explicitly need every packet received and in the right order? UDP was made to use so that we can tolerate some packet loss... Like with any streaming application we usually can tolerate some packet loss... – Desinence

@Desinence You use UDP when low latency is important. Don't use UDP If low latency is not a high priority in your application. Of course when using with UDP, you must deal with it's side effects such as packet loss and out of order arrival. Out of order arrivals can be handled in the application by using sequence numbers in each packet. Packet loss can be addressed at the application level by either forward error correction or loss concealment. So you can use UDP for it's low latency and then deal with (but not completely avoid) the side effects at the application level. – Echinate