Jitsi - Playing a WAV file during call - If possible, mix with audio

Question

I am customizing Jitsi to play a Wav file when a call is in progress.

I am facing trouble doing it, and would appreciate if you can help me out.

I can switch the data source before the call starts, by using a custom AudioFileMediaDevice and switching it on in CallPeerMediaHandler.

But I am having problems in replacing the datasource when the call is in progress.

=============================================================

I've tried the following but couldn't make it work.

1) I tried getting the device's output datasource and added a URLDatasource of the wav file using addInDataSource method. Didn't work.

DataSource dataSource = device.createOutputDataSource();

DataSource fileDataSource = Manager.createDataSource(new URL("file://resources/sounds/Sample.wav"));
((AudioMixingPushBufferDataSource)dataSource).addInDataSource(fileDataSource);

2) I tried adding a custom Capture device and switch it, but its not working too:

CaptureDeviceInfo2 fileDevice =
                new CaptureDeviceInfo2("Recorded Audio 1",
                    fileDataSource.getLocator(), null, null, null, null);

            ((MediaServiceImpl) LibJitsi.getMediaService())
                .getDeviceConfiguration().getAudioSystem().setDevice(AudioSystem.DataFlow.CAPTURE, fileDevice, false);

This is working for playback though, not as a capture device.

3) I even tried adding a new Audio system with the playback device as the file data source, but thats not working too.

=============================================================

I am new to libjitsi, so I'm having tough time trying to decode what is happening. Any directions on how to resolve this would be great.

Punish

Did you solved this task? I trying to do the same and get stuck. – Saving

If what you're looking for can also be solved by playing a .WAV file from normal java, let me know. – Translatable

Answer 1

I made playback sound in call with this code:

public void startPlaying(CallPeer callPeer, DataSource soundDataSource) throws OperationFailedException {
    assert callPeer instanceof CallPeerSipImpl;
    CallPeerSipImpl cp = (CallPeerSipImpl) callPeer;
    AudioMediaStreamImpl audioMediaStream = (AudioMediaStreamImpl) cp.getMediaHandler().getStream(MediaType.AUDIO);
    AudioMediaDeviceSession deviceSession = audioMediaStream.getDeviceSession();
    assert deviceSession != null;
    assert deviceSession.getDevice() instanceof AudioMixerMediaDevice;
    AudioMixerMediaDevice dev = (AudioMixerMediaDevice) deviceSession.getDevice();
    dev.getAudioMixer().addInDataSource(soundDataSource);
}

Note that AudioMixerMediaDevice.getAudioMixer() has private access in libjitsi, so I made it public and recompiled.

Saving

Answer 2

I needed to play an audio file during a call, but only on the remote side of the call. So I played around a little bit with stokitos example and modified it for my needs. In case somebody ever needs it, here is what I did:

private void playAudioFromDataSource(final CallPeerSipImpl callPeer, final DataSource audioDataSource, final MediaDirection direction) {
    final CallPeerMediaHandlerSipImpl mediaHandler = callPeer.getMediaHandler();
    final AudioMediaStreamImpl audioMediaStream = (AudioMediaStreamImpl) mediaHandler.getStream(AUDIO);
    final AudioMediaDeviceSession deviceSession = audioMediaStream.getDeviceSession();
    if (null != deviceSession) {
        if (RECVONLY == direction) {
            // plays audio local only:
            deviceSession.addPlaybackDataSource(audioDataSource);
        } else {
            final AudioMixerMediaDevice mediaDevice = (AudioMixerMediaDevice) deviceSession.getDevice();
            final AudioMixer audioMixer = getAudioMixer(mediaDevice);
            if (null != audioMixer) {
                if (SENDONLY == direction) {
                    // plays audio remote only:
                    audioMixer.getLocalOutDataSource().addInDataSource(audioDataSource);
                } else if (SENDRECV == direction) {
                    // plays audio on both sides of call (local and remote):
                    audioMixer.addInDataSource(audioDataSource);
                }
            }
        }
    }
}

private AudioMixer getAudioMixer(final AudioMixerMediaDevice device) {
    try {
        final Method privateGetAudioMixerMethod = device.getClass().getDeclaredMethod("getAudioMixer");
        privateGetAudioMixerMethod.setAccessible(true);
        final Object audioMixerObject = privateGetAudioMixerMethod.invoke(device, (Object[]) null);
        return (AudioMixer) audioMixerObject;
    } catch (final Exception e) {
        log.error("Could not get AudioMixer", e);
    }
    return null;
}

NOTE: I sued reflection to get the private AudioMixer object. I admit it's not the cleanest approach, but it works. :)

Proximal