How to join one audio and video file -- Xuggler

Question

I want to combine a video file (flv) with no audio with an audio file (mp3) using Xuggler. At the moment I have taken two streams and combined the video and audio parts of those streams separately like picture in picture. Now i want to combine the audio and video files with each other.. Any suggestion or hints will be appreciated. I am using red5 server. Thanks.

Hung

Answer 1

Use a MediaConcatenator. See example code "Concatenate Audio And Video"

Reluctant

it concatenate mate. We need both audio and video to run simultaneously... this first adds video and then audio file. We don't need that – Serrell

Thanks for your response. But i want to make one file with audio of the second file running for the video of the first file. – Hung

So instead of concatenate, you could try to create an IContainer with 2 streams 1 for audio 1 for video. Then decode mp3 and flv in a loop, create an IPacket then use encodeVideo and encodeAudio with the same packet. Once your packet is complete use writePacket (on your output IContainer) to record video and audio. I think I saw a tutorial on xuggle's wiki explaining how to do such thing but I couldn't find it back. – Reluctant

I found a link that could help to merge your audio and video file. There is no code in it but some useful advices to do read: groups.google.com/group/xuggler-users/browse_thread/thread/… – Reluctant

Answer 2

I developed a class that will take two inputs, one audio file and second video file and will merge them to a single audio video file.

 public static void main(String[] args)
  {

    String filenamevideo = "f:/testvidfol/video.mp4"; //video file on your disk
    String filenameaudio = "f:/testvidfol/audio.wav"; //audio file on your disk


    IMediaWriter mWriter = ToolFactory.makeWriter("f:/testvidfol/videowriter.flv"); //output file

    IContainer containerVideo = IContainer.make();
    IContainer containerAudio = IContainer.make();

    if (containerVideo.open(filenamevideo, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + filenamevideo);

    if (containerAudio.open(filenameaudio, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + filenameaudio);

    int numStreamVideo = containerVideo.getNumStreams();
    int numStreamAudio = containerAudio.getNumStreams();

    System.out.println("Number of video streams: "+numStreamVideo + "\n" + "Number of audio streams: "+numStreamAudio );

int videostreamt = -1; //this is the video stream id
int audiostreamt = -1;

IStreamCoder  videocoder = null;

    for(int i=0; i<numStreamVideo; i++){
        IStream stream = containerVideo.getStream(i);
        IStreamCoder code = stream.getStreamCoder();

        if(code.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO)
        {
            videostreamt = i;
            videocoder = code;
            break;
        }

    }

    for(int i=0; i<numStreamAudio; i++){
        IStream stream = containerAudio.getStream(i);
        IStreamCoder code = stream.getStreamCoder();

        if(code.getCodecType() == ICodec.Type.CODEC_TYPE_AUDIO)
        {
            audiostreamt = i;
            break;
        }

    }

    if (videostreamt == -1) throw new RuntimeException("No video steam found");
    if (audiostreamt == -1) throw new RuntimeException("No audio steam found");

    if(videocoder.open()<0 ) throw new RuntimeException("Cant open video coder");
    IPacket packetvideo = IPacket.make();

    IStreamCoder audioCoder = containerAudio.getStream(audiostreamt).getStreamCoder();

    if(audioCoder.open()<0 ) throw new RuntimeException("Cant open audio coder");
    mWriter.addAudioStream(0, 0, audioCoder.getChannels(), audioCoder.getSampleRate());

    mWriter.addVideoStream(1, 1, videocoder.getWidth(), videocoder.getHeight());

    IPacket packetaudio = IPacket.make();

    while(containerVideo.readNextPacket(packetvideo) >= 0 ||
            containerAudio.readNextPacket(packetaudio) >= 0){

        if(packetvideo.getStreamIndex() == videostreamt){

            //video packet
            IVideoPicture picture = IVideoPicture.make(videocoder.getPixelType(),
                    videocoder.getWidth(),
                    videocoder.getHeight());
            int offset = 0;
            while (offset < packetvideo.getSize()){
                int bytesDecoded = videocoder.decodeVideo(picture, 
                        packetvideo, 
                        offset);
                if(bytesDecoded < 0) throw new RuntimeException("bytesDecoded not working");
                offset += bytesDecoded;

                if(picture.isComplete()){
                    System.out.println(picture.getPixelType());
                    mWriter.encodeVideo(1, picture);

                }
            }
        } 

        if(packetaudio.getStreamIndex() == audiostreamt){   
        //audio packet

            IAudioSamples samples = IAudioSamples.make(512, 
                    audioCoder.getChannels(),
                    IAudioSamples.Format.FMT_S32);  
            int offset = 0;
            while(offset<packetaudio.getSize())
            {
                int bytesDecodedaudio = audioCoder.decodeAudio(samples, 
                        packetaudio,
                        offset);
                if (bytesDecodedaudio < 0)
                    throw new RuntimeException("could not detect audio");
                offset += bytesDecodedaudio;

                if (samples.isComplete()){
                     mWriter.encodeAudio(0, samples);

        }
            }

    }

  }
}

Hope it will help u.

Bonina

I was searching for a means to combine two audio files (as if played at same time) can I modify your video stream into audio stream2 and make it work? – Soundless

@Soundless they this code is used to combine two files one is audio and second is video. but you can customize it to achieve required functionality. Just understand the code and you can get what you want. – Bonina

Thanks for your reply, I will definitely tell you my progress (over the weekend) and share the code as I started using yours as the base :) – Soundless

Does this code work on Android ? to merge video and audio on the device ? – Masseur

@Masseur i think it should be, i don't have any experience in android but i used this code in applet. – Bonina

@Masseur yes it is required. – Bonina

as i know Xuggler isn't compatible with Android – Masseur

@Masseur then its a bad news. Xuggler use some native code, so i wrote 3 applets, like for mac, windows and linux separately. then please search if its possible. – Bonina

the requirements that xuggler needs to work are not existent on android. the only way it "might" work is a rooted device where you might be able to install an android port of the requirements. – Canicula

Answer 3

I know this thread is very old but the provided solution did not work for me and I figured I'd share my solution here...

I added the following dependencies to my pom.xml

<!-- https://mvnrepository.com/artifact/org.bytedeco/ffmpeg -->
   <dependency>
       <groupId>org.bytedeco</groupId>
       <artifactId>ffmpeg</artifactId>
       <version>RELEASE</version>
   </dependency>

   <!-- https://mvnrepository.com/artifact/org.bytedeco/javacv-platform -->
   <dependency>
       <groupId>org.bytedeco</groupId>
       <artifactId>javacv-platform</artifactId>
       <version>RELEASE</version>
   </dependency>

   <!-- https://mvnrepository.com/artifact/org.bytedeco/javacpp -->
   <dependency>
       <groupId>org.bytedeco</groupId>
       <artifactId>javacpp</artifactId>
       <version>RELEASE</version>
   </dependency>

Then I use ProcessBuilder to execute FFMPEG commands. To merge audio and video, I use the following command:

String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
    try {
        ProcessBuilder pb = new ProcessBuilder(
                ffmpeg,
                "-i",
                [PATH TO AUDIO FILE],
                "-i",
                [PATH TO VIDEO FILE],
                "-acodec",
                "copy",
                "-vcodec",
                "copy",
                [PATH TO OUTPUT FILE]
        );
        
        pb.redirectErrorStream(true);
        Process process = pb.start();
        process.waitFor();

    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    } 

Colin