Using mp4parser , how can I handle videos that are taken from Uri and ContentResolver?

Question

Background

We want to let the user choose a video from any app, and then trim a video to be of max of 5 seconds.

The problem

For getting a Uri to be selected, we got it working fine (solution available here) .

As for the trimming itself, we couldn't find any good library that has permissive license, except for one called "k4l-video-trimmer" . The library "FFmpeg", for example, is considered not permission as it uses GPLv3, which requires the app that uses it to also be open sourced. Besides, as I've read, it takes quite a lot (about 9MB).

Sadly, this library (k4l-video-trimmer) is very old and wasn't updated in years, so I had to fork it (here) in order to handle it nicely. It uses a open sourced library called "mp4parser" to do the trimming.

Problem is, this library seems to be able to handle files only, and not a Uri or InputStream, so even the sample can crash when selecting items that aren't reachable like a normal file, or even have paths that it can't handle. I know that in many cases it is possible to get a path of a file, but in many other cases, it's not, and I also know it's possible to just copy the file (here), but this isn't a good solution, as the file could be large and take a lot of space even though it's already accessible.

What I've tried

There are 2 places that the library uses a file:

1. In "K4LVideoTrimmer" file, in the "setVideoURI" function, which just gets the file size to be shown. Here the solution is quite easy, based on Google's documentation:

   public void setVideoURI(final Uri videoURI) {
       mSrc = videoURI;
       if (mOriginSizeFile == 0) {
           final Cursor cursor = getContext().getContentResolver().query(videoURI, null, null, null, null);
           if (cursor != null) {
               int sizeIndex = cursor.getColumnIndex(OpenableColumns.SIZE);
               cursor.moveToFirst();
               mOriginSizeFile = cursor.getLong(sizeIndex);
               cursor.close();
               mTextSize.setText(Formatter.formatShortFileSize(getContext(), mOriginSizeFile));
           }
       }
    ...
   

2. In "TrimVideoUtils" file, in "startTrim" which calls "genVideoUsingMp4Parser" function. There, it calls the "mp4parser" library using :

   Movie movie = MovieCreator.build(new FileDataSourceViaHeapImpl(src.getAbsolutePath()));
   

   It says that they use FileDataSourceViaHeapImpl (from "mp4parser" library) to avoid OOM on Android, so I decided to stay with it.

   Thing is, there are 4 CTORS for it, all expect some variation of a file: File, filePath, FileChannel , FileChannel+fileName .

The questions

1. Is there a way to overcome this?

Maybe implement FileChannel and simulate a real file, by using ContentResolver and Uri ? I guess it might be possible, even if it means re-opening the InputStream when needed...

In order to see what I got working, you can clone the project here. Just know that it doesn't do any trimming, as the code for it in "K4LVideoTrimmer" file is commented:

//TODO handle trimming using Uri
//TrimVideoUtils.startTrim(file, getDestinationPath(), mStartPosition, mEndPosition, mOnTrimVideoListener);

2. Is there perhaps a better alternative to this trimming library, which is also permissive (meaning of Apache2/MIT licences , for example) ? One that don't have this issue? Or maybe even something of Android framework itself? I think MediaMuxer class could help (as written here), but I think it might need API 26, while we need to handle API 21 and above...

---

EDIT:

I thought I've found a solution by using a different solution for trimming itself, and wrote about it here, but sadly it can't handle some input videos, while mp4parser library can handle them.

Please let me know if it's possible to modify mp4parser to handle such input videos even if it's from Uri and not a File (without a workaround of just copying to a video file).

Chadwick

The MediaMuxer class you mentioned is supported by API 18 Onwards [developer.android.com/reference/android/media/MediaMuxer]. And the answer https://mcmap.net/q/746911/-how-to-trim-video-with-mediacodec refers to Lollipop, which is API21. I believe you can easily use this! Further, if you have the URI, what's wrong in creating a File object around it and passing it over to the library for processing? Have you tried this? What was the response/error/problem, if you did? – Toad

@RahulShukla The code I've found worked (and it's from API 18, BTW, not 21 like on the link) , but only for some input files. For some others, it threw an exception, and I wrote about it here : https://mcmap.net/q/710609/-how-can-i-trim-a-video-from-uri-including-files-that-mp4parser-library-can-handle-but-using-android-39-s-framework-instead/878126 . You can check the POC I've made for this here: github.com/AndroidDeveloperLB/VideoTrimmer . As for using the File from the Uri, that's not always possible. Example is from Google Drive app. My current solution uses both methods, but can still fail :( – Chadwick

Answer 1

First of all a caveat: I am not familiar with the mp4parser library but your question looked interesting so I took a look.

I think its worth you looking at one of the classes the code comments say is "mainly for testing". InMemRandomAccessSourceImpl. To create a Movie from any URI, the code would be as follows:

try {
    InputStream  inputStream = getContentResolver().openInputStream(uri);
    Log.e("InputStream Size","Size " + inputStream);
    int  bytesAvailable = inputStream.available();
    int bufferSize = Math.min(bytesAvailable, MAX_BUFFER_SIZE);
    final byte[] buffer = new byte[bufferSize];

    int read = 0;
    int total = 0;
    while ((read = inputStream.read(buffer)) !=-1 ) {
        total += read;
    }
    if( total < bytesAvailable ){
        Log.e(TAG, "Read from input stream failed")
        return;
    }
    //or try inputStream.readAllBytes() if using Java 9
    inputStream.close();

    ByteBuffer bb = ByteBuffer.wrap(buffer);
    Movie m2 = MovieCreator.build(new ByteBufferByteChannel(bb),
        new InMemRandomAccessSourceImpl(bb), "inmem");

} catch (FileNotFoundException e) {
    e.printStackTrace();
} catch (IOException e) {
    e.printStackTrace();
}

But I would say, there looks to be somewhat of a conflict between what you want to achieve and the approach the parser takes. It is depending on local files to avoid large memory overheads, and random access to bytes can only be done if the entire set of data is available, which differs from a streaming approach.

It will require buffering at least the amount of data required for your clip in one go before the parser is given the buffer. That might be workable for you if you are looking to grab short sections and the buffering is not too cumbersome. You may be subject to IO exceptions and the like if the read from the InputStream has issues, especially if it is remote content, whereas you really aren't expecting that with a file on a modern system.

There is also MemoryFile to consider which provides an ashmem backed file-like object. I think somehow that could be worked in.

Litta

Having a buffer makes sense, and trimming is supposed to use some memory (not tons of memory, but still...). This seems logical, but how can I do it? Do you have a working solution for this, given a Uri and/or way to create InputStream? Assume nothing about the given input. Meaning it could be with very high quality and/or long duration, and we could be trimming to a tiny duration or a huge one (look at the parameters of the real function of trimming)... – Chadwick

In your question you mention 2 distinct places: 1. In "K4LVideoTrimmer" file, and 2. "TrimVideoUtils". You posted a solution for 1. to replace with Uri. The code example above should provide you a solution to 2 starting from Uri & content resolver. But I agree, high quality and long duration would need thought into how you buffered. MemoryFile could provide a flexible intermediary as you read from the InputStream. And in low memory situations, you can enable the setting for system to purge the file. This task could be further optimised using ring buffers and/or reading sections at a time – Litta

The "k4l-video-trimmer" library uses "mprparser", which requires File usage. The second solution, which can handle a Uri is mentioned here: https://mcmap.net/q/710609/-how-can-i-trim-a-video-from-uri-including-files-that-mp4parser-library-can-handle-but-using-android-39-s-framework-instead/878126 , but it doesn't seem to support some videos (in my case those that have "audio/ac3" in them). For now you can check my Github repository which uses both ways (second is a fallback of the first) : github.com/AndroidDeveloperLB/VideoTrimmer . About MemoryFile, have you tried it? Does it use a lot of memory ? Please share the code for it. You can use my repository for this. – Chadwick

@androiddeveloper What did you end up doing? – Dodiedodo

@androiddeveloper It looks like you haven't resolved this issue, looking at this -github.com/AndroidDeveloperLB/VideoTrimmer/blob/… – Dodiedodo

@Dodiedodo I used my solution and hoped for no issues. Later we ditched it all and didn't have anything to do with it. But if you know how to do those things, please let me know. What I might help you with, is that usually you can find the path, and even with SAF you can access files (if they are files) . Link: https://mcmap.net/q/344269/-how-to-get-information-of-an-apk-file-in-the-file-system-not-just-installed-ones-without-using-file-or-file-path – Chadwick

@androiddeveloper I may have found a solution, please refer to the comment on your question here - github.com/sannies/mp4parser/issues/357 – Dodiedodo

Answer 2

Next a snipped shows how to open a MediaStore Uri with IsoFile from Mp4Parser. So, you can see how to get a FileChannel from a Uri.

public void test(@NonNull final Context context, @NonNull final Uri uri) throws IOException
{
    ParcelFileDescriptor fileDescriptor = null;

    try
    {
        final ContentResolver resolver = context.getContentResolver();
        fileDescriptor = resolver.openFileDescriptor(uri, "rw");

        if (fileDescriptor == null)
        {
            throw new IOException("Failed to open Uri.");
        }

        final FileDescriptor  fd          = fileDescriptor.getFileDescriptor();
        final FileInputStream inputStream = new FileInputStream(fd);
        final FileChannel     fileChannel = inputStream.getChannel();

        final DataSource channel = new FileDataSourceImpl(fileChannel);
        final IsoFile    isoFile = new IsoFile(channel);

        ... do what you need ....
    }
    finally
    {
        if (fileDescriptor != null)
        {
            fileDescriptor.close();
        }
    }
}

Rideout