Looking for a working example of addTimedTextSource for adding subtitle to a video from an .srt file in Android 4.1
Asked Answered
S

3

34

I have been trying to use an .srt file for a timed text source (Only available in android 4.1+ http://developer.android.com/about/versions/android-4.1.html#Multimedia) . The first problem has to do with getting a file descriptor for the .srt file (in the assets folder, how else would you bundle it in your app?). The file gets compressed automatically so you won't be able to even see the file without changing compile settings or doing a custom build. The easiest solution was to rename the .srt file to .jpg so that it does not get compressed and the openFD method still works. I am now adding the TimedTextSource with:

_myMP.addTimedTextSource(getAssets().openFd("captions.jpg").getFileDescriptor(),   MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP);

Now the file loads correctly and using myMP.getTrackInfo() to get a list of tracks, can see that after adding the timed text source, the 6th track has type "3" which is timed text track type. I have used selectTrack to choose this track as said in the google documentation but after doing so no captions ever appear and on my TimedTextListener:

 _myMP.setOnTimedTextListener(new OnTimedTextListener(){
        @Override
        public void onTimedText(MediaPlayer mp, TimedText text) {
                if (text!=null)
                   Log.d("TimedText", text.getText());  
            }       
        });

Fires only once (I have like 20 timed text events in the file) but the text parameter is always null. I have done searches and cannot find a single working code example of using timeText and it does not appear in any sample projects, there is literally no documentation other than the api docs from google but as far as I can tell, NO one has posted a working example of it yet. I am testing this on a google Nexus updated to Android 4.2

Scrofula answered 16/11, 2012 at 18:55 Comment(5)
Did you get it to work? I ran into same problem.Chapiter
no, I did get better text events by putting the srt file directly on the sd card (instead of changing the extension) and loading it from there, but it seems this functionality hasnt been implemented yet, you are still responsible for rendering the text, also, I am not sure how I would bundle it with the app to avoid the compression problem.Scrofula
Any updates? did you try a ttml file instead of a crt format file?Seigneury
what is the solution did you got any answers.Ferdinand
I have included my complete solution as an answer.Chemist
C
32

I was able to get this to work and since it is still an open question I will include the complete solution here.

Although the idea of changing the file extension to prevent the compression is nice, but I prefer to copy the srt file from the resources to the app local directory on the device, but anyways for the sake of completeness here is a list of extensions that won't be compressed.

".jpg", ".jpeg", ".png", ".gif", ".wav", ".mp2", ".mp3", ".ogg", ".aac", ".mpg", ".mpeg", ".mid", ".midi", ".smf", ".jet", ".rtttl", ".imy", ".xmf", ".mp4", ".m4a", ".m4v", ".3gp", ".3gpp", ".3g2", ".3gpp2",".amr", ".awb", ".wma", ".wmv"

The solution steps are simple:

  1. Create a MediaPlayer instance and prepare it by either calling MediaPlayer.create() or player.setDataSource() then player.prepare()

  2. If the subtitle files does not already exists on the android device, copy it from the resource folder to the device

  3. Call player.addTimedTextSource() with the first argument a String that contains the full path of the subtitle file on the device and MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP as the second argument

  4. Select the TimedText track by calling player.selectTrack() and pass the index of timedTextType by searching the TrackInfo[] returned from player.getTrackInfo() (I find it usually 2)

  5. Set up a listener with player.setOnTimedTextListener() and then start playing the media file player.start()

Here is the complete class:

To run this exact class you will need two files under your res/raw folder sub.srt and video.mp4 (or whatever extensions). Then define a TextView with the id txtDisplay. Finally your project/device/emulator must support API 16

public class MainActivity extends Activity implements OnTimedTextListener {
    private static final String TAG = "TimedTextTest";
    private TextView txtDisplay;
    private static Handler handler = new Handler();

    @Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    txtDisplay = (TextView) findViewById(R.id.txtDisplay);
    MediaPlayer player = MediaPlayer.create(this, R.raw.video);
    try {
        player.addTimedTextSource(getSubtitleFile(R.raw.sub),
                MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP);
        int textTrackIndex = findTrackIndexFor(
                TrackInfo.MEDIA_TRACK_TYPE_TIMEDTEXT, player.getTrackInfo());
        if (textTrackIndex >= 0) {
            player.selectTrack(textTrackIndex);
        } else {
            Log.w(TAG, "Cannot find text track!");
        }
        player.setOnTimedTextListener(this);
        player.start();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

private int findTrackIndexFor(int mediaTrackType, TrackInfo[] trackInfo) {
    int index = -1;
    for (int i = 0; i < trackInfo.length; i++) {
        if (trackInfo[i].getTrackType() == mediaTrackType) {
            return i;
        }
    }
    return index;
}

private String getSubtitleFile(int resId) {
    String fileName = getResources().getResourceEntryName(resId);
    File subtitleFile = getFileStreamPath(fileName);
    if (subtitleFile.exists()) {
        Log.d(TAG, "Subtitle already exists");
        return subtitleFile.getAbsolutePath();
    }
    Log.d(TAG, "Subtitle does not exists, copy it from res/raw");

    // Copy the file from the res/raw folder to your app folder on the
    // device
    InputStream inputStream = null;
    OutputStream outputStream = null;
    try {
        inputStream = getResources().openRawResource(resId);
        outputStream = new FileOutputStream(subtitleFile, false);
        copyFile(inputStream, outputStream);
        return subtitleFile.getAbsolutePath();
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        closeStreams(inputStream, outputStream);
    }
    return "";
}

private void copyFile(InputStream inputStream, OutputStream outputStream)
        throws IOException {
    final int BUFFER_SIZE = 1024;
    byte[] buffer = new byte[BUFFER_SIZE];
    int length = -1;
    while ((length = inputStream.read(buffer)) != -1) {
        outputStream.write(buffer, 0, length);
    }
}

// A handy method I use to close all the streams
private void closeStreams(Closeable... closeables) {
    if (closeables != null) {
        for (Closeable stream : closeables) {
            if (stream != null) {
                try {
                    stream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

@Override
public void onTimedText(final MediaPlayer mp, final TimedText text) {
    if (text != null) {
        handler.post(new Runnable() {
            @Override
            public void run() {
                int seconds = mp.getCurrentPosition() / 1000;

                txtDisplay.setText("[" + secondsToDuration(seconds) + "] "
                        + text.getText());
            }
        });
    }
}

// To display the seconds in the duration format 00:00:00
public String secondsToDuration(int seconds) {
    return String.format("%02d:%02d:%02d", seconds / 3600,
            (seconds % 3600) / 60, (seconds % 60), Locale.US);
}
}

And here is the subtitle file I am using as example:

1
00:00:00,220 --> 00:00:01,215
First Text Example

2
00:00:03,148 --> 00:00:05,053
Second Text Example

3
00:00:08,004 --> 00:00:09,884
Third Text Example

4
00:00:11,300 --> 00:00:12,900
Fourth Text Example

5
00:00:15,500 --> 00:00:16,700
Fifth Text Example

6
00:00:18,434 --> 00:00:20,434
Sixth Text Example

7
00:00:22,600 --> 00:00:23,700
Last Text Example

Here are few screenshots from the test app showing that the TextView is changing automatically (i.e. reading from the subtitle file) as the media file progresses

TimedText Example

Edit:

Here is the code for an example project

Chemist answered 18/2, 2013 at 4:5 Comment(13)
hi, i tried the above one. in onTimedText we are checking for text, here i am getting null. But else should be taken care. Please let me know.Ferdinand
Make sure that your video is more than 23 seconds. What phone/API you are testing with?Chemist
my video file is around 32 seconds . I am testing in emulator , API 17. But why you are specifying the video to be more than 23 seconds, any specific reason. can i get your email address so that i can send my code accross.Ferdinand
hi I able to use this successfully with a video file/mp4 as datasource to the mediaplayer,but when I use an mp3 file as data source the timedText callback is never run ...is the subtitles only supported for mp4 files?Please adviceWadewadell
Added link to the example project on GitHubChemist
@Itech hi! I tried the sample project you've uploaded. But then seems there's a bug? player.getTrackInfo() only returns an array with length 2 (i.e the audio and video) so it doesn't include the timed text and thus the textview never updated in my case. Do you happen to know why? Thanks in advance!Pinxit
My own solution based on the docs was nearly the same, and didn't work. This solution also didn't work, in the same sense. I get null texts and I don't get all of them that I should. API 16 S3.Serf
@Chemist We use multiple subtitle tracks for video which is selected via a Spinner. The problem is: i can't switch between subtitles since findTrackIndexFor always returns the same value: 2. I've planned a workaround manipulating index values since i'can delete track from track list, can't select track by filename or path. Is there a more solid way you can offer?Oconnor
The above solution worked on 4.2.2, but on 4.1.1 & 4.1.2, the 'text' parameter in onTimedText always returns null.Melchor
@iTech: How about internal(in-band) subtitles?Trumpet
I have tried it but the problem is after seeking the video, subtitles stop updating. That's onTimedText() is not called after seekingBughouse
Yeah, that seeking bug gave me such a headache. I had to abandon the timedtext altogether.Gyrfalcon
That doesn't support UTF-8 characters!Fredette
G
13

EDIT: I should point out that in the last few years, post-KitKat versions of Android have become most of the app-using android device market share. The implementation below was an attempt to bridge compatibility with older devices. At this point, I suggest using the TimedText framework (which worked fine in KitKat), or newer alternatives released by android, as a custom solution might have significant maintenance cost.


I went through 2 days looking at android source trying to iron out all the bugs that this TimedText Framework was causing.

My recommendation is to skip their implementation altogether. It is incomplete and inconsistent. In earlier versions, A lot of the Text synchronization is done in the native mediaplayer, so it's prone to state errors.

My alternative is to use a Textview Subclass:

package ca.yourpackage.yourapp;

import android.content.Context;
import android.media.MediaPlayer;
import android.util.AttributeSet;
import android.util.Log;
import android.widget.TextView;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.LineNumberReader;
import java.util.Locale;
import java.util.Map;
import java.util.TreeMap;

/**
 * Created by MHDante on 2015-07-26.
 */
public class SubtitleView extends TextView implements Runnable{
    private static final String TAG = "SubtitleView";
    private static final boolean DEBUG = false;
    private static final int UPDATE_INTERVAL = 300;
    private MediaPlayer player;
    private TreeMap<Long, Line> track;

    public SubtitleView(Context context) {
        super(context);
    }


    public SubtitleView(Context context, AttributeSet attrs) {
        super(context, attrs);
    }
    public SubtitleView(Context context, AttributeSet attrs, int defStyleAttr) {
        super(context, attrs, defStyleAttr);
    }


    @Override
    public void run() {
        if (player !=null && track!= null){
            int seconds = player.getCurrentPosition() / 1000;
            setText((DEBUG?"[" + secondsToDuration(seconds) + "] ":"")
                    + getTimedText(player.getCurrentPosition()));
        }
        postDelayed(this, UPDATE_INTERVAL);
    }

    private String getTimedText(long currentPosition) {
        String result = "";
        for(Map.Entry<Long, Line> entry: track.entrySet()){
            if (currentPosition < entry.getKey()) break;
            if (currentPosition < entry.getValue().to) result = entry.getValue().text;
        }
        return result;
    }

    // To display the seconds in the duration format 00:00:00
    public String secondsToDuration(int seconds) {
        return String.format("%02d:%02d:%02d", seconds / 3600,
                (seconds % 3600) / 60, (seconds % 60), Locale.US);
    }

    @Override
    protected void onAttachedToWindow() {
        super.onAttachedToWindow();
        postDelayed(this, 300);
    }

    @Override
    protected void onDetachedFromWindow() {
        super.onDetachedFromWindow();
        removeCallbacks(this);
    }
    public void setPlayer(MediaPlayer player) {
        this.player = player;
    }

    public void setSubSource(int ResID, String mime){
        if(mime.equals(MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP))
            track = getSubtitleFile(ResID);
        else
            throw new UnsupportedOperationException("Parser only built for SRT subs");
    }

    /////////////Utility Methods:
    //Based on https://github.com/sannies/mp4parser/
    //Apache 2.0 Licence at: https://github.com/sannies/mp4parser/blob/master/LICENSE

    public static TreeMap<Long, Line> parse(InputStream is) throws IOException {
        LineNumberReader r = new LineNumberReader(new InputStreamReader(is, "UTF-8"));
        TreeMap<Long, Line> track = new TreeMap<>();
        while ((r.readLine()) != null) /*Read cue number*/{
            String timeString = r.readLine();
            String lineString = "";
            String s;
            while (!((s = r.readLine()) == null || s.trim().equals(""))) {
                lineString += s + "\n";
            }
            long startTime = parse(timeString.split("-->")[0]);
            long endTime = parse(timeString.split("-->")[1]);
            track.put(startTime, new Line(startTime, endTime, lineString));
        }
        return track;
    }

    private static long parse(String in) {
        long hours = Long.parseLong(in.split(":")[0].trim());
        long minutes = Long.parseLong(in.split(":")[1].trim());
        long seconds = Long.parseLong(in.split(":")[2].split(",")[0].trim());
        long millies = Long.parseLong(in.split(":")[2].split(",")[1].trim());

        return hours * 60 * 60 * 1000 + minutes * 60 * 1000 + seconds * 1000 + millies;

    }

    private TreeMap<Long, Line> getSubtitleFile(int resId) {
        InputStream inputStream = null;
        try {
            inputStream = getResources().openRawResource(resId);
            return parse(inputStream);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (inputStream != null) {
                try {
                    inputStream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
        return null;
    }

    public static class Line {
        long from;
        long to;
        String text;


        public Line(long from, long to, String text) {
            this.from = from;
            this.to = to;
            this.text = text;
        }
    }
}

Usage:

//I used and reccomend asyncPrepare()
MediaPlayer mp = MediaPlayer.create(context, R.raw.video);
SubtitleView subView = (SubtitleView) getViewbyId(R.id.subs_box);
subView.setPlayer(mp);
subView.setSubSource(R.raw.subs_intro, MediaPlayer.MEDIA_MIMETYPE_TEXT_SUBRIP);

In your layout xml file, just create a textView as you would want the subtitles to show, then change the class to ca.yourpagckage.yourapp.SubtitleView

<ca.yourpagckage.yourapp.SubtitleView
    android:layout_width="300dp"
    android:layout_height="300dp"
    android:text="Subtitles go Here"
    android:id="@+id/subs_box"/>

Good Luck.

Gyrfalcon answered 27/7, 2015 at 2:52 Comment(2)
is it possible to use something like that setSubSource(uri), use URI instead of file from raw folder. thank youMoreta
Yeah, you'll just have to change getSubtitleFile(int resId) into getSubtitleFile(URI uri). In order to do that, you'll have to open the input stream from the uri.Gyrfalcon
B
-1

To get it to work with .mp3 files, call player.start(); immediately after declaring a new mediaplayer and before the addtimedtext code. Right after the line below

MediaPlayer player = MediaPlayer.create(this, R.raw.video);
Bacon answered 22/9, 2013 at 2:25 Comment(1)
So far, this is the only way I could make this work with an mp3. It does seem wrong to call start before finishing the setup.Adelric

© 2022 - 2024 — McMap. All rights reserved.