Generate all the files (.vtt + sprite) for the Tooltip Thumbnails options of Jwplayer

Question

What is the best way to generate the ".VTT" file and the jpg sprite attached with it for the Tooltip Thumbnails of Jwplayer (http://www.jwplayer.com/blog/building-tooltip-thumbnails-with-encodingcom/- ?

I know how to make an image sprite with php, but i dont know how to make the screenshots of each video with the time in second.. I think there must be a server tool to do all the tasks it but i cant find it.

Thanks

Entomophagous

I have not done this with ffmpeg before, but I have used a program called Video Thumbnails Maker by Scorp which generates the thumbs as well as the VTT files. It is here - suu-design.com/projects.html – Sindhi

Thanks Ethan but is this for Ubuntu server to ? I mean can i do it automatically or is a manual app ? – Entomophagous

This is a manual app. – Sindhi

Answer 1

I wrote a script to do this task. Given a video file (MP4 or M4v), generate thumbnail images, compress into a sprite, and generate a VTT file compatible with JWPlayer tooltip thumbnails. All of the image manipulation uses tools from ffmpeg, ImageMagick, and optionally sips and optipng. The WebVTT generation part, I had to write.

You will have to install ffmpeg & imagemagick, at a minimum to use this.

Github code is here: https://github.com/vlanard/videoscripts (under sprites/).

The basic gist is:

1. Create a bunch of thumbnails, e.g. every 45th second from a video

   ffmpeg -i ../archive/myvideofile.mp4 -f image2 -bt 20M -vf fps=1/45 thumbs/myvideofile/tv%03d.png 
   

2. Resize those thumbnails to be small, e.g. 100pixels wide

   sips --resampleWidth 100 thumbs/myvideofile/tv001.png thumbs/myvideofile/tv002.png thumbs/myvideofile/tv003.png
   

   OR if sips not available, use imageMagick utility:

   mogrify -geometry 100x thumbs/myvideofile/tv001.png thumbs/myvideofile/tv002.png thumbs/myvideofile/tv003.png
   

3. Get the height & width dimensions of one of the thumbnails to use as the basis of our grid coordinates, using ImageMagick utility

   identify -format "%g - %f" thumbs/myvideofile/tv001.png 
   

   which returns output like : 100x55+0+0 - tv001.png

   from which we parse 100 and 55 as our Width & Height, and the general geometry of each thumbnail (W, H, X, Y)

4. We then generate our single spritemap from the individual thumbnails. We determine the target grid size (e.g. 2x2, 8x8) to suit the number of thumbnails we generated for this video, as well as passing in the sprite geometry, using an ImageMagick utility

   montage thumbs/myvideofile/tv*.png -tile 2x2 -geometry 100x55+0+0 thumbs/myvideofile/myvideofile_sprite.png
   

5. Optionally we can run an extra compression step here to make the sprite smaller

   optipng thumbs/myvideofile/myvideofile_sprite.png
   

6. We then generate a VTT file based on the number of thumbnails we created, using the interval that we used to space out the thumbnails to label each time segment, and using the known coordinates of each consecutive image within our sprite that maps to the associated segment.

Guizot

Thanks, i finally did something similar but before taking the thumbnails i take the length of the video so i have the same amount of thumbs for all videos, i find this easier for generate the vtt. – Entomophagous

This is absolutely a fantastic job! I would like to make a Ruby wrapper for this as a gem if you give permission? – Corymb

@scaryguy, please do! I checked in latest to github with several key updates. 1)using jpg instead of png, much smaller sprites, 2)added adjustment to better sync the snapshot image to the start time in vtt file, to offset time drift occurring in ffmpeg; 3)omitting first image (0seconds) from sprite because JwPlayer doesn't show it anyway; 4) configurable snapshot timing now via command line parameter – Guizot

@Guizot you've made a great work! Congrats! I'm pretty busy nowadays but want to do it in some way. let's see what I can do. – Corymb

@Guizot it's online :) Thanks for inspiring! https://mcmap.net/q/693616/-generate-all-the-files-vtt-sprite-for-the-tooltip-thumbnails-options-of-jwplayer – Corymb

Thanks randalv. Very helpful. I just created a PHP adaptation here : github.com/F2000-FR/videoscripts-php – Overdone

After few research i finally make it, get duration, make thumbnail, join to sprites, generate vtt file :) – Abscess

That is incredible! – Inartistic

This is really cool. I am close to getting this to work but hit a snag. I get this error ... subprocess.CalledProcessError: Command '['sips', '--resampleWidth', '100']' returned non-zero exit status 4 – Holliman

@Holliman Try running which sips in your terminal to make sure you have it installed. If you see it I would also try manually running sips --resampleWidth 100 yourfilewhatever.png to see what that returns (obviously replacing file name with a legit one). – Guizot

Figured it out. I was a using a short clip of 30 seconds and the increment was too long; reduced it to every 5th second and it worked great. Thanks – Holliman

Generated sprites are very big even after compression. In which step should I reduce the size and how? – Interpleader

@ufokomer if optipng compression is still too large, you might want to review how many images are in your sprite (and how big they are) and play with that. you could also manually try another compression program like ImageOptim. If that doesn't help much, it's likely an issue with what's in the image itself. – Guizot

@Guizot I've followed your guide and created C# version of it, however its seems to really slow with large files. E.g. I have 2GB files, Full HD, H264 encoded and generating 1min thumbnail takes like 10-20 mins, I'm scared to try with thumbnails every 5 secs for example. :( – Dictate

@expressingx possibly some helpful advice here? #18535335 – Guizot

Answer 2

I've developed a Ruby gem to easily create .VTT file and sprite of thumbnails.

Thanks for inspiring @randalv!

You can take a look at it here: https://github.com/scaryguy/jwthumbs

---

Usage

Instantiate your video file:

movie = Jwthumbs::Movie.new("YOUR_VIDEO.mp4")

Jwthumbs::Movie.new accepts second parameter as a options hash. You can configure several stuff at the same time you instantiate your video like this:

movie = Jwthumbs::Movie.new("YOUR_VIDEO.mp4", seconds_between: 60, sprite_name: "my_sprite_name.jpg")

or after you instentiated your video, you can use Jwthumbs::Movie file to configure things:

movie = Jwthumbs::Movie.new("YOUR_VIDEO.mp4")
movie.seconds_between = 60
movie.sprite_name = "my_sprite_name.jpg"

and then to create your thumbnails and .VTT file just run this command.

movie.create_thumbs!

Corymb

Nice to have 2 language options now. Great contribution! – Guizot

@Guizot maybe you can give a back link to my repo, as I did for you :) You know, back links make it easier to find things through search engines. And don't forget to up vote this answer ;) – Corymb

Dear down voter, rather than down voting an answer which only contains link of an 'open source' project; maybe you should try to do something for the community. – Corymb

Answer 3

I know this is already a few years old but I had the same problem and found a command line tool which generates sprites pretty fast and since 1.0.6 supports WebVTT creation out of the box. The name is mt and you can check it here.

Quoting from their documentation you can use it like this:

just run mt and provide any video file as args: mt video.avi

Some of the settings can be changed through runtime flags provided directly to mt for more information just run mt --help

Dirham

Answer 4

Option 1 :

You can use the encoding.com's API and tell them to export vtt file too

I recommend to read "How can I create time synced thumbnails for use in JW player?" explanation from encoding.com's Knowledge base

Option 2 :

use movie thumbnailer (mtn), this is a command line tools running on UNIX, Windows systems. But you will have to write a custom script to generate the VTT file corresponding

• Super fast! Thanks to FFmpeg's libavcodec.
• Command line program: canbe used on remote connections to co-location servers, or used in scripts.
• Batch mode: recursively search directories for movie files. Run at lower priority (nice 10 on Linux, idle on Windows) by default. To run at normal priority use -n option.
• Thumbnails are group together in one jpeg file and can be saved individually too (-I option).
• Work fine with Unicode filenames in both Linux & Windows (might need to change the font with -f fontfile).

Geof