Create thumbnail from video file via file input

Question

I am attempting to create a thumbnail preview from a video file (mp4,3gp) from a form input type='file'. Many have said that this can be done server side only. I find this hard to believe since I just recently came across this Fiddle using HTML5 Canvas and Javascript.

Thumbnail Fiddle

The only problem is this requires the video to be present and the user to click play before they click a button to capture the thumbnail. I am wondering if there is a way to get the same results without the player being present and user clicking the button. For example: User click on file upload and selects video file and then thumbnail is generated. Any help/thoughts are welcome!

Tirza

The player obviously has to be present, as that is what is producing the image for capture. – Koniology

you can hide the player using css, and call videoTag.play() to start it playing. i recommend jumping 18 seconds in, waiting until it shows, and then sending it to a canvas drawImage routine. i turned a folder of movies into a thumbnail gallery in chrome this way, so i can assure you it works. – Continental

Nicely done "dandavis". Now can does the file have to be uploaded to a server somewhere or can this be done client side. Say the user selects a video on his/her desktop and then run this script from what is in input? – Tirza

If the video uses a codec that the current browser supports, you can handle the dropped/selected video file, get an object URL for the video file, and set that value to the src attribute of a video element. – Within

I've recently been working on a plug-in that addresses the items in your question (thumbnail generation from <video>s, conversion of Blob/File to a <video>, etc). Have a look, and perhaps it will prove to be useful for you. github.com/rnicholus/frame-grab.js – Within

@RayNicholus this looks like a very promising plug-in. I read through the documentation on github. Question is, can this work client side without the video being uploaded to a server? Also, what all file types does it work with? Reason is, I am allowing users to upload .mp4 and .3gp only since my clients are for mobile. – Tirza

frame-grab's features are 100% client-side. For example if you include a file input element, or a drop zone on your page. When your user selects/drops a video file, you can pass the Blob from the file input/drop event on to frab-grab's blob_to_video method, get a <video> and feed that into a frame-grab instance, where you can have images generated via the various workflows/methods exposed in frame-grab's API. This all happens in the browser, nothing is sent to the server. I encourage you to ask questions or leave feature requests in the github repo, and we can discuss more there. – Within

@RayNicholus I am having trouble finding out how to ask questions on github. But I downloaded your plug-in and opened your test. Nothing is the test file was linked correctly and once I had everything linked, nothing happened. I was able to play a video that was already on embeded in a video tag but there was no input or thumbnails of the video file. – Tirza

If you want to run the tests, you'll need to clone the repo using git, run npm install in the cloned directory, and then run grunt. Pre-req: grunt must be installed. If you just want to use the plug-in, just drop it into your project, along with RSVP (a promise impl frame-grab depends on due to all of the async stuff it does). I only develop this on my off-time, late at night, so docs could be better. Please suggest how it can be improved. To ask a question, do so in the frame-grab issue tracker. Here's a link to create a new "issue":github.com/rnicholus/frame-grab.js/issues/new – Within

Note that I have already used frame-grab in a real prototype project a couple weeks ago. I've made some adjustments to the code since then, but the code is heavily unit tested, so I'm fairly confident that nothing major is broken. – Within

@RayNicholus I wrote a comment on github. Is there any way to contact you directly, possibly email, skype , link. I have a pretty hefty project I am working on and the deadline is tight. Some insight and direction on how to get started would be wonderful. Thanks – Tirza

Sorry, I'm working on that plug-in during my free time as I have a day job. I saw your issue posted in the repo and I'll try to respond tonight (CST), schedule permitting. I won't be able to guarantee any specific turnaround times for responses on this project. It's mostly a leisurely side project that I'm not being paid for and develop as my schedule allows. – Within

@RayNicholus Thank you for you quick responses. I totally understand about having a full-time job and working on projects for free. I too have came up with some plug-ins that works with using Canvas and PHP to allow users(kids) to send paintings they draw to cell phones. Still in the early stages. But if you can keep me posted that would be wonderful. Have a great day. – Tirza

Answer 1

Canvas.drawImage must be based on html content.

source

here is a simplier jsfiddle

//and code
function capture(){
    var canvas = document.getElementById('canvas');
    var video = document.getElementById('video');
    canvas.getContext('2d').drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
}

The advantage of this solution is that you can select the thumbnail you want based on the time of the video.

Georgettageorgette

Is their any way to do this without loading video on document – Gastroenteritis

I only can get video size 0 0 and white image. – Zendavesta

The thumbnail is trimmed from bottom, I can't get the full image – Gegenschein

@Gegenschein The video size is loaded from the metadata and so you may need to add an event handler to the 'loadedmetadata' event and fetch the video width and height from there. Once you've got that you should then be able to resize the canvas and draw the image. – Nugent

Answer 2

Recently needed this so I wrote a function, to take in a video file and a desired timestamp, and return an image blob at that time of the video.

Sample Usage:

try {
    // get the frame at 1.5 seconds of the video file
    const cover = await getVideoCover(file, 1.5);
    // print out the result image blob
    console.log(cover);
} catch (ex) {
    console.log("ERROR: ", ex);
}

Function:

function getVideoCover(file, seekTo = 0.0) {
    console.log("getting video cover for file: ", file);
    return new Promise((resolve, reject) => {
        // load the file to a video player
        const videoPlayer = document.createElement('video');
        videoPlayer.setAttribute('src', URL.createObjectURL(file));
        videoPlayer.load();
        videoPlayer.addEventListener('error', (ex) => {
            reject("error when loading video file", ex);
        });
        // load metadata of the video to get video duration and dimensions
        videoPlayer.addEventListener('loadedmetadata', () => {
            // seek to user defined timestamp (in seconds) if possible
            if (videoPlayer.duration < seekTo) {
                reject("video is too short.");
                return;
            }
            // delay seeking or else 'seeked' event won't fire on Safari
            setTimeout(() => {
              videoPlayer.currentTime = seekTo;
            }, 200);
            // extract video thumbnail once seeking is complete
            videoPlayer.addEventListener('seeked', () => {
                console.log('video is now paused at %ss.', seekTo);
                // define a canvas to have the same dimension as the video
                const canvas = document.createElement("canvas");
                canvas.width = videoPlayer.videoWidth;
                canvas.height = videoPlayer.videoHeight;
                // draw the video frame to canvas
                const ctx = canvas.getContext("2d");
                ctx.drawImage(videoPlayer, 0, 0, canvas.width, canvas.height);
                // return the canvas image as a blob
                ctx.canvas.toBlob(
                    blob => {
                        resolve(blob);
                    },
                    "image/jpeg",
                    0.75 /* quality */
                );
            });
        });
    });
}

Jerrine

Re the use of setTimeout, could the problem be that you're adding the event listener after you do the seek? Might make sense to add the listener first, then cause the seek to happen. Same with the loadedmetadata – Doormat

@Doormat I remember I tried even with setTimeout 0, which pushes the callback onto the event queue, still does not make it reliable enough on safari. It needs an actual delay. Maybe just Safari being too slow in loading the video? Have you been able to verify your suggestions on Safari? – Jerrine

on my iMac with safari stable latest, it didn't need any setTimeout -- or at least, the thumbnail generated fine. That's with having the listeners added before the seek is requested. – Doormat

@Doormat I'm not on Safari 14 yet, but without set timeout, about 8 out of 10 videos would be fine, some occasional videos would fail for no unknown reason – Jerrine

Thank you for the info. I’ll leave it in then to be safe. – Doormat

these solutions only work with HTML5-supported video codes. and then tried to use web assembly version of ffmpeg and then quickly realized that it needs cross-origin-opener policy to same origin inorder to enable sharedarrayBuffer. our site has lot of 3rd party integrations which don't work with that policy. any other ideas? – Azole

This solution is not working with stream URL. Are there any ways to generate thumbnails from "m3u8" URL in client side javascript? – Tadeo

For me this solution is working well on desktop browsers. On mobile browsers (safari, chrome) when i try to generate a thumbnail of a .mov file, the seeked event doesnt fire. – Pung

Answer 3

Recently needed this and did quite some testing and boiling it down to the bare minimum, see https://codepen.io/aertmann/pen/mAVaPx

There are some limitations where it works, but fairly good browser support currently: Chrome, Firefox, Safari, Opera, IE10, IE11, Android (Chrome), iOS Safari (10+).

 video.preload = 'metadata';
 video.src = url;
 // Load video in Safari / IE11
 video.muted = true;
 video.playsInline = true;
 video.play();

Passmore

Your codepen seems to be good, but when I try to upload > 100mb iPhone portrait shoot mode videos sometimes the screenshots are blank or sometimes they dont get uploaded only. FYI, I want canvas size to be 170x100. Had you faced the similar issues for iPhone portrait mode videos? – Jillayne

@Jillayne No can't say that I have, sorry. – Passmore

Answer 4

The easiest way to display a thumbnail is using the <video> tag itself.

<video src="http://www.w3schools.com/html/mov_bbb.mp4"></video>

Use #t in the URL, if you want the thumbnail of x seconds.

E.g.:

<video src="http://www.w3schools.com/html/mov_bbb.mp4#t=5"></video>

Make sure that it does not include any attributes like autoplay or controls and it should not have a source tag as a child element.

With a little bit of JavaScript, you may also be able to play the video, when the thumbnail has been clicked.

document.querySelector('video').addEventListener('click', (e) => {
  if (!e.target.controls) { // Proceed, if there are no controls
    e.target.src = e.target.src.replace(/#t=\d+/g, ''); // Remove the time, which is set in the URL
    e.target.play(); // Play the video
    e.target.controls = true; // Enable controls
  }
});

<video src="http://www.w3schools.com/html/mov_bbb.mp4#t=5"></video>

Maurilia

Also, the question is for video coming from file input, not url. – Glaydsglaze

this method is eating up a lot of bandwidth – Rakeoff

@Glaydsglaze We can create a objectUrl from the file and pass to the src attribute. – Dime

Answer 5

You can use this function that I've written. You just need to pass the video file to it as an argument. It will return the dataURL of the thumbnail(i.e image preview) of that video. You can modify the return type according to your need.

const generateVideoThumbnail = (file: File) => {
  return new Promise((resolve) => {
    const canvas = document.createElement("canvas");
    const video = document.createElement("video");

    // this is important
    video.autoplay = true;
    video.muted = true;
    video.src = URL.createObjectURL(file);

    video.onloadeddata = () => {
      let ctx = canvas.getContext("2d");

      canvas.width = video.videoWidth;
      canvas.height = video.videoHeight;

      ctx.drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
      video.pause();
      return resolve(canvas.toDataURL("image/png"));
    };
  });
};

Please keep in mind that this is a async function. So make sure to use it accordingly.

For instance:

const handleFileUpload = async (e) => {
  const thumbnail =  await generateVideoThumbnail(e.target.files[0]);
  console.log(thumbnail)
}

Whisky

Please make sure to call URL.revokeObjectURL(file); after you're done (eg, in onloadedmetadata, otherwise it would impact your page performance due to memory overload (specially with videos), because URL.createObjectURL(file); creates new object url every time and memory needs to be released for page to work properly. – Saulsauls

Answer 6

With jQuery Lib you can use my code here. $video is a Video element.This function will return a string

function createPoster($video) {
    //here you can set anytime you want
    $video.currentTime = 5;
    var canvas = document.createElement("canvas");
    canvas.width = 350;
    canvas.height = 200;
    canvas.getContext("2d").drawImage($video, 0, 0, canvas.width, canvas.height);
    return canvas.toDataURL("image/jpeg");;
}

Example usage:

$video.setAttribute("poster", createPoster($video));

Muskellunge

Answer 7

I was coming across this issue in a nextJS site where I was using the mozilla MediaRecorder to record a video client side in browser. On mobile the first frame was always empty and the hacks to use a few milliseconds in weren't helpful since my video was coming from the MediaRecorder blob (rather than being streamed from a file).

If it helps anyone looking to do this in react and utilising "react-media-recorder" below worked perfectly for me taking a snapshot of the video after it starts recording and then using this as the thumbnail as the "poster" attribute.

import React, { useEffect, useRef, useState } from "react";
import { useReactMediaRecorder } from "react-media-recorder";

export default function VideoRecorder() {
  const { status, startRecording, stopRecording, mediaBlobUrl, error } =
    useReactMediaRecorder({ video: true });

  // State to hold the thumbnail URL
  const [thumbnailUrl, setThumbnailUrl] = useState(null);

  // Reference to the video element
  const videoRef = useRef(null);

  // Reference to the canvas element
  const canvasRef = useRef(null);

  useEffect(() => {
    if (videoRef.current) {
      videoRef.current.addEventListener("playing", captureSnapshot);
    }

    return () => {
      if (videoRef.current) {
        videoRef.current.removeEventListener("playing", captureSnapshot);
      }
    };
  }, []);

  const captureSnapshot = () => {
    /* Get canvas context */
    const ctx = canvasRef.current.getContext("2d");

    /* Set canvas dimensions to match video */
    canvasRef.current.width = videoRef.current.videoWidth;
    canvasRef.current.height = videoRef.current.videoHeight;

    /* Draw current video frame onto the canvas */
    ctx.drawImage(videoRef.current, 0, 0);

    /* Export canvas content to data URL and set as thumbnail */
    setThumbnailUrl(canvasRef.current.toDataURL("image/jpeg", 0.95));
  };

  if (error) {
    return <p>Error: {error}</p>;
  }

  return (
    <div>
      <p>{status}</p>
      <button onClick={startRecording} disabled={status === "recording"}>
        Start Recording
      </button>
      <button onClick={stopRecording} disabled={status !== "recording"}>
        Stop Recording
      </button>
      {mediaBlobUrl && (
        <video
          ref={videoRef}
          poster={thumbnailUrl}
          src={mediaBlobUrl}
          controls
          autoPlay
          loop
          style={{ maxWidth: "100%", maxHeight: "100%", objectFit: "contain" }}
        />
      )}
      <canvas ref={canvasRef} style={{ display: "none" }}></canvas>
    </div>
  );
}

Realism

Answer 8

I recently stumbled on the same issue and here is how I got around it.

firstly it will be easier if you have the video as an HTML element, so you either have it in the HTML like this

<video src="http://www.w3schools.com/html/mov_bbb.mp4"></video>

or you take from the input and create an HTML element with it.

The trick is to set the start time in the video tag to the part you want to seek and have as your thumbnail, you can do this by adding #t=1.5 to the end of the video source.

<video src="http://www.w3schools.com/html/mov_bbb.mp4#t=1.5"></video>

where 1.5 is the time you want to seek and get a thumbnail of.

This, however, makes the video start playing from that section of the video so to avoid that we add an event listener on the video's play button(s) and have the video start from the beginning by setting video.currentTime = 0

const video = document.querySelector('video');

video.addEventListener('click', (e)=> {
video.currentTime = 0 ; 
video.play();
})

Typeset