💻  Ian Wootten

~ 2 min read

Extracting Shot Thumbnails from a video using FFMPEG

Recently, Andy Davies asked the following question on twitter:

I found this pretty interesting, especially given I’d used various open source tools for assembling storyboards in the past. I knew it was possible to extract images at desired intervals and left it as that.

After a bit of digging last night, I discovered it’s actually possible to extract images based on changes in the video. We can use this to extract a rough approximation to individual shots.

I’ve run the following command on the Big Buck Bunny in order to extract 138 individual changes from the 10 min short. I’m using ffmpeg version 2.3 (I had tried using ffmbc 0.7, but it is unable to parse some of the options).

ffmpeg -ss 3 -i big_buck_bunny_1080p_surround.avi -vf "select=gt(scene\,0.2)" -vsync vfr -s vga -f image2 out%02d.jpg

The real gold here is the ‘-vf “select=gt(scene,0.2)”’, which grabs scene changes based on a difference in the frame of 20% or more. There are certain discrepancies in a couple of shots, but in the main this method works well and quickly. I specify the output format (-f) and size (-s) so that I can get small thumbs rather than the 1920x1080 the short is in.

You can see how well this performs through the selections made for the first 10 changes shown below.

In order to assemble the thumbs into an individual image I’ve used the “montage” command from imagemajick.

montage *.jpg -geometry 320x180+2+2 -tile 3x4 sample.jpg

This assembles the images in the current directory in a 3x4 grid and outputs them to sample.jpg. If there are more images that will fit in a 3x4 grid, multiple output images will be created to represent the strip.

Animation is a really good exemplar for this technique, probably due to the obvious changes between each shot. However I can imagine it would fare less well with say a live action feature, where scene changes may be more gradual and/or lighting is more subtle.