Video Basics

The Wolfram Language supports video objects as first-class citizens, enabling programmatic access, processing and analysis of a large number of multimedia containers and codecs. Together with complete stacks for image and audio processing, this opens up video processing from simple processing to highly sophisticated analysis.
Video Creation and Representation
A video object can be created by importing or referencing a video file on the disk.
a video object representing a video in file
a video object representing a video in url
a video generated from a Manipulate object
a video generated from a model
import a specific element from a video file
Video creation functions.
The video object is a reference to a local or remote file, with interactive video rendering capabilities. The actual data of the video file remains out-of-core, but parts of it such as image frames or audio chunks can be imported into the memory.
This creates an out-of-core video object taken from the Wolfram Language documentation "ExampleData" directory:
Use Import to create the Video object:
Create a Video object referencing a URL:
Generate a Video from a Manipulate:
Generate a 10-second Video of images showing a cellular automaton evolution:
You can use Import to obtain specific data or metadata elements from a video file, such as duration, list of frames or audio data.
Available import elements for a video file are:
Import two video frames as images:
Retrieve the video duration:
Import the audio content of the video as an Audio object or raw data:
Video Properties
The Video object has more than one purpose. It is a video file reference, a video data importer and a viewer. With respect to these functionalities, the Video object exhibits different sets of properties.
Video properties related to the video file and its content can be obtained by the following functions.
give the duration of video
return property prop of video
import a summary of the properties of file
Video data properties.
Get the total duration of a video object:
The Information function provides a video summary:
Use Information to extract a single property from the video object:
The "Summary" element of Import is also a convenient way to retrieve relevant properties of a video file.
Import summary of a video file:
The Video object can take options that control access and extraction of video data. These options will not change the content of the file, but instead determine the properties of extracted data for processing and display.
select audio tracks of interest
gives the raster size of extracted video frames
select subtitle tracks of interest
select video tracks of interest
Video access properties.
Most video formats supports storage of multiple audio, subtitle and even video tracks.
This is a video file with multiple audio and subtitle tracks:
Construct a video from this multi-track file that includes only one video and one audio track:
By default, the original dimensions of the frames, as stored in the file, are used when extracting or processing frames.
Import the size of frames in the file:
Reduce the video resolution by specifying a smaller RasterSize:
The extracted frame size from the Video object matches the specified RasterSize:
A video object can take more options that control the video and audio playback.
the overall appearance of the video player
audio output device to use for video playback
the image size of the displayed video
sound volume of the video player
Video playback properties.
Create a "Minimal" GUI with a width of 200 pixels:
Set the volume to 50%:
Content Extraction
To extract audio data or specific frames from a video file the following functions can be used.
returns an Audio object corresponding to the first audio track
returns a list of n video frames
extracts video frames at times t1,t2,
Video content extraction functions.
Generate a video summary by extracting frames.
Extract frames from a video at specific time instances:
Extract equally spaced frames from a video and create a thumbnail grid:
Video Editing
It is a common operation to trim a part of a video, delete portions of a video, add an audio track to a video and more. These video editing operations can be obtained by the following functions.
combine video and audio objects into one video
delete a segment of a video
sequentially join multiple video objects
split a video into multiple video segments
transcode a video to a specific format and codec
trim a segment of a video
Video editing functions.
Trim a time interval out of a video:
Check the resulting video duration:
Delete a time interval out of a video:
Split a video at specific times:
Processing and Analysis
Video processing or filtering is a video-in, video-out operation. Functions may operate on individual frames or a sequence of video frames to create new frames for the output video. Video analysis is the process of understanding the content of video by computing statistics or extracting features from single frame or a sequence of video frames.
applies f to the audio track of video, returning a new video
applies f to partitions of video frames, returning a new video
applies f to video partitions, returning a new video
applies f to video partitions, returning a list
applies f to video partitions, returning a time series
checks crit on video partitions, returning intervals of interst
Video processing and analysis functions.
VideoFrameMap applies image functions to individual frames or a sequence of video frames, resulting in a new video object.
Negate frames in a video scene:
Create motion blur on a video by blending five consecutive frames:
VideoMapTimeSeries can be used for analyzing and detecting video content by applying functions to video frames that return an arbitrary result, returned as a TimeSeries.
Determine the entropy of each frame:
Measure the difference between consecutive image frames:
The VideoStream expression can be used to programmatically manipulate the video playback.
creates a new VideoStream object from video
starts playing a VideoStream object vstream
pauses the playback of vstream
stops the playback vstream
returns all existing video streams
deletes the VideoStream object vstream
Video streaming functions.
Create a VideoStream object from a Video object:
Get a list of available properties of a video stream:
Get a specific stream property:
Programmatically start, pause and stop a video stream:
Display the current frame of a video stream:
Check all available video streams:
Remove all video streams: