Video Basics

The Wolfram Language supports video objects as first-class citizens, enabling programmatic access, processing and analysis of a large number of multimedia containers and codecs. Together with complete stacks for image and audio processing, this opens up video processing from simple processing to highly sophisticated analysis.
Video Creation and Representation
A video object can be created by importing or referencing a video file on the disk.
Video[file]
a video object representing a video in file
Video[url]
a video object representing a video in url
Video[Manipulate[]]
a video generated from a Manipulate object
VideoGenerator[model]
a video generated from a model
Import[file,elem]
import a specific element from a video file
Video creation functions.
The video object is a reference to a local or remote file, with interactive video rendering capabilities. The actual data of the video file remains out-of-core, but parts of it such as image frames or audio chunks can be imported into the memory.
This creates an out-of-core video object taken from the Wolfram Language documentation "ExampleData" directory:
Use Import to create the Video object:
Create a Video object referencing a URL:
Generate a Video from a Manipulate:
Generate a 10-second Video of images showing a cellular automaton evolution:
You can use Import to obtain specific data or metadata elements from a video file, such as duration, list of frames or audio data.
Available import elements for a video file are:
Import two video frames as images:
Retrieve the video duration:
Import the audio content of the video as an Audio object or raw data:
Video Properties
The Video object has more than one purpose. It is a video file reference, a video data importer and a viewer. With respect to these functionalities, the Video object exhibits different sets of properties.
Video properties related to the video file and its content can be obtained by the following functions.
Duration[video]
give the duration of video
Information[video,prop]
return property prop of video
Import[file,"Summary"]
import a summary of the properties of file
Video data properties.
Get the total duration of a video object:
The Information function provides a video summary:
Use Information to extract a single property from the video object:
The "Summary" element of Import is also a convenient way to retrieve relevant properties of a video file.
Import summary of a video file:
The Video object can take options that control access and extraction of video data. These options will not change the content of the file, but instead determine the properties of extracted data for processing and display.
AudioTrackSelection
select audio tracks of interest
RasterSize
gives the raster size of extracted video frames
SubtitleTrackSelection
select subtitle tracks of interest
VideoTrackSelection
select video tracks of interest
Video access properties.
Most video formats supports storage of multiple audio, subtitle and even video tracks.
This is a video file with multiple audio and subtitle tracks:
Construct a video from this multi-track file that includes only one video and one audio track:
By default, the original dimensions of the frames, as stored in the file, are used when extracting or processing frames.
Import the size of frames in the file:
Reduce the video resolution by specifying a smaller RasterSize:
The extracted frame size from the Video object matches the specified RasterSize:
A video object can take more options that control the video and audio playback.
Appearance
the overall appearance of the video player
AudioOutputDevice
audio output device to use for video playback
ImageSize
the image size of the displayed video
SoundVolume
sound volume of the video player
Video playback properties.
Create a "Minimal" GUI with a width of 200 pixels:
Set the volume to 50%:
Content Extraction
To extract audio data or specific frames from a video file the following functions can be used.
Audio[video]
returns an Audio object corresponding to the first audio track
VideoFrameList[video,n]
returns a list of n video frames
VideoExtractFrames[video,{t1,t2,}]
extracts video frames at times t1,t2,
Video content extraction functions.
Generate a video summary by extracting frames.
Extract frames from a video at specific time instances:
Extract equally spaced frames from a video and create a thumbnail grid:
Video Editing
It is a common operation to trim a part of a video, delete portions of a video, add an audio track to a video and more. These video editing operations can be obtained by the following functions.
VideoCombine[video,audio,]
combine video and audio objects into one video
VideoDelete[video,{t1,t2}]
delete a segment of a video
VideoJoin[video1,video2,]
sequentially join multiple video objects
VideoSplit[video,{t1,t2}]
split a video into multiple video segments
VideoTranscode[video,format]
transcode a video to a specific format and codec
VideoTrim[video,{t1,t2}]
trim a segment of a video
Video editing functions.
Trim a time interval out of a video:
Check the resulting video duration:
Delete a time interval out of a video:
Split a video at specific times:
Processing and Analysis
Video processing or filtering is a video-in, video-out operation. Functions may operate on individual frames or a sequence of video frames to create new frames for the output video. Video analysis is the process of understanding the content of video by computing statistics or extracting features from single frame or a sequence of video frames.
AudioTrackApply[f,video]
applies f to the audio track of video, returning a new video
VideoFrameMap[f,video,]
applies f to partitions of video frames, returning a new video
VideoMap[f,video,]
applies f to video partitions, returning a new video
VideoMapList[f,video,]
applies f to video partitions, returning a list
VideoMapTimeSeries[f,video,]
applies f to video partitions, returning a time series
VideoIntervals[video,crit,]
checks crit on video partitions, returning intervals of interst
Video processing and analysis functions.
VideoFrameMap applies image functions to individual frames or a sequence of video frames, resulting in a new video object.
Negate frames in a video scene:
Create motion blur on a video by blending five consecutive frames:
VideoTimeSeries can be used for analyzing and detecting video content by applying functions to video frames that return an arbitrary result, returned as a TimeSeries.
Determine the entropy of each frame:
Measure the difference between consecutive image frames:
Streaming
The VideoStream expression can be used to programmatically manipulate the video playback.
VideoStream[video]
creates a new VideoStream object from video
VideoPlay[vstream]
starts playing a VideoStream object vstream
VideoPause[vstream]
pauses the playback of vstream
VideoStop[vstream]
stops the playback vstream
VideoStreams[]
returns all existing video streams
RemoveVideoStream[vstream]
deletes the VideoStream object vstream
Video streaming functions.
Create a VideoStream object from a Video object:
Get a list of available properties of a video stream:
Get a specific stream property:
Programmatically start, pause and stop a video stream:
Display the current frame of a video stream:
Check all available video streams:
Remove all video streams: