Video Basics
Video Creation and Representation | Video Editing |
Video Properties | Processing and Analysis |
Content Extraction | Streaming |
The Wolfram Language supports video objects as first-class citizens, enabling programmatic access, processing and analysis of a large number of multimedia containers and codecs. Together with complete stacks for image and audio processing, this opens up video processing from simple processing to highly sophisticated analysis.
Video[file] | a video object representing a video in file |
Video[url] | a video object representing a video in url |
Video[Manipulate[…]] | a video generated from a Manipulate object |
VideoGenerator[model] | a video generated from a model |
Import[file,elem] | import a specific element from a video file |
The video object is a reference to a local or remote file, with interactive video rendering capabilities. The actual data of the video file remains out-of-core, but parts of it such as image frames or audio chunks can be imported into the memory.
This creates an out-of-core video object taken from the Wolfram Language documentation "ExampleData" directory:
Generate a 10-second Video of images showing a cellular automaton evolution:
You can use Import to obtain specific data or metadata elements from a video file, such as duration, list of frames or audio data.
Import the audio content of the video as an Audio object or raw data:
The Video object has more than one purpose. It is a video file reference, a video data importer and a viewer. With respect to these functionalities, the Video object exhibits different sets of properties.
Video properties related to the video file and its content can be obtained by the following functions.
Duration[video] | give the duration of video |
Information[video,prop] | return property prop of video |
Import[file,"Summary"] | import a summary of the properties of file |
The Information function provides a video summary:
Use Information to extract a single property from the video object:
The "Summary" element of Import is also a convenient way to retrieve relevant properties of a video file.
The Video object can take options that control access and extraction of video data. These options will not change the content of the file, but instead determine the properties of extracted data for processing and display.
AudioTrackSelection | select audio tracks of interest |
RasterSize | gives the raster size of extracted video frames |
SubtitleTrackSelection | select subtitle tracks of interest |
VideoTrackSelection | select video tracks of interest |
By default, the original dimensions of the frames, as stored in the file, are used when extracting or processing frames.
Reduce the video resolution by specifying a smaller RasterSize:
Appearance | the overall appearance of the video player |
AudioOutputDevice | audio output device to use for video playback |
ImageSize | the image size of the displayed video |
SoundVolume | sound volume of the video player |
Audio[video] | returns an Audio object corresponding to the first audio track |
VideoFrameList[video,n] | returns a list of n video frames |
VideoExtractFrames[video,{t1,t2,…}] | extracts video frames at times t1,t2,… |
It is a common operation to trim a part of a video, delete portions of a video, add an audio track to a video and more. These video editing operations can be obtained by the following functions.
VideoCombine[video,audio,…] | combine video and audio objects into one video |
VideoDelete[video,{t1,t2}] | delete a segment of a video |
VideoJoin[video1,video2,…] | sequentially join multiple video objects |
VideoSplit[video,{t1,t2}] | split a video into multiple video segments |
VideoTranscode[video,format] | transcode a video to a specific format and codec |
VideoTrim[video,{t1,t2}] | trim a segment of a video |
Video processing or filtering is a video-in, video-out operation. Functions may operate on individual frames or a sequence of video frames to create new frames for the output video. Video analysis is the process of understanding the content of video by computing statistics or extracting features from single frame or a sequence of video frames.
AudioTrackApply[f,video] | applies f to the audio track of video, returning a new video |
VideoFrameMap[f,video,…] | applies f to partitions of video frames, returning a new video |
VideoMap[f,video,…] | applies f to video partitions, returning a new video |
VideoMapList[f,video,…] | applies f to video partitions, returning a list |
VideoMapTimeSeries[f,video,…] | applies f to video partitions, returning a time series |
VideoIntervals[video,crit,…] | checks crit on video partitions, returning intervals of interst |
VideoFrameMap applies image functions to individual frames or a sequence of video frames, resulting in a new video object.
VideoMapTimeSeries can be used for analyzing and detecting video content by applying functions to video frames that return an arbitrary result, returned as a TimeSeries.
The VideoStream expression can be used to programmatically manipulate the video playback.
VideoStream[video] | creates a new VideoStream object from video |
VideoPlay[vstream] | starts playing a VideoStream object vstream |
VideoPause[vstream] | pauses the playback of vstream |
VideoStop[vstream] | stops the playback vstream |
VideoStreams[] | returns all existing video streams |
RemoveVideoStream[vstream] | deletes the VideoStream object vstream |