"VideoFrames" (Net Encoder)

NetEncoder["VideoFrames"]

represents an encoder that converts a video file or object into a sequence of rank-3 tensors of pixel values.

NetEncoder[{"VideoFrames","param"->val,}]

represents an encoder with specific parameters for preprocessing.

Details

  • The "VideoFrames" encoder returns a list of video frames from the input.
  • NetEncoder[][input] applies the encoder to an input to produce an output.
  • NetEncoder[][{input1,input2,}] applies the encoder to a list of inputs to produce a list of outputs.
  • The input to the encoder can be a Video[] object or a File[] expression.
  • An encoder can be attached to an input port of a net by specifying "port"->NetEncoder[] when constructing the net.
  • Parameters
  • The following parameters are supported:
  • ColorSpace"RGB"color space to use for encoding
    FrameRateAutomaticthe frame rate at which frames are extracted
    InterleavingFalsewhether channels are interleaved
    "MeanImage"Nonemean image to subtract
    RasterSize{128,128}raster size used to resize the frames
    "StartTime"0the time at which frame extraction starts
    "TargetLength"Automatictarget output length
    "VarianceImage"Nonevariance image to normalize by
  • Some parameters can be specified at evaluation time using NetEncoder[][{input,"param"val,}].
  • The output of the encoder is a list of image data, with each item represented as a rank-3 array of dimensions {c,height,width} when InterleavingFalse and {height,width,c} when InterleavingTrue, where c is the number of color channels.
  • The following settings can be used for "MeanImage" and "VarianceImage":
  • Nonedo not subtract anything
    mvalue to subtract from each pixel
    {m1,m2,}values to be subtracted from different channels
    Image[]overall image to subtract
  • Pixels are normalized to lie between 0 and 1 before doing subtraction.
  • The following settings can be used for "TargetLength":
  • nextracts n frames
    Allextracts all available frames
    Automaticautomatically chooses the number of extracted frames

Examples

open allclose all

Basic Examples  (2)

Create a video frames NetEncoder:

Apply the encoder to a Video object:

Display the result of the encoder as images:

Scope  (3)

NetEncoder["VideoFrames"] can encode either File or Video objects. Create a video frames encoder:

Apply the encoder to a File object:

Apply the encoder to a Video object:

NetEncoder["VideoFrames"] maps across a batch of inputs:

Create a video frames NetEncoder:

Attach the encoder to the input of a net:

Apply the net to a Video object:

Parameters  (7)

ColorSpace  (1)

Set the ColorSpace of the output image to "Grayscale":

The output only has one channel dimension, appropriate for grayscale images:

FrameRate  (1)

Create a video frames encoder that extracts frames at a specified rate:

Specify the "FrameRate" parameter at evaluation time:

Interleaving  (1)

With InterleavingFalse, the channel dimension of the output is the second dimension in the dimensions list:

With InterleavingTrue, the channel dimension of the output is the last dimension in the dimensions list:

"MeanImage"  (1)

Create a video frames encoder that subtracts a list of mean values from the encoded frames:

Create a video with every pixel value being 0.8:

Apply the encoder to the video:

"StartTime"  (1)

Create a video frames encoder that starts the encoding at a specified time:

Specify the "StartTime" parameter at evaluation time:

"TargetLength"  (1)

Create a video frames encoder that returns 5 frames:

Create an encoder that returns all frames in the input video:

If the specified "TargetLength" is automatic, the length of the result can be controlled at evaluation time:

"VarianceImage"  (1)

Create a video frames encoder that normalizes the encoded frames by a list of variances:

Create a video with every pixel value being 0.8:

Apply the encoder to the video: