"VideoFrames" (Net Encoder)

NetEncoder["VideoFrames"]

represents an encoder that converts a video file or object into a sequence of rank-3 tensors of pixel values.

NetEncoder[{"VideoFrames","param"->val,…}]

represents an encoder with specific parameters for preprocessing.

Details

  • The "VideoFrames" encoder returns a list of video frames from the input.
  • NetEncoder[…][input] applies the encoder to an input to produce an output.
  • The input to the encoder can be one of the following:
  • videoa Video object
    filea video file given as a File expression
    {image1,image2,…}a list of images, assumed to be extracted from a video
  • NetEncoder[…][{input1,input2,…}] applies the encoder to a list of inputs to produce a list of outputs.
  • An encoder can be attached to an input port of a net by specifying "port"->NetEncoder[…] when constructing the net.
    • Parameters
  • The following parameters are supported:
  • ColorSpace"RGB"color space to use for encoding
    FrameRate24the frame rate at which frames are extracted
    "Levels"Automaticthe order of the levels in the result
    "MeanImage"Nonemean image to subtract
    RasterSize{128,128}raster size used to resize the frames
    "StartTime"0the time at which frame extraction starts
    "TargetLength"Automatictarget output length
    "VarianceImage"Nonevariance image to normalize by
    Method"Stretch"how to conform the size
    ResamplingAutomaticresampling method
    AlignmentCenterhow to align the image for Method"Fit" or "Fill"
    PaddingBlackpadding scheme for Method"Fit"
  • Possible values for Method are:
  • "Stretch"stretch the image to fit by resampling
    "Fit"fit the whole image; keep the aspect ratio; pad if necessary
    "Fill"fit the smaller dimension; crop the other if necessary
  • Some parameters can be specified at evaluation time using NetEncoder[…][{input,"param"val,…}].
  • By default, frames are extracted at 24 frames per second. Use FrameRateInherited to extract at the original frame rate.
  • By default, the result is returned as a rank-4 array of dimensions {time,channels,height,width}. Set the "Levels" parameter to a list including "Time", "Channels", "Height" and "Width" to control the order of the levels.
  • The following settings can be used for "MeanImage" and "VarianceImage":
  • Nonedo not subtract anything
    mvalue to subtract from each pixel
    {m1,m2,…}values to be subtracted from different channels
    Image[…]overall image to subtract
  • Pixels are normalized to lie between 0 and 1 before doing subtraction.
  • The following settings can be used for "TargetLength":
  • nextracts n frames
    Allextracts all available frames
    Automaticautomatically chooses the number of extracted frames

Examples

open allclose all

Basic Examples  (2)

Create a video frames NetEncoder:

Apply the encoder to a Video object:

Display the result of the encoder as images:

Scope  (3)

NetEncoder["VideoFrames"] can encode either File or Video objects. Create a video frames encoder:

Apply the encoder to a File object:

Apply the encoder to a Video object:

NetEncoder["VideoFrames"] maps across a batch of inputs:

Create a video frames NetEncoder:

Attach the encoder to the input of a net:

Apply the net to a Video object:

Parameters  (6)

ColorSpace  (1)

Set the ColorSpace of the output image to "Grayscale":

The output only has one channel dimension, appropriate for grayscale images:

FrameRate  (1)

Create a video frames encoder that extracts frames at a specified rate:

Specify the "FrameRate" parameter at evaluation time:

"MeanImage"  (1)

Create a video frames encoder that subtracts a list of mean values from the encoded frames:

Create a video with every pixel value being 0.8:

Apply the encoder to the video:

"StartTime"  (1)

Create a video frames encoder that starts the encoding at a specified time:

Specify the "StartTime" parameter at evaluation time:

"TargetLength"  (1)

Create a video frames encoder that returns 5 frames:

Create an encoder that returns all frames in the input video:

If the specified "TargetLength" is automatic, the length of the result can be controlled at evaluation time:

"VarianceImage"  (1)

Create a video frames encoder that normalizes the encoded frames by a list of variances:

Create a video with every pixel value being 0.8:

Apply the encoder to the video: