"Tokens" (Net Decoder)
-
See Also
- NetDecoder
- NetEncoder
- NetChain
- NetGraph
-
- Net Encoders
- Tokens
- Characters
- Class
- SubwordTokens
-
- Net Decoders
- Characters
- Class
- SubwordTokens
- Related Guides
- Tech Notes
"Tokens" (Net Decoder)
NetDecoder["Tokens"]
represents a decoder that converts a sequence of probability vectors to a string of English vocabulary words.
NetDecoder[{"Tokens","language"}]
represents a decoder that uses a standard vocabulary for the given language.
NetDecoder[{"Tokens",{"token1","token2",…}}]
represents a decoder that uses a specified list of tokens as the vocabulary.
NetDecoder[{"Tokens",…,"param"->val}]
represents a decoder in which additional parameters have been specified.
Details

- NetDecoder[…][input] applies the decoder to an input to produce an output.
- NetDecoder[…][{input1,input2,…}] applies the decoder to a list of inputs to produce a list of outputs.
- The input to the decoder input is either a vector of probabilities or a sequence of probability vectors. Each probability vector sums to 1. The length of each probability vector is the size of the vocabulary.
- NetDecoder["Tokens"] is equivalent to NetDecoder[{"Tokens","English"}].
- A decoder can be attached to an output port of a net by specifying "port"->NetDecoder[…] when constructing the net.
- With the parameter "IgnoreCase"->True, tokens are converted to lowercase after decoding. The default is True.
- NetDecoder[…][data,prop] can be used to calculate a specific property for the input data.
- When a "Tokens" decoder is attached to a net, net[data,prop] or net[data,"oport"->prop] can be used to calculate a specific property of the decoded output.
- The "Tokens" decoder only supports the bypass property. Setting prop to None bypasses decoding and returns the input to the decoder.
Parameters
Properties
Examples
open all close allBasic Examples (2)
See Also
NetDecoder NetEncoder NetChain NetGraph
Net Encoders: Tokens Characters Class SubwordTokens
Net Decoders: Characters Class SubwordTokens