tokenize

text →
tokenize(
    separator?: text
) → array

Returns all tokens in the argument value in source order. By default, tokenization uses white-space characters as delimiters. If a character is specified, that character delimits the tokens.

Parameters

Name	Type	Required	Description
separator	text	No	A character that delimits the tokens in the argument value.

Argument evaluation

• separator: Evaluated once in the enclosing context.

Examples

"Hello World" | tokenize → {"Hello", "World"}
"Hello,World" | tokenize(",") → {"Hello", "World"}

Kind: Function
Scope: text/tokenization
Aliases: text-to-tokenize