tokenize-regex

text →
tokenize-regex(
    pattern: text
) → array

Returns segments separated by regular expression matches in source order. Preserves spaces and empty segments without including captured delimiters. Returns an empty array for null or empty input.

Parameters

Name	Type	Required	Description
pattern	text	Yes	The .NET regular expression identifying delimiters. Matching is case-sensitive unless inline options specify otherwise.

Argument evaluation

• pattern: Evaluated once in the enclosing context.

Behavior

Zero-width delimiters split at each match position with forward progress. Invalid patterns raise a regular expression error when evaluated on nonempty text. Matching uses a one-second timeout and raises RegexMatchTimeoutException when the limit is exceeded. The blank special value is treated as one space.

Examples

"a, b;c" | tokenize-regex("[,;] *") → {"a", "b", "c"}

Kind: Function
Scope: text/tokenization
Aliases: None