Scan String For Tokens

Scans input string, starting at offset, and returns the next token found. Refer to the Parse Arithmetic Expression VI in the examples\general\strings.llb for an example of using this function.

If allow empty tokens? is FALSE (default), multiple adjacent delimiters can separate two valid tokens in input string. If allow empty tokens? is TRUE, an empty token string is returned between every pair of adjacent delimiters.
input string is the string to scan for elements in operators. A token is a substring of input string, which is surrounded by delimiters, or which matches an element in operators. Typically, tokens represent individual keywords, numeric values, or operators found when parsing a configuration file or other text-based data format. This function scans input string, starting at offset, returning the next token found.
offset is the offset in input string to begin scanning. It must be numeric and defaults to 0, which specifies the beginning of the string.
operators contains an array of strings which are to be considered as tokens when they appear in input string, even if they are not surrounded by delimiters. If a portion of input string can match more than one string in operators, the longest match is chosen. For example, if >, = and >= are operators, the input string >=0 would produce >= as the next token string.

A string in operators may contain the following special format codes, which you can use to scan entire numbers as single tokens.

%dmatch decimal integer
%omatch octal integer
%xmatch hexadecimal integer
%bmatch binary integer
%e,%f,%gmatch floating point or scientific real number
%%match a single % character
Note  If the strings + or – appear as operators, these format codes do not match leading (unary) +/- signs; these return as separate tokens. This is an exception to the "longest match" rule.
delimiters is an array of strings that are to act as delimiters between tokens. Strings in delimiters are skipped during scanning and not returned as tokens but serve to separate adjacent tokens from each other. The default is white space characters (SPACE, TAB, new line, carriage return).
use cached delim/oper data? is an advanced optional input. If unwired, token string still behaves correctly. However, you can use use cached delim/oper data? to greatly improve string parsing performance. Set use cached delim/oper data? to FALSE the first time token string is executed, and TRUE each subsequent time as long as operators and delimiters have not changed. Use a shift register with a constant FALSE coming in and a constant TRUE going out to ensure correct behavior if operators and delimiters do not change during the execution of the loop. If use cached delim/oper data? is TRUE and operators or delimiters has changed since the last execution, incorrect output might result. If both operators and delimiters are unwired or are wired to block diagram constants, you can leave use cached delim/oper data? unwired and still achieve optimal performance.
dup string contains input string passed through unchanged, for wiring convenience.
offset past token is the offset in the input string immediately past the token that was found and any trailing delimiter. Any subsequent scanning of the same input string should begin at this offset. If offset is less than 0 or greater than the number of characters in input string, offset past token is –1.
token string is the matched token. It might be one of the strings in operators, or any string that appeared between delimiters in input string. Use Scan String for Tokens in a while loop that processes one set of operator(s), at a time until token index becomes –2 (indicating end of string). offset past token can be passed through a shift register into offset, so that each scan starts where the subsequent scan ended.
If token string matches one of the strings in operators, token index is the index of this string in operators. If token string is any other string, token index has the value –1. If the end of input string was reached without finding any valid operator(s), token index has the value –2.