Scilab Function

tokens - returns the tokens of a character string.

### Calling Sequence

  `T=tokens(str [,delimiter])`

### Parameters

- `str` : a character string. The string where to search the tokens.
- `delimiter` : (optional) a character or a vector of characters. The tokens delimeters.
- `T` : column vector of found tokens

### Description

`T=tokens(str [,delimiter])` searches the tokens included in the string `str`. The `delimiter` default value is [" ",<Tab>] where <Tab> stands for `ascii(9)`.

### Examples

    tokens('This is a character string')

    tokens('SCI/demos/scicos','/')

    tokens('y=a+b*2',['=','+','*'])

### See Also

`strindex` `tokenpos`