Tools.h++ Manual

21-88 104011 Tandem Computers Incorporated
21
RWCTokenizer
Synopsis #include <rw/ctoken.h>
RWCString str("a string of tokens");
RWCTokenizer(str); // Lex the above string
Description Class RWCTokenizer is designed to break a string up into separate tokens,
delimited by an arbitrary "white space". It can be thought of as an iterator for
strings and as an alternative to the ANSI C function strtok() which has the
unfortunate side effect of changing the string being tokenized.
Example
Program output:
Something
is
rotten
in
the
state
of
Denmark
#include <rw/ctoken.h>
#include <rw/rstream.h>
main()
{
RWCString a("Something is rotten in the state of Denmark");
RWCTokenizer next(a); // Tokenize the string a
RWCString token; // Will receive each token
// Advance until the null string is returned:
while (!(token=next()).isNull())
cout << token << "\n";
}