Difference between?

A string is a sequence of characters.

A stringtokenizer will parse a string into multiple strings, dividing it at whatever characters you want.

string = "the cat in the hat"

If we load this string into a stringtokenizer and use the space character as the delimiter then nextToken will return "the". Successive calls will return; "cat","in","the","hat".

If you have more questions please feel free to post back.

Nice post.

String.split() and Pattern.split() give you an easy syntax for doing the latter, but that's essentially all that they do. If you want to parse the resulting strings, or change the delimiter halfway through depending on a particular token, they won't help you with that.

StringTokenizer is even more restrictive than String.split(), and also a bit fiddlier to use. It is essentially designed for pulling out tokens delimited by fixed substrings. Because of this restriction, it's about twice as fast as String.split(). (See my comparison of String.split() and StringTokenizer.) It also predates the regular expressions API, of which String.split() is a part.