Tokens in Java Programming  are the basic building blocks which can be put together to construct a program.
Tokens can be a reserved word ,a identifier,a constant , or a operator.  

Class String Tokenizier of java.util package  is allows an application to break a string into tokens.

 StringTokenizier method of StringTokenizier class is break the  string into tokens.
StingTokenizier method do not distiguishes between the reserved words ,identifiers,constants delimiters nor they recognise and skip comments.

StringTokenizer st = new StringTokenizer("this is a test");   
  while (st.hasMoreTokens()) 
{         System.out.println(st.nextToken());  
   } 

this
is 
a 
test