TokenizerProperties instance use
the constructor TokenizerProperty(int, java.lang.String[], java.lang.Object, int, int)Tokenizer to count lines and columns.Tokenizer buffer all data.TokenizerProperties instance
(see #setParseFlags, input data is generally treated case-insensitive.TokenizerProperties instance, a
Tokenizer or for a single property, a tokenizer returns not only
the token images but also image parts (see Token.getImageParts()).