TokenizerProperties
instance use
the constructor TokenizerProperty(int, java.lang.String[], java.lang.Object, int, int)
Tokenizer
to count lines and columns.Tokenizer
buffer all data.TokenizerProperties
instance
(see #setParseFlags
, input data is generally treated case-insensitive.TokenizerProperties
instance, a
Tokenizer
or for a single property, a tokenizer returns not only
the token images but also image parts (see Token.getImageParts()
).