Class StrTokenizer

java.lang.Object
org.apache.commons.lang3.text.StrTokenizer
All Implemented Interfaces:
Cloneable, Iterator<String>, ListIterator<String>
Direct Known Subclasses:
StrBuilder.StrBuilderTokenizer

@Deprecated public class StrTokenizer extends Object implements ListIterator<String>, Cloneable
Deprecated.
As of 3.6, use Apache Commons Text StringTokenizer instead
Tokenizes a string based on delimiters (separators) and supporting quoting and ignored character concepts.

This class can split a String into many smaller strings. It aims to do a similar job to StringTokenizer, however it offers much more control and flexibility including implementing the ListIterator interface. By default, it is set up like StringTokenizer.

The input String is split into a number of tokens. Each token is separated from the next String by a delimiter. One or more delimiter characters must be specified.

Each token may be surrounded by quotes. The quote matcher specifies the quote character(s). A quote may be escaped within a quoted section by duplicating itself.

Between each token and the delimiter are potentially characters that need trimming. The trimmer matcher specifies these characters. One usage might be to trim whitespace characters.

At any point outside the quotes there might potentially be invalid characters. The ignored matcher specifies these characters to be removed. One usage might be to remove new line characters.

Empty tokens may be removed or returned as null.

 "a,b,c"         - Three tokens "a","b","c"   (comma delimiter)
 " a, b , c "    - Three tokens "a","b","c"   (default CSV processing trims whitespace)
 "a, ", b ,", c" - Three tokens "a, " , " b ", ", c" (quoted text untouched)
 
StrTokenizer properties and options
PropertyTypeDefault
delimCharSetMatcher{ \t\n\r\f}
quoteNoneMatcher{}
ignoreNoneMatcher{}
emptyTokenAsNullbooleanfalse
ignoreEmptyTokensbooleantrue
Since:
2.2
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private char[]
    Deprecated.
    The text to work on.
    private static final StrTokenizer
    Deprecated.
     
    private StrMatcher
    Deprecated.
    The delimiter matcher
    private boolean
    Deprecated.
    Whether to return empty tokens as null
    private StrMatcher
    Deprecated.
    The ignored matcher
    private boolean
    Deprecated.
    Whether to ignore empty tokens
    private StrMatcher
    Deprecated.
    The quote matcher
    private int
    Deprecated.
    The current iteration position
    private String[]
    Deprecated.
    The parsed tokens
    private StrMatcher
    Deprecated.
    The trimmer matcher
    private static final StrTokenizer
    Deprecated.
     
  • Constructor Summary

    Constructors
    Constructor
    Description
    Deprecated.
    Constructs a tokenizer splitting on space, tab, newline and formfeed as per StringTokenizer, but with no text to tokenize.
    StrTokenizer(char[] input)
    Deprecated.
    Constructs a tokenizer splitting on space, tab, newline and formfeed as per StringTokenizer.
    StrTokenizer(char[] input, char delim)
    Deprecated.
    Constructs a tokenizer splitting on the specified character.
    StrTokenizer(char[] input, char delim, char quote)
    Deprecated.
    Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified quote character.
    StrTokenizer(char[] input, String delim)
    Deprecated.
    Constructs a tokenizer splitting on the specified string.
    StrTokenizer(char[] input, StrMatcher delim)
    Deprecated.
    Constructs a tokenizer splitting using the specified delimiter matcher.
    StrTokenizer(char[] input, StrMatcher delim, StrMatcher quote)
    Deprecated.
    Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified quote matcher.
    Deprecated.
    Constructs a tokenizer splitting on space, tab, newline and formfeed as per StringTokenizer.
    StrTokenizer(String input, char delim)
    Deprecated.
    Constructs a tokenizer splitting on the specified delimiter character.
    StrTokenizer(String input, char delim, char quote)
    Deprecated.
    Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified quote character.
    StrTokenizer(String input, String delim)
    Deprecated.
    Constructs a tokenizer splitting on the specified delimiter string.
    Deprecated.
    Constructs a tokenizer splitting using the specified delimiter matcher.
    StrTokenizer(String input, StrMatcher delim, StrMatcher quote)
    Deprecated.
    Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified quote matcher.
  • Method Summary

    Modifier and Type
    Method
    Description
    void
    add(String obj)
    Deprecated.
    Unsupported ListIterator operation.
    private void
    addToken(List<String> list, String tok)
    Deprecated.
    Adds a token to a list, paying attention to the parameters we've set.
    private void
    Deprecated.
    Checks if tokenization has been done, and if not then do it.
    Deprecated.
    Creates a new instance of this Tokenizer.
    (package private) Object
    Deprecated.
    Creates a new instance of this Tokenizer.
    Deprecated.
    Gets the String content that the tokenizer is parsing.
    private static StrTokenizer
    Deprecated.
    Returns a clone of CSV_TOKENIZER_PROTOTYPE.
    Deprecated.
    Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
    getCSVInstance(char[] input)
    Deprecated.
    Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
    Deprecated.
    Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
    Deprecated.
    Gets the field delimiter matcher.
    Deprecated.
    Gets the ignored character matcher.
    Deprecated.
    Gets the quote matcher currently in use.
    Deprecated.
    Gets a copy of the full token list as an independent modifiable array.
    Deprecated.
    Gets a copy of the full token list as an independent modifiable list.
    Deprecated.
    Gets the trimmer character matcher.
    private static StrTokenizer
    Deprecated.
    Returns a clone of TSV_TOKENIZER_PROTOTYPE.
    Deprecated.
    Gets a new tokenizer instance which parses Tab Separated Value strings.
    getTSVInstance(char[] input)
    Deprecated.
    Gets a new tokenizer instance which parses Tab Separated Value strings.
    Deprecated.
    Gets a new tokenizer instance which parses Tab Separated Value strings.
    boolean
    Deprecated.
    Checks whether there are any more tokens.
    boolean
    Deprecated.
    Checks whether there are any previous tokens that can be iterated to.
    boolean
    Deprecated.
    Gets whether the tokenizer currently returns empty tokens as null.
    boolean
    Deprecated.
    Gets whether the tokenizer currently ignores empty tokens.
    private boolean
    isQuote(char[] srcChars, int pos, int len, int quoteStart, int quoteLen)
    Deprecated.
    Checks if the characters at the index specified match the quote already matched in readNextToken().
    Deprecated.
    Gets the next token.
    int
    Deprecated.
    Gets the index of the next token to return.
    Deprecated.
    Gets the next token from the String.
    Deprecated.
    Gets the token previous to the last returned token.
    int
    Deprecated.
    Gets the index of the previous token.
    Deprecated.
    Gets the previous token from the String.
    private int
    readNextToken(char[] srcChars, int start, int len, StrBuilder workArea, List<String> tokenList)
    Deprecated.
    Reads character by character through the String to get the next token.
    private int
    readWithQuotes(char[] srcChars, int start, int len, StrBuilder workArea, List<String> tokenList, int quoteStart, int quoteLen)
    Deprecated.
    Reads a possibly quoted string token.
    void
    Deprecated.
    Unsupported ListIterator operation.
    Deprecated.
    Resets this tokenizer, forgetting all parsing and iteration already completed.
    reset(char[] input)
    Deprecated.
    Reset this tokenizer, giving it a new input string to parse.
    reset(String input)
    Deprecated.
    Reset this tokenizer, giving it a new input string to parse.
    void
    set(String obj)
    Deprecated.
    Unsupported ListIterator operation.
    setDelimiterChar(char delim)
    Deprecated.
    Sets the field delimiter character.
    Deprecated.
    Sets the field delimiter matcher.
    Deprecated.
    Sets the field delimiter string.
    setEmptyTokenAsNull(boolean emptyAsNull)
    Deprecated.
    Sets whether the tokenizer should return empty tokens as null.
    setIgnoredChar(char ignored)
    Deprecated.
    Sets the character to ignore.
    Deprecated.
    Sets the matcher for characters to ignore.
    setIgnoreEmptyTokens(boolean ignoreEmptyTokens)
    Deprecated.
    Sets whether the tokenizer should ignore and not return empty tokens.
    setQuoteChar(char quote)
    Deprecated.
    Sets the quote character to use.
    Deprecated.
    Sets the quote matcher to use.
    Deprecated.
    Sets the matcher for characters to trim.
    int
    Deprecated.
    Gets the number of tokens found in the String.
    protected List<String>
    tokenize(char[] srcChars, int offset, int count)
    Deprecated.
    Internal method to performs the tokenization.
    Deprecated.
    Gets the String content that the tokenizer is parsing.

    Methods inherited from class java.lang.Object

    equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface java.util.Iterator

    forEachRemaining
  • Field Details

    • CSV_TOKENIZER_PROTOTYPE

      private static final StrTokenizer CSV_TOKENIZER_PROTOTYPE
      Deprecated.
    • TSV_TOKENIZER_PROTOTYPE

      private static final StrTokenizer TSV_TOKENIZER_PROTOTYPE
      Deprecated.
    • chars

      private char[] chars
      Deprecated.
      The text to work on.
    • tokens

      private String[] tokens
      Deprecated.
      The parsed tokens
    • tokenPos

      private int tokenPos
      Deprecated.
      The current iteration position
    • delimMatcher

      private StrMatcher delimMatcher
      Deprecated.
      The delimiter matcher
    • quoteMatcher

      private StrMatcher quoteMatcher
      Deprecated.
      The quote matcher
    • ignoredMatcher

      private StrMatcher ignoredMatcher
      Deprecated.
      The ignored matcher
    • trimmerMatcher

      private StrMatcher trimmerMatcher
      Deprecated.
      The trimmer matcher
    • emptyAsNull

      private boolean emptyAsNull
      Deprecated.
      Whether to return empty tokens as null
    • ignoreEmptyTokens

      private boolean ignoreEmptyTokens
      Deprecated.
      Whether to ignore empty tokens
  • Constructor Details

    • StrTokenizer

      public StrTokenizer()
      Deprecated.
      Constructs a tokenizer splitting on space, tab, newline and formfeed as per StringTokenizer, but with no text to tokenize.

      This constructor is normally used with reset(String).

    • StrTokenizer

      public StrTokenizer(char[] input)
      Deprecated.
      Constructs a tokenizer splitting on space, tab, newline and formfeed as per StringTokenizer.
      Parameters:
      input - the string which is to be parsed, not cloned
    • StrTokenizer

      public StrTokenizer(char[] input, char delim)
      Deprecated.
      Constructs a tokenizer splitting on the specified character.
      Parameters:
      input - the string which is to be parsed, not cloned
      delim - the field delimiter character
    • StrTokenizer

      public StrTokenizer(char[] input, char delim, char quote)
      Deprecated.
      Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified quote character.
      Parameters:
      input - the string which is to be parsed, not cloned
      delim - the field delimiter character
      quote - the field quoted string character
    • StrTokenizer

      public StrTokenizer(char[] input, String delim)
      Deprecated.
      Constructs a tokenizer splitting on the specified string.
      Parameters:
      input - the string which is to be parsed, not cloned
      delim - the field delimiter string
    • StrTokenizer

      public StrTokenizer(char[] input, StrMatcher delim)
      Deprecated.
      Constructs a tokenizer splitting using the specified delimiter matcher.
      Parameters:
      input - the string which is to be parsed, not cloned
      delim - the field delimiter matcher
    • StrTokenizer

      public StrTokenizer(char[] input, StrMatcher delim, StrMatcher quote)
      Deprecated.
      Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified quote matcher.
      Parameters:
      input - the string which is to be parsed, not cloned
      delim - the field delimiter character
      quote - the field quoted string character
    • StrTokenizer

      public StrTokenizer(String input)
      Deprecated.
      Constructs a tokenizer splitting on space, tab, newline and formfeed as per StringTokenizer.
      Parameters:
      input - the string which is to be parsed
    • StrTokenizer

      public StrTokenizer(String input, char delim)
      Deprecated.
      Constructs a tokenizer splitting on the specified delimiter character.
      Parameters:
      input - the string which is to be parsed
      delim - the field delimiter character
    • StrTokenizer

      public StrTokenizer(String input, char delim, char quote)
      Deprecated.
      Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified quote character.
      Parameters:
      input - the string which is to be parsed
      delim - the field delimiter character
      quote - the field quoted string character
    • StrTokenizer

      public StrTokenizer(String input, String delim)
      Deprecated.
      Constructs a tokenizer splitting on the specified delimiter string.
      Parameters:
      input - the string which is to be parsed
      delim - the field delimiter string
    • StrTokenizer

      public StrTokenizer(String input, StrMatcher delim)
      Deprecated.
      Constructs a tokenizer splitting using the specified delimiter matcher.
      Parameters:
      input - the string which is to be parsed
      delim - the field delimiter matcher
    • StrTokenizer

      public StrTokenizer(String input, StrMatcher delim, StrMatcher quote)
      Deprecated.
      Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified quote matcher.
      Parameters:
      input - the string which is to be parsed
      delim - the field delimiter matcher
      quote - the field quoted string matcher
  • Method Details

    • getCSVClone

      private static StrTokenizer getCSVClone()
      Deprecated.
      Returns a clone of CSV_TOKENIZER_PROTOTYPE.
      Returns:
      a clone of CSV_TOKENIZER_PROTOTYPE.
    • getCSVInstance

      public static StrTokenizer getCSVInstance()
      Deprecated.
      Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input. The default for CSV processing will be trim whitespace from both ends (which can be overridden with the setTrimmer method).

      You must call a "reset" method to set the string which you want to parse.

      Returns:
      a new tokenizer instance which parses Comma Separated Value strings
    • getCSVInstance

      public static StrTokenizer getCSVInstance(char[] input)
      Deprecated.
      Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input. The default for CSV processing will be trim whitespace from both ends (which can be overridden with the setTrimmer method).
      Parameters:
      input - the text to parse
      Returns:
      a new tokenizer instance which parses Comma Separated Value strings
    • getCSVInstance

      public static StrTokenizer getCSVInstance(String input)
      Deprecated.
      Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input. The default for CSV processing will be trim whitespace from both ends (which can be overridden with the setTrimmer method).
      Parameters:
      input - the text to parse
      Returns:
      a new tokenizer instance which parses Comma Separated Value strings
    • getTSVClone

      private static StrTokenizer getTSVClone()
      Deprecated.
      Returns a clone of TSV_TOKENIZER_PROTOTYPE.
      Returns:
      a clone of TSV_TOKENIZER_PROTOTYPE.
    • getTSVInstance

      public static StrTokenizer getTSVInstance()
      Deprecated.
      Gets a new tokenizer instance which parses Tab Separated Value strings. The default for CSV processing will be trim whitespace from both ends (which can be overridden with the setTrimmer method).

      You must call a "reset" method to set the string which you want to parse.

      Returns:
      a new tokenizer instance which parses Tab Separated Value strings.
    • getTSVInstance

      public static StrTokenizer getTSVInstance(char[] input)
      Deprecated.
      Gets a new tokenizer instance which parses Tab Separated Value strings. The default for CSV processing will be trim whitespace from both ends (which can be overridden with the setTrimmer method).
      Parameters:
      input - the string to parse
      Returns:
      a new tokenizer instance which parses Tab Separated Value strings.
    • getTSVInstance

      public static StrTokenizer getTSVInstance(String input)
      Deprecated.
      Gets a new tokenizer instance which parses Tab Separated Value strings. The default for CSV processing will be trim whitespace from both ends (which can be overridden with the setTrimmer method).
      Parameters:
      input - the string to parse
      Returns:
      a new tokenizer instance which parses Tab Separated Value strings.
    • add

      public void add(String obj)
      Deprecated.
      Unsupported ListIterator operation.
      Specified by:
      add in interface ListIterator<String>
      Parameters:
      obj - this parameter ignored.
      Throws:
      UnsupportedOperationException - always
    • addToken

      private void addToken(List<String> list, String tok)
      Deprecated.
      Adds a token to a list, paying attention to the parameters we've set.
      Parameters:
      list - the list to add to
      tok - the token to add
    • checkTokenized

      private void checkTokenized()
      Deprecated.
      Checks if tokenization has been done, and if not then do it.
    • clone

      public Object clone()
      Deprecated.
      Creates a new instance of this Tokenizer. The new instance is reset so that it will be at the start of the token list. If a CloneNotSupportedException is caught, return null.
      Overrides:
      clone in class Object
      Returns:
      a new instance of this Tokenizer which has been reset.
    • cloneReset

      Object cloneReset() throws CloneNotSupportedException
      Deprecated.
      Creates a new instance of this Tokenizer. The new instance is reset so that it will be at the start of the token list.
      Returns:
      a new instance of this Tokenizer which has been reset.
      Throws:
      CloneNotSupportedException - if there is a problem cloning
    • getContent

      public String getContent()
      Deprecated.
      Gets the String content that the tokenizer is parsing.
      Returns:
      the string content being parsed
    • getDelimiterMatcher

      public StrMatcher getDelimiterMatcher()
      Deprecated.
      Gets the field delimiter matcher.
      Returns:
      the delimiter matcher in use
    • getIgnoredMatcher

      public StrMatcher getIgnoredMatcher()
      Deprecated.
      Gets the ignored character matcher.

      These characters are ignored when parsing the String, unless they are within a quoted region. The default value is not to ignore anything.

      Returns:
      the ignored matcher in use
    • getQuoteMatcher

      public StrMatcher getQuoteMatcher()
      Deprecated.
      Gets the quote matcher currently in use.

      The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data. The default value is '"' (double quote).

      Returns:
      the quote matcher in use
    • getTokenArray

      public String[] getTokenArray()
      Deprecated.
      Gets a copy of the full token list as an independent modifiable array.
      Returns:
      the tokens as a String array
    • getTokenList

      public List<String> getTokenList()
      Deprecated.
      Gets a copy of the full token list as an independent modifiable list.
      Returns:
      the tokens as a String array
    • getTrimmerMatcher

      public StrMatcher getTrimmerMatcher()
      Deprecated.
      Gets the trimmer character matcher.

      These characters are trimmed off on each side of the delimiter until the token or quote is found. The default value is not to trim anything.

      Returns:
      the trimmer matcher in use
    • hasNext

      public boolean hasNext()
      Deprecated.
      Checks whether there are any more tokens.
      Specified by:
      hasNext in interface Iterator<String>
      Specified by:
      hasNext in interface ListIterator<String>
      Returns:
      true if there are more tokens
    • hasPrevious

      public boolean hasPrevious()
      Deprecated.
      Checks whether there are any previous tokens that can be iterated to.
      Specified by:
      hasPrevious in interface ListIterator<String>
      Returns:
      true if there are previous tokens
    • isEmptyTokenAsNull

      public boolean isEmptyTokenAsNull()
      Deprecated.
      Gets whether the tokenizer currently returns empty tokens as null. The default for this property is false.
      Returns:
      true if empty tokens are returned as null
    • isIgnoreEmptyTokens

      public boolean isIgnoreEmptyTokens()
      Deprecated.
      Gets whether the tokenizer currently ignores empty tokens. The default for this property is true.
      Returns:
      true if empty tokens are not returned
    • isQuote

      private boolean isQuote(char[] srcChars, int pos, int len, int quoteStart, int quoteLen)
      Deprecated.
      Checks if the characters at the index specified match the quote already matched in readNextToken().
      Parameters:
      srcChars - the character array being tokenized
      pos - the position to check for a quote
      len - the length of the character array being tokenized
      quoteStart - the start position of the matched quote, 0 if no quoting
      quoteLen - the length of the matched quote, 0 if no quoting
      Returns:
      true if a quote is matched
    • next

      public String next()
      Deprecated.
      Gets the next token.
      Specified by:
      next in interface Iterator<String>
      Specified by:
      next in interface ListIterator<String>
      Returns:
      the next String token
      Throws:
      NoSuchElementException - if there are no more elements
    • nextIndex

      public int nextIndex()
      Deprecated.
      Gets the index of the next token to return.
      Specified by:
      nextIndex in interface ListIterator<String>
      Returns:
      the next token index
    • nextToken

      public String nextToken()
      Deprecated.
      Gets the next token from the String. Equivalent to next() except it returns null rather than throwing NoSuchElementException when no tokens remain.
      Returns:
      the next sequential token, or null when no more tokens are found
    • previous

      public String previous()
      Deprecated.
      Gets the token previous to the last returned token.
      Specified by:
      previous in interface ListIterator<String>
      Returns:
      the previous token
    • previousIndex

      public int previousIndex()
      Deprecated.
      Gets the index of the previous token.
      Specified by:
      previousIndex in interface ListIterator<String>
      Returns:
      the previous token index
    • previousToken

      public String previousToken()
      Deprecated.
      Gets the previous token from the String.
      Returns:
      the previous sequential token, or null when no more tokens are found
    • readNextToken

      private int readNextToken(char[] srcChars, int start, int len, StrBuilder workArea, List<String> tokenList)
      Deprecated.
      Reads character by character through the String to get the next token.
      Parameters:
      srcChars - the character array being tokenized
      start - the first character of field
      len - the length of the character array being tokenized
      workArea - a temporary work area
      tokenList - the list of parsed tokens
      Returns:
      the starting position of the next field (the character immediately after the delimiter), or -1 if end of string found
    • readWithQuotes

      private int readWithQuotes(char[] srcChars, int start, int len, StrBuilder workArea, List<String> tokenList, int quoteStart, int quoteLen)
      Deprecated.
      Reads a possibly quoted string token.
      Parameters:
      srcChars - the character array being tokenized
      start - the first character of field
      len - the length of the character array being tokenized
      workArea - a temporary work area
      tokenList - the list of parsed tokens
      quoteStart - the start position of the matched quote, 0 if no quoting
      quoteLen - the length of the matched quote, 0 if no quoting
      Returns:
      the starting position of the next field (the character immediately after the delimiter, or if end of string found, then the length of string
    • remove

      public void remove()
      Deprecated.
      Unsupported ListIterator operation.
      Specified by:
      remove in interface Iterator<String>
      Specified by:
      remove in interface ListIterator<String>
      Throws:
      UnsupportedOperationException - always
    • reset

      public StrTokenizer reset()
      Deprecated.
      Resets this tokenizer, forgetting all parsing and iteration already completed.

      This method allows the same tokenizer to be reused for the same String.

      Returns:
      this, to enable chaining
    • reset

      public StrTokenizer reset(char[] input)
      Deprecated.
      Reset this tokenizer, giving it a new input string to parse. In this manner you can re-use a tokenizer with the same settings on multiple input lines.
      Parameters:
      input - the new character array to tokenize, not cloned, null sets no text to parse
      Returns:
      this, to enable chaining
    • reset

      public StrTokenizer reset(String input)
      Deprecated.
      Reset this tokenizer, giving it a new input string to parse. In this manner you can re-use a tokenizer with the same settings on multiple input lines.
      Parameters:
      input - the new string to tokenize, null sets no text to parse
      Returns:
      this, to enable chaining
    • set

      public void set(String obj)
      Deprecated.
      Unsupported ListIterator operation.
      Specified by:
      set in interface ListIterator<String>
      Parameters:
      obj - this parameter ignored.
      Throws:
      UnsupportedOperationException - always
    • setDelimiterChar

      public StrTokenizer setDelimiterChar(char delim)
      Deprecated.
      Sets the field delimiter character.
      Parameters:
      delim - the delimiter character to use
      Returns:
      this, to enable chaining
    • setDelimiterMatcher

      public StrTokenizer setDelimiterMatcher(StrMatcher delim)
      Deprecated.
      Sets the field delimiter matcher.

      The delimiter is used to separate one token from another.

      Parameters:
      delim - the delimiter matcher to use
      Returns:
      this, to enable chaining
    • setDelimiterString

      public StrTokenizer setDelimiterString(String delim)
      Deprecated.
      Sets the field delimiter string.
      Parameters:
      delim - the delimiter string to use
      Returns:
      this, to enable chaining
    • setEmptyTokenAsNull

      public StrTokenizer setEmptyTokenAsNull(boolean emptyAsNull)
      Deprecated.
      Sets whether the tokenizer should return empty tokens as null. The default for this property is false.
      Parameters:
      emptyAsNull - whether empty tokens are returned as null
      Returns:
      this, to enable chaining
    • setIgnoredChar

      public StrTokenizer setIgnoredChar(char ignored)
      Deprecated.
      Sets the character to ignore.

      This character is ignored when parsing the String, unless it is within a quoted region.

      Parameters:
      ignored - the ignored character to use
      Returns:
      this, to enable chaining
    • setIgnoredMatcher

      public StrTokenizer setIgnoredMatcher(StrMatcher ignored)
      Deprecated.
      Sets the matcher for characters to ignore.

      These characters are ignored when parsing the String, unless they are within a quoted region.

      Parameters:
      ignored - the ignored matcher to use, null ignored
      Returns:
      this, to enable chaining
    • setIgnoreEmptyTokens

      public StrTokenizer setIgnoreEmptyTokens(boolean ignoreEmptyTokens)
      Deprecated.
      Sets whether the tokenizer should ignore and not return empty tokens. The default for this property is true.
      Parameters:
      ignoreEmptyTokens - whether empty tokens are not returned
      Returns:
      this, to enable chaining
    • setQuoteChar

      public StrTokenizer setQuoteChar(char quote)
      Deprecated.
      Sets the quote character to use.

      The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data.

      Parameters:
      quote - the quote character to use
      Returns:
      this, to enable chaining
    • setQuoteMatcher

      public StrTokenizer setQuoteMatcher(StrMatcher quote)
      Deprecated.
      Sets the quote matcher to use.

      The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data.

      Parameters:
      quote - the quote matcher to use, null ignored
      Returns:
      this, to enable chaining
    • setTrimmerMatcher

      public StrTokenizer setTrimmerMatcher(StrMatcher trimmer)
      Deprecated.
      Sets the matcher for characters to trim.

      These characters are trimmed off on each side of the delimiter until the token or quote is found.

      Parameters:
      trimmer - the trimmer matcher to use, null ignored
      Returns:
      this, to enable chaining
    • size

      public int size()
      Deprecated.
      Gets the number of tokens found in the String.
      Returns:
      the number of matched tokens
    • tokenize

      protected List<String> tokenize(char[] srcChars, int offset, int count)
      Deprecated.
      Internal method to performs the tokenization.

      Most users of this class do not need to call this method. This method will be called automatically by other (public) methods when required.

      This method exists to allow subclasses to add code before or after the tokenization. For example, a subclass could alter the character array, offset or count to be parsed, or call the tokenizer multiple times on multiple strings. It is also be possible to filter the results.

      StrTokenizer will always pass a zero offset and a count equal to the length of the array to this method, however a subclass may pass other values, or even an entirely different array.

      Parameters:
      srcChars - the character array being tokenized, may be null
      offset - the start position within the character array, must be valid
      count - the number of characters to tokenize, must be valid
      Returns:
      the modifiable list of String tokens, unmodifiable if null array or zero count
    • toString

      public String toString()
      Deprecated.
      Gets the String content that the tokenizer is parsing.
      Overrides:
      toString in class Object
      Returns:
      the string content being parsed