KinoSearch1::Analysis::TokenBatch - a collection of tokens

       All of TokenBatch's accessor methods affect the current Token.  Calling any of these methods when the
       TokenBatch is not located at a valid Token will trigger an exception.

   set_textget_text
       Set/get the text of the current Token.

   set_start_offsetget_start_offset
       Set/get the start_offset of the current Token.

   set_end_offsetget_end_offset
       Set/get the end_offset of the current Token.

   set_pos_incget_pos_inc
       Set/get the position increment of the current Token.

       Copyright 2005-2010 Marvin Humphrey

       A TokenBatch is a collection of Tokens which you can add to, then iterate over.

       TokenBatch's API should be considered experimental and is likely to change.

       See KinoSearch1 version 1.01.

perl v5.40.0                                       2024-10-20             KinoSearch1::Analysis::TokenBatch(3pm)

new
           my $batch = KinoSearch1::Analysis::TokenBatch->new;

       Constructor.

   append
           $batch->append( $text, $start_offset, $end_offset, $pos_inc );

       Add a Token to the end of the batch.  Accepts either three or four arguments: text, start_offset,
       end_offset, and an optional position increment which defaults to 1 if not supplied.  For a description of
       what these arguments mean, see the docs for Token.

   next
           while ( $batch->next ) {
               # ...
           }

       Proceed to the next token in the TokenBatch.  Returns true if the TokenBatch ends up located at valid
       token.

       KinoSearch1::Analysis::TokenBatch - a collection of tokens

           while ( $batch->next ) {
               $batch->set_text( lc( $batch->get_text ) );
           }