TokenBatch should really be Field

Greets,

Java Lucene assumes that tokens should be processed one by one, in a  
stream.  I assert that while that may work fine in Java, in Perl it  
is best to process them in batches.
A batch of tokens is generated by tokenizing a single string.  A  
Field has one string.  I assert that the fundamental unit that should  
be passed around in Plucene is the Field.  (This is an approach I was  
moving towards with Kinosearch.)  texts, starts, ends, and types  
arrays can be added to the Field class.
Oh, and it would be nice if it were a perfect world.  ;)

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/