Mulitple Indexes or one big index??

I was wondering, is it better to maintain one large index or several
smaller indexes?

I am allowing users to upload files whose corresponding data is
subsequently parsed and indexed.  The problem is that each of these
documents has different fields (which the user defines) and this makes
searching and displaying search results slightly difficult.  I tried to
attach a unique ID to each file indexed and then conduct a search within
the documents that have the proper ID, but this failed.  My code:

all_databases=OtherDatabase.find(:all)
all_databases.each do |dbase|
   index.search_each('id:("#{dbase.id}) *:(*'+"#{@phrase}"+'*)',
:limit=>100) do |doc, score|
     @scores << score # add score to score array
     @init_results << index[doc].load #
  end
end

My solution to this would be to just have many indexes (eventually these
may number in the hundreds) each stored in its own unique directory and
then open, search, and close each index as needed.  Is this a good idea?

Any input would really help,

Thanks

-- 
Posted via http://www.ruby-forum.com/.