Re: Max obtainable performance for a single OST

On Apr 13, 2007  12:00 +0200, Andrei Maslennikov wrote:
> We are currently evaluating possible commodity hardware candidates
> suitable for a single OSS with a single OST served to the clients via the
> IB/RDMA. The goal is to provide the peak performance around 1 GB/sec
> for large streaming I/O for a single file at the client level, *without*
> striping.
> In other words, we want to see if we could build a high performance
> standalone box which would be acting as a Lustre Head for a couple
> of clients (obviously, we will have to run also the metadata service on it).
> 
> Now comes the interesting point: if we run a single write process against
> a striped logical volume built upon the three available drives, we only are
> able to obtain 750 MB/sec. The writer process eats 100% of CPU, and
> there is no way to improve this.  This behaviour, of course, is perfectly
> normal, but for us this means that if we would base our OST on this
> combination of CPU + striped volume, we probably will never be able to
> spit out more than 750 MB/sec peak i/o perf to the clients. Unless
> the OST backend service itself is multithreaded!

You should try with 3 OSTs on the box, and set the filesystem default stripe
count to 3 and see if the mutli-threading on the client can work better than
that in LVM.  You will get 3x as many requests in flight, 3x as much write
cache space in Lustre.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.