On Tue, 24 Mar 2003, Steven Kehlet wrote:
> Have you tried tcp?
I have with some interesting results.
> I'm still trying various kernel revs/mount options to get rid of my NFS
> woes... tcp looks promising, would probably help your problem of
> fragmentation, see the following NetApp bug:
So far the only other thing i've been able to think of is having smart
switches which are jumbo frame capable and using that. But i don't
have such switches so i haven't been able to try.. (anyone want to
donate a couple of cisco 3550-12s to me ? :-)
> I'm currently trying 2.4.18-27 with the following mount options:
>
> tcp,rsize=32768,wsize=32768,hard,intr,timeo=600
>
> so far no errors on a very lightly loaded system. I'll try my Oracle
> server later this week...
>From talking to some redhat guys their view was it was best to use 4096
rsize/wsize even with tcp due to the intel page size and readahead.
I have used tcp with upto 32768 and it does work.. but i would get a
couple of interesting behaviours
o the server will spew a lot of 'svc: server socket destroyed' which is
apparently harmless. to 'fix' this you have to edit a link of code
in /usr/src/linux-2.4/net/sunrpc/svcsock.c
/* printk(KERN_NOTICE "svc: server socket destroy delayed\n"); */
dprintk(KERN_NOTICE "svc: server socket destroy delayed\n");
o your read performance will improve a lot but your latency might
show a few issues on heavily loaded nfs filesystems. we had all
sorts of strange delays being introduced after trying out tcp.
it worked much better for lightly loaded (100G/day or less) filesystems
than heavily loaded (500G a day or more) ones.
other things to try in /etc/sysctl.conf include:
# Performance tuning to increase fragment buffer memory
net.ipv4.ipfrag_high_thresh = 4194304
net.ipv4.ipfrag_low_thresh = 1048576
# Performance tuning to increase tcp read/write buffers
net.ipv4.tcp_rmem = 4096 349520 1048576
net.ipv4.tcp_wmem = 4096 131072 1048576
and in /etc/init.d/nfs
echo -n $"Starting NFS daemon: "
echo 1056768 > /proc/sys/net/core/rmem_default
echo 1056768 > /proc/sys/net/core/rmem_max
echo 1056768 > /proc/sys/net/core/wmem_default
echo 1056768 > /proc/sys/net/core/wmem_max
daemon rpc.nfsd $RPCNFSDCOUNT
echo 65536 > /proc/sys/net/core/rmem_default
echo 131072 > /proc/sys/net/core/rmem_max
echo 65536 > /proc/sys/net/core/wmem_default
echo 131072 > /proc/sys/net/core/wmem_max
this value was calculated as per a recommendation from chris pascoe
to be 1024*(RPCNFSDCOUNT+1) where for me this is 128.
regards,
-jason
_______________________________________________
Linux-PowerEdge mailing list
Linux-PowerEdge@xxxxxxxx
http://lists.us.dell.com/mailman/listinfo/linux-poweredge
Please read the FAQ at http://lists.us.dell.com/faq or search the list archives
at http://lists.us.dell.com/htdig/
|
