Subject: RE: Rsync died on big directories/file-lists - msg#00081

Previous Message by Date: click to view message preview

Re: Rsync died on big directories/file-lists

On Tue, Dec 09, 2003 at 01:54:25PM -0500, Vu Huynh wrote: > > > Hi, > > My name is Vu Huynh. I'm currently work for Atmel Corporation. We're > using Rsync v2.5.7 to rsync design kits between our remote sites. It > works > fine for all small kit directories. However, there is 1 BIG kit directory > (But there is no file has the size of bigger than 100MB) that always gets > "rsync: connection unexpectedly closed" Error. The Full Error Message is > below: > > % rsync --password-file -avz user@host::KIT1 KIT1 --exclude '5.*' > --exclude '6.*' > receiving file list ... done > ./ > 7.0/ > 7.0/bin/ > 7.0/bin/AWK/ > 7.0/bin/IA32,Linux_2.4.7-10,32bits/ > .............. > .............. > 7.0/tools_sources/ > 7.0/tools_sources/templates/ > rsync: connection unexpectedly closed (1892713 bytes read so far) > rsync error: error in rsync protocol data stream (code 12) at io.c(165) > > Does file-list has a size limitation? Or something was not setup right > in this kit directory? I hope you can have any clue/suggestion of what > was > wrong or limitation of rsync. If you have gotten to the point of processing files it is unlikely that the size of the file list is the issue. With the information given my best guess is that something is causing a delay where no data is being sent and the firewall is dropping the session. You might try another -v option to see if you can get more specific info about what file rsync is processing or other possible causes for a stall. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@xxxxxxxxxx Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Next Message by Date: click to view message preview

Problem with the file order....

Hi to all, I've looked at the archives, asked google, and found no solution, so that's why i'm asking this here. I thought that this should be solved in some way, but surprisilngly, it looks like it isn't... I'm running a debian mirror (debian.ludost.net) which has 128k connectivity to the world, and 100mbps to it's clients. The problem is the following: once a while, rsync starts to synchronize the local copy. It firstly syncs the Packages* files, and then the packages themselves. So, if someone is updating their debian in the mean time, and is trying to get a package that wasn't synced, he'll get a file not found error. If there was any way to tell rsync to get some files last, or some files first, or even to reverse the order of the file list, that would solve the problem... Of course, the script that runs rsync can be modified to run rsync twice, once for the packages, and once for the Packages* files, but that means that for every new or removed branch, the script has to be modified again, and that sounds like very bad idea to me... Has anyone had the same problem, and can someone think of any normal solution? signature.asc Description: This is a digitally signed message part -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Previous Message by Thread: click to view message preview

Re: Rsync died on big directories/file-lists

On Tue, Dec 09, 2003 at 01:54:25PM -0500, Vu Huynh wrote: > % rsync --password-file -avz user@host::KIT1 KIT1 --exclude '5.*' > --exclude '6.*' Since you're using compression, be sure to try the compression bugfix: http://www.blorf.net/rsync-2.5.7-compress-fix.patch ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Next Message by Thread: click to view message preview

Problem with the file order....

Hi to all, I've looked at the archives, asked google, and found no solution, so that's why i'm asking this here. I thought that this should be solved in some way, but surprisilngly, it looks like it isn't... I'm running a debian mirror (debian.ludost.net) which has 128k connectivity to the world, and 100mbps to it's clients. The problem is the following: once a while, rsync starts to synchronize the local copy. It firstly syncs the Packages* files, and then the packages themselves. So, if someone is updating their debian in the mean time, and is trying to get a package that wasn't synced, he'll get a file not found error. If there was any way to tell rsync to get some files last, or some files first, or even to reverse the order of the file list, that would solve the problem... Of course, the script that runs rsync can be modified to run rsync twice, once for the packages, and once for the Packages* files, but that means that for every new or removed branch, the script has to be modified again, and that sounds like very bad idea to me... Has anyone had the same problem, and can someone think of any normal solution? signature.asc Description: This is a digitally signed message part -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html