Re: [Bacula-users] Faster reading of Volumes from
disk



On Feb 17, 2016, at 7:57 AM, Josh Fisher <jfisher@xxxxxxxx> wrote:

On 2/16/2016 6:59 PM, Dan Langille wrote:
On Feb 16, 2016, at 3:21 PM, Dan Langille <dan@xxxxxxxxxxxx> wrote:
I can scp a 5GB Volume from one system to another in about 90 seconds.
Why does that take the SD 8-10 minutes to do the same?
The two systems are on the same 1Gb/s network.  We are copying from HDD local to SSD remote.
I'm running a copy job, from disk on one SD to tape on another SD.  Data spooling is enabled.
The full job can be seen here https://gist.github.com/dlangille/210fd8a8c3418f9004f6 :
15-Feb 13:32 crey-sd JobId 231187: Forward spacing Volume "FullAuto-3531" to file:block 0:216.
15-Feb 13:41 crey-sd JobId 231187: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3531"
15-Feb 13:41 crey-sd JobId 231187: Ready to read from volume "FullAuto-3534" on file device "vDrive-0" (/usr/local/bacula/volumes).
15-Feb 13:41 crey-sd JobId 231187: Forward spacing Volume "FullAuto-3534" to file:block 0:216.
15-Feb 13:51 crey-sd JobId 231187: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3534"
15-Feb 13:51 crey-sd JobId 231187: Ready to read from volume "FullAuto-3582" on file device "vDrive-0" (/usr/local/bacula/volumes).
15-Feb 13:51 crey-sd JobId 231187: Forward spacing Volume "FullAuto-3582" to file:block 0:216.
15-Feb 13:59 crey-sd JobId 231187: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3582"
15-Feb 13:59 crey-sd JobId 231187: Ready to read from volume "FullAuto-3594" on file device "vDrive-0" (/usr/local/bacula/volumes).
15-Feb 13:59 crey-sd JobId 231187: Forward spacing Volume "FullAuto-3594" to file:block 0:218.
Each volume is 5G and it's taking the SD nearly three times longer than scp to transfer.
As a test, I scp'd over all the volumes for one job and timed it:
real    167m15.956s user    117m51.006s sys     23m56.653s
FWIW, these copies were done while a ZFS scrub was underway, so if anything, the potential throughput is higher.
The job above, took nearly 11 hours.  Something is up.
I tried another test run, just now, but cancelled it after the first two Volumes were read; it was going at about the same rate as the full job mentioned above.
What might account for this vast difference?
You mention that data spooling is enabled, but what about attribute spooling? There are a lot of records being added to the catalog for a Copy job.

A valid point.
From http://www.bacula.org/7.4.x-manuals/en/main/Data_Spooling.html
   "When data spooling is enabled, Bacula automatically turns on attribute spooling."
I have confirmed attribute spooling is on, by seeing the files and thinking it was data spooling.. when I didn't have data spooling enabled.
see http://marc.info/?l=bacula-users&m=145548993523222&w=2
I have since turned on Data spooling.
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________
Bacula-users mailing list
Bacula-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/bacula-users



Privacy