Subject: Re: [Bacula-users] Speed of backups
On Thu, Apr 28, 2011 at 11:41 AM, John Drescher <drescherjm@xxxxxxxxx> wrote:
>> How can I know where's the bottleneck? I'm using an ext4 filesystem.
>> Are these tests useful?
>> [root@qsrpsbk1 ~]# hdparm -t /dev/sda
>> Timing buffered disk reads: 370 MB in 3.01 seconds = 122.89 MB/sec
>> [root@qsrpsbk1 ~]# hdparm -tT /dev/sda
>> Timing cached reads: 3770 MB in 2.00 seconds = 1885.16 MB/sec
>> Timing buffered disk reads: 370 MB in 3.00 seconds = 123.20 MB/sec
> That is expected for a hard drive purchased in 2010 or newer.
>> First I disabled 'signature=SHA1' at my Jobs and I gained little speed
>> of my backup (between 79 and 83 MB/s). Then I enabled Data Spooling
>> (so attribute spooling also is enabled) and my backups became slower
>> (between 36 and 45 MB/s).
> Your benchmark does not measure random or small file performance
> (smaller than a few MB). Any mechanical hard drive will not have > 100
> MB/s for this. SSDs or raid will but not a regular hard drive.
So do you believe these speeds of my backups are normal? I though my
Library tape with LTO-5 tapes could write at 140 MB/s approx. It isn't
possible to achieve higher speeds?
I recently made an additional test modify the buffer size setting of
my storage daemon like this:
Maximum Network Buffer Size = 262144 #65536
Maximum block size = 262144
With these settings, signature=SHA1 disabled, data & attribute
spooling enabled and bacula software compression disabled I'm still
geting slow speeds.
I don't understand why the speed is so variable in each test I run. My
speed keeps being different each time with 32, 36, 45, 54, 67, ..., 80
MB/s when the backup Job is always the same (8 GB of data).
I'm looking at the speed of backups from the final report of the logs like:
Build OS: x86_64-unknown-linux-gnu redhat Enterprise release
Backup Level: Full
Client: "qsrpsfs1" 5.0.3 (04Aug10)
FileSet: "fset-qsrpsfs1test" 2011-04-28 10:06:18
Pool: "FS1Weekly" (From User input)
Catalog: "Default" (From Client resource)
Storage: "TapeLibrary-TS3100" (From Pool resource)
Scheduled time: 28-Apr-2011 11:40:42
Start time: 28-Apr-2011 11:40:46
End time: 28-Apr-2011 11:45:15
Elapsed time: 4 mins 29 secs
FD Files Written: 14,589
SD Files Written: 14,589
FD Bytes Written: 8,712,538,848 (8.712 GB)
SD Bytes Written: 8,714,926,626 (8.714 GB)
Rate: 32388.6 KB/s
Software Compression: None
Volume name(s): L5BA0008
Volume Session Id: 1
Volume Session Time: 1304008831
Last Volume Bytes: 52,326,931,456 (52.32 GB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
That "Rate" is the speed of the backup right?
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
Bacula-users mailing list