Subject: Re: [Bacula-users] Speed of backups

> From: Jason Voorhees <jvoorhees1@xxxxxxxxx>
>> >
>> > to get the maximum speed with your LTO-5 drive you should enable data
>> > spooling and change the "Maximum File Size" parameter. The spool disk
>> > must be a fast one, especially if you want to run concurrent jobs.
>> > Forget hdparm as benchmark, use bonnie++, tiobench, iozone.
>> >
>> > Then after after you have enabled spooling, restarted the SD, start
>> > the backup. Look at the log file for "Despooling elapsed time". This
>> > will show you how fast the spool file can be written to tape.
>> >
>> > The backup time will increase overall because the data will first be
>> > written to disk and then to tape, but at least you eliminate the
>> > network and the data source (server) as bottleneck.
>> >
>> > With spooling enabled and LTO-4 drives I get up 100 - 140 MB/s.
>> >
>> > Ralf
>> >
> Hi, thanks for your help.
> I think I was confusing some terms. The speed I reported was the total
> elapsed time that my backup took. But now according to your comments I
> got this from my logs:
> With spooling enabled:
> - Job write elapsed time: 102 MB/s average
> - Despooling elapsed time: 84 MB/s average
> Without spooling enabled:
> - Job write elapsed time: 68 MB/s average
> These are averages obtained from a group of 5 or more jobs of each
> case (with and without spooling). So I can see that with spooling
> enabled the process of writing to tape get higher speeds than
> copy-from-fd/write-to-tape without spooling enabled.
> Now the question is, why am I getting so low despooling speeds if I
> use LTO-5 tapes? Shouldn't I have higher speeds than you with LTO-4
> tapes?
> I think the write speed of my tape it isn't the best. Some of my tests
> (all with spooling enabled) were done with "Buffer & Block size"
> settings at my storage daemon, but other tests were done without those
> settings, so I didn't see any appreciable enhancement of performance
> changing the value of block sizes.
> What can I do?

Here's some suggestions from me...

1. In another message I think you said you were tweaking the Maximum
Network Buffer Size... don't mess with that.

2. The other parameter you tried to change was Maximum Block Size. You
should make that a lot bigger-- like 1048576 (1MB) or even twice that.

3. Your catalog and/or temp space for the catalog should also be on a high
performance disk. So ideally you'd have catalog on one raid-1 set, temp
space for your db server on another, and spooling on something else that's
raid-0. If you have to combine any of this, combine the temp and catalog.
Note that catalog temp space is not necessarily the same as /tmp.

4. As Ralf said, use a bigger Max File Size. He uses 5MB. The downside to
using a larger number is that restore seeks might take longer but the
difference between 1MB and 5MB will probably be in the few seconds range.
The upside is that the tape will spend less time writing file marks.

5. Don't turn off the File hash algorithm. If your CPU is not quick enough
to process data for a LTO-5 drive you'll be shoe-shining the tape anyway.
Also don't mess with hardware compression default values (let the tape
drive do it's thing there). Don't do software compression or encryption,
at least until you get the parameters worked out. As far as CPU, you need
something relatively beefy like a high clock speed (2.5GHz or more) Core2
or better yet something like the Core i-series. You need at least two
cores or even two full cpu's so that the other processes needed by your OS
can be handled separately instead of by interrupting the data being sent to
the tape drive. I think the next version of bacula due out soon will have
some performance improvements, but to send > 150MB/sec through a computer
and still do other things is still going to take substantial processing
power. I don't think we're at the point where Bacula can use more than one
CPU core in a single storage daemon, but maybe we'll be at that point in
the not so distant future.

WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
Bacula-users mailing list