[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

CloudStack - KVM / Ceph Performance


We are days away from completing our production CloudStack / Ceph deployment but have run into a small challenge in regards to Ceph performance. As you run a setup similar to ours I was wondering if you’ve seen a similar issue?

 

Our Ceph cluster consists of 60 X 7.2K SAS drive, spread over 5 nodes each with a 960GB Samsung NVMe. Our KVM hypervisors are connected to this nodes via 2 x 10Gbps links, per node.

 

3 centos Cloudstack management servers 4.11.0

4 centos KVM hosts

 

The RAW Ceph performance looks great!

 

CEPH RAW

Average IOPS

Max IOPS

MB/sec

Write 4K

5,885

6,190

22.9907

Read 4K

28,985

35,025

113

Write 4MB

204

219

816

Read 4MB

361

399

1447.51

 

However when we run the tests from within KVM we get significantly reduced performance, particular in writes:

 

VM Running on KVM Hypervisor

Average IOPS

Max IOPS

MB/sec

Write 4K

946.07

1390

3.69

Write 4MB

472

614

116

 

We believe the libvirt/ceph library could be causing the problem with in centos as we don’t see this on a Ubuntu KVM HOST

 

Kind Regards,

Oakford Internet Services

Adam Witwicki | Hosted Systems Specialist

01380 710278 / awitwicki@xxxxxxxxxxxxx

Oakford Internet Services Office: 01380 888088 
10 Prince Maurice Court, Devizes, Wiltshire. SN10 2RT
www.oakfordis.com sales@xxxxxxxxxxxxx

 

Disclaimer Notice:
This email has been sent by Oakford Technology Limited, while we have checked this e-mail and any attachments for viruses, we can not guarantee that they are virus-free. You must therefore take full responsibility for virus checking.
This message and any attachments are confidential and should only be read by those to whom they are addressed. If you are not the intended recipient, please contact us, delete the message from your computer and destroy any copies. Any distribution or copying without our prior permission is prohibited.
Internet communications are not always secure and therefore Oakford Technology Limited does not accept legal responsibility for this message. The recipient is responsible for verifying its authenticity before acting on the contents. Any views or opinions presented are solely those of the author and do not necessarily represent those of Oakford Technology Limited.
Registered address: Oakford Technology Limited, 10 Prince Maurice Court, Devizes, Wiltshire. SN10 2RT.
Registered in England and Wales No. 5971519