[nova][scheduler] - Stack VMs based on RAM
On Wed, 17 Apr 2019 22:02:43 +0300, Georgios Dimitrakakis
<giorgis at acmac.uoc.gr> wrote:
> Thank you both Melanie and Matt for trying to assist me.
> I have double checked the nova.conf at the controller and here is what
> I have (ignored hashed lines and obfuscating sensitive data):
> As you can see I have everything with default values as discussed
> before with Melanie except the filters and the weight that I have
> applied that should lead to VM stacking instead of spreading.
> My case scenario is with two compute hosts (let's call them "cpu1" and
> "cpu2") and when an instance is already placed on "cpu2" I expect the
> next instance to be placed also there. But instead is placed on "cpu1"
> as you can see from the scheduler log that can find here:
> Do you see something strange that I fail to recognize?
Thanks for providing the helpful data. It appears you have set your
nova.conf correctly (this is where your scheduler is running, yes?). I
notice you have duplicated the ram_weight_multiplier setting but that
shouldn't hurt anything.
The relevant scheduler log is this one:
2019-04-17 19:53:07.303 98874 DEBUG nova.scheduler.filter_scheduler
6a4c2e32919e4a6fa5c5d956beb68eef 9f22e9bfa7974e14871d58bbb62242b2 -
default default] Weighed [(cpu1, cpu1) ram: 32153MB disk: 1906688MB
io_ops: 0 instances: 0, (cpu2, cpu2) ram: 30105MB disk: 1886208MB
io_ops: 0 instances: 1] _get_sorted_hosts
and here we see that host 'cpu1' is being weighed ahead of host 'cpu2',
which is the problem. I don't understand this considering the docs say
that setting the ram_weight_multiplier to a negative value should result
in the host with the lesser RAM being weighed higher/first. According to
your log, the opposite is happening -- 'cpu1' with 32153MB RAM is being
weighed higher than 'cpu2' with 30105MB RAM.
Either your ram_weight_multiplier setting is not being picked up or
there's a bug causing weight to be applied with reverse logic?
Can you look at the scheduler debug log when the service first started
up and verify what value of ram_weight_multiplier the service is using?
>> On 4/16/2019 7:03 PM, melanie witt wrote:
>>> To debug further, you should set debug to True in the nova.conf on
>>> your scheduler host and look for which filter is removing the desired
>>> host for the second VM. You can find where to start by looking for a
>>> message like, "Starting with N host(s)". If you have two hosts with
>>> enough RAM, you should see "Starting with 2 host(s)" and then look for
>>> the log message where it says "Filter returned 1 host(s)" and that
>>> will be the filter that is removing the desired host. Once you know
>>> which filter is removing it, you can debug further.
>> If the other host isn't getting filtered out, it could be the
>> weighers that aren't prioritizing the host you expect, but debug logs
>> should dump the weighed hosts as well which might give a clue.