osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[placement][ptg] Enabling other projects to continue with placement or get started


On 4/10/19 1:51 PM, Chris Dent wrote:
> On Wed, 10 Apr 2019, Dmitry Tantsur wrote:
> 
>> On 4/10/19 1:25 PM, Chris Dent wrote:
>>> On Wed, 10 Apr 2019, Dmitry Tantsur wrote:
>>>> We would rather have ironic aware that some nodes are reserved for internal 
>>>> bookkeeping.
>>>
>>> Could you query placement for that "bookkeeping"? That's what
>>> placement does.
>>>
>>
>> It means replacing a short database query with a call to a remote resource 
>> with all the associated performance and reliability penalties. Also if someone 
>> creates an allocation in Placement directly, we won't have an associated 
>> ironic allocation, which is fine, but may be confusing for users. E.g. they 
>> see a free node in ironic, but it's actually occupied in Placement.
> 
> This gets at some of the general concepts of "how best to use
> placement" which are still emerging and evolving and one of the
> reasons why I think these threads are useful: We can spend some time
> reflecting on that kind of big topic without feeling like it is
> wasting precious PTG time.

++ and enable more people to participate as well.

> 
>  From what I've been able to discern with the various ways I've used
> placement, it works most easily and effectively when you allow it to
> be the single source of truth for inventory and use of that
> inventory [1]. This suggests that if you want to optionally use
> placement with ironic, then having placement be a one of several
> possible backends to the ironic bookkeeping system might be worth
> considering.

I'm totally okay with it, but it will need many internal changes to ironic, 
including calling to Placement for some popular API calls. I'll try to explain 
the background a bit better now:

Ironic has a notion of instance_uuid. It use to mean "Nova server ID", but now 
is used more freely. We don't impose any restrictions on its value, except that 
it must be a UUID.

Now, an important property of instance_uuid is that it enables cooperative 
locking. It is implemented by making it possible to assign and unset 
instance_uuid, but not change it. So if you're doing

$ openstack baremetal node set --instance-uuid <UUID> <node>

and it succeeds, you know that the <node> is now "reserved" for allocation, 
instance or whatever designated by <UUID>. Fairly primitive, but works. Now, the 
challenge is that we must make sure it works no matter what we invent, because 
it's a critical part of Bare Metal API contract.

Now, instance_uuid is not a scheduler. We've built an API that can find a node 
by resource_class and traits:

$ openstack baremetal allocation create --resource-class baremetal-large --trait 
CUSTOM_OPENCV

You could argue that we should have gone with Placement instead, and you would 
probably be right. But see below about resistance to add and maintain new 
services, and blah-blah. Also the Placement split was in its early phases, and a 
solution was long overdue. My thought was that we could optionally integrate 
with Placement as a backend instead. Anyway, this is a bit of a side discussion.

What I'm worried about is compatibility with the old case of

$ openstack baremetal node set --instance-uuid <UUID> <node>

and listing free nodes with

$ openstack baremetal node list --unassociated

(which ends up filtering by instance_uuid=NULL).

Our allocation API handles it by setting node.instance_uuid = allocation.uuid in 
the same atomic fashion. Thus, it can co-operate with older code, as well as 
Nova (which also does this trick, although after going to Placement first).

If we tell users to go through Placement for reservation, we need to provide 
synchronization between Ironic and Placement somehow, so that

$ openstack baremetal node list --unassociated

does not yield unpredictable results and

$ openstack baremetal node set --instance-uuid <UUID> <node>

ends up checking for a Placement allocation for this node.

> 
> That said, I'm not sure placement is expensive or risky enough to
> not just use (solely) it. Yes, it is another http service and
> database but it is super lightweight; easy to install, scale and
> manage. If the http service is multi-homed, then there are pretty
> much the same network partitioning issues (for talking to the
> database) with or without a placement. You could easily choose to
> co-locate the placement service (web and database) on the same
> host(s) as Ironic.

Resistance to add new services to a standalone deployment can be borderline 
religious at times :(

> 
> Note, I'm not trying to twist your arm anything here, just present
> options. I recognize that achieving a standalone ironic and then
> going and adding a placement to it feels like a step backwards. It
> might help to think of placement as a library that happens to be
> packaged as an HTTP microservice.
> 
> [1] Which, if you're going to have N less than many sources of
> truth, it may as well be _one_ to avoid synchronization problems.
> Before placement came along I thought (and sometimes still do) that
> having a thing which _represents_ a view of the truth (about
> inventory) as placement does is overkill if it is possible for
> things themselves to represent their own truth, and be event/pubsub
> driven about their willingness to accept a workload (instead of
> being told by some authority).
>