[qa][openstackclient] Debugging devstack slowness
On Fri, Jul 26, 2019 at 5:57 PM Clark Boylan <cboylan at sapwetik.org> wrote:
> Today I have been digging into devstack runtime costs to help Donny Davis
> understand why tempest jobs sometimes timeout on the FortNebula cloud. One
> thing I discovered was that the keystone user, group, project, role, and
> domain setup  can take many minutes  (in the examples here almost
> I've rewritten create_keystone_accounts to be a python tool  and get
> the runtime for that subset of setup from ~100s to ~9s . I imagine that
> if we applied this to the other create_X_accounts functions we would see
> similar results.
> I think this is so much faster because we avoid repeated costs in
> openstack client including: python process startup, pkg_resource disk
> scanning to find entrypoints, and needing to convert names to IDs via the
> API every time osc is run. Given my change shows this can be so much
> quicker is there any interest in modifying devstack to be faster here? And
> if so what do we think an appropriate approach would be?
In tripleo, we've also run into the same thing for other actions. While you
can do bulk openstack client actions, it's not the best thing if you
need to create a resource and fetch an ID for a subsequent action. We
ported our post-installation items to python and noticed a dramatic
improvement as well. It might be beneficial to maybe add some caching into
openstackclient so that the startup cost isn't so large every time?
>  https://review.opendev.org/#/c/673108/
> Note the jobs compared above all ran on rax-dfw.
-------------- next part --------------
An HTML attachment was scrubbed...