osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[osc] Very slow openstack commands in Stein Devstack running on VM


On Sun, 2019-07-28 at 13:37 +0900, Bernd Bausch wrote:
> After a little bit of self-education, I found that 
> /proc/sys/kernel/random/entropy_avail was below 100, which is probably a 
> good explanation for my problem. A good value would have four digits, it 
> seems.
> 
> My Devstack host runs Ubuntu 18, and I am now trying the rngd-tools5 
> package to improve the entropy situation. Since this is a test system, I 
> am not so concerned about random number quality.
> 
> In short, the problem seems to be solved, but if anybody has additional 
> comments, I would be interested to hear them.
am i have not encountered this on my own ububtu 18.04 devstack vms but
i tend to enable the RNG device whcich effectivly presnets the host /dev/urandom
to the guests so perhaps that helps. if you are virtulaising with kvm/libvit
i would suggest adding the RNG device and seeing if that has a similar effect.

we had also condidered change openstack default at one point to make it so that
RNG devices would be added by default in the futre rather im not sure if we ever
did anything in that regard but if it can have that much of an impact on things
like osc it proably would be a performace imporvment for many applicaitons that use
openssl and need entropy.
> 
> Bernd
> 
> On 7/28/2019 1:06 PM, Bernd Bausch wrote:
> > 
> > I run Stein Devstack in a virtual machine. Occasionally, openstack 
> > client commands take minutes to complete, which didn't happen at 
> > earlier releases, and which is rather annoying. The delay seems to 
> > take place at an early stage: Even when I forget to set the OS_* 
> > environment, it takes minutes until I am told that the request 
> > requires authentication.
> > 
> > Interrupting the command, the stack trace confirms this (as a side 
> > remark, it also takes a long while until my ^C is processed):
> > 
> > Traceback (most recent call last):
> >   File "/usr/local/bin/openstack", line 7, in <module>
> >     from openstackclient.shell import main
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/openstackclient/shell.py", 
> > line 22, in <module>
> >     from osc_lib.api import auth
> >   File "/usr/local/lib/python2.7/dist-packages/osc_lib/api/auth.py", 
> > line 19, in <module>
> >     from keystoneauth1.loading import base
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/__init__.py", 
> > line 13, in <module>
> >     from keystoneauth1.loading import adapter
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/adapter.py", 
> > line 13, in <module>
> >     from keystoneauth1 import adapter
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", 
> > line 17, in <module>
> >     from keystoneauth1 import session
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", 
> > line 25, in <module>
> >     import requests
> >   File "/usr/local/lib/python2.7/dist-packages/requests/__init__.py", 
> > line 95, in <module>
> >     from urllib3.contrib import pyopenssl
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", 
> > line 48, in <module>
> >     from cryptography.hazmat.backends.openssl import backend as 
> > openssl_backend
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/__init__.py", 
> > line 7, in <module>
> >     from cryptography.hazmat.backends.openssl.backend import backend
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", 
> > line 2419, in <module>
> >     backend = Backend()
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", 
> > line 119, in __init__
> >     self.activate_osrandom_engine()
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", 
> > line 163, in activate_osrandom_engine
> >     with self._get_osurandom_engine() as e:
> >   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
> >     return self.gen.next()
> >   File 
> > "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", 
> > line 146, in _get_osurandom_engine
> >     res = self._lib.ENGINE_init(e)
> > KeyboardInterrupt
> > 
> > My wild speculation is that the delay has something to do with 
> > obtaining a random number, which can be a problem on virtual machines 
> > due to their lack of entropy. My knowledge in this area is paper-thin, 
> > though. And I wonder what happens to all the Devstacks used in CI?
> > 
> > Thus my questions: *Is the delay really caused by my running the cloud 
> > on a VM? What can I do to improve this?*
> > 
> > Thanks,
> > 
> > Bernd
> >