[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[kolla][nova][cinder] Got Gateway-Timeout error on VM evacuation if it has volume attached.

On Tue, 6 Aug 2019 at 16:33, Matt Riedemann <mriedemos at gmail.com> wrote:

> On 8/6/2019 7:18 AM, Mark Goddard wrote:
> > We do use a larger timeout for glance-api
> > (haproxy_glance_api_client_timeout
> > and haproxy_glance_api_server_timeout, both 6h). Perhaps we need
> > something similar for cinder-api.
> A 6 hour timeout for cinder API calls would be nuts IMO. The thing that
> was failing was a volume attachment delete/create from what I recall,
> which is the newer version (as of Ocata?) for the old
> initialize_connection/terminate_connection APIs. These are synchronous
> RPC calls from cinder-api to cinder-volume to do things on the storage
> backend and we have seen them take longer than 60 seconds in the gate CI
> runs with the lvm driver. I think the investigation normally turned up
> lvchange taking over 60 seconds on some concurrent operation locking out
> the RPC call which eventually results in the MessagingTimeout from
> oslo.messaging. That's unrelated to your gateway timeout from HAProxy
> but the point is yeah you likely want to bump up those timeouts since
> cinder-api has these synchronous calls to the cinder-volume service. I
> just don't think you need to go to 6 hours :). I think the keystoneauth1
> default http response timeout is 10 minutes so maybe try that.
Yeah, wasn't advocating for 6 hours - just showing which knobs are
available :)

> --
> Thanks,
> Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190806/fc6f3d9d/attachment.html>