[kolla][nova][cinder] Got Gateway-Timeout error on VM evacuation if it has volume attached.
On 8/6/2019 7:18 AM, Mark Goddard wrote:
> We do use a larger timeout for glance-api
> andÂ haproxy_glance_api_server_timeout, both 6h). Perhaps we need
> something similar for cinder-api.
A 6 hour timeout for cinder API calls would be nuts IMO. The thing that
was failing was a volume attachment delete/create from what I recall,
which is the newer version (as of Ocata?) for the old
initialize_connection/terminate_connection APIs. These are synchronous
RPC calls from cinder-api to cinder-volume to do things on the storage
backend and we have seen them take longer than 60 seconds in the gate CI
runs with the lvm driver. I think the investigation normally turned up
lvchange taking over 60 seconds on some concurrent operation locking out
the RPC call which eventually results in the MessagingTimeout from
oslo.messaging. That's unrelated to your gateway timeout from HAProxy
but the point is yeah you likely want to bump up those timeouts since
cinder-api has these synchronous calls to the cinder-volume service. I
just don't think you need to go to 6 hours :). I think the keystoneauth1
default http response timeout is 10 minutes so maybe try that.