OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 4.11.0 -> 4.11.1 problem: Guest VMs losing connection after few minutes


Eventually I fixed the problem, without clear understanding of the root
cause.

I destroyed routerVM, and cloudstack recreated it. But I discovered, that
it is version 4.11.0. Not 4.11.1!
I checked and systemVM template for 4.11.1 is registered in cloudstack -
all OK.
I noticed however, that its "Type" property is USER, not SYSTEM.

Then I wanted to delete template for 4.11.0, but cloudstack does not offer
me this option. I can only add new ones.

So, I ended with manipulations in the DB itself to make template for 4.11.1
the only one in the system and have Type = SYSTEM.

After that I destroyed again routerVM. It was recreated and it is 4.11.1.
And now everything works fine for over an hour already.

I hope, thats it.

On Fri, Jul 20, 2018 at 12:10 PM ilya musayev <ilya.mailing.lists@xxxxxxxxx>
wrote:

> Have you tried destroying router vm and let CloudStack create new one ?
>
> On Fri, Jul 20, 2018 at 1:33 AM Jevgeni Zolotarjov <j.zolotarjov@xxxxxxxxx
> >
> wrote:
>
> > - an ip-address conflict.
> >   JZ: unlikely, but not impossible. I tried to restart router VM in
> > Network-Guest networks -> defaultGuestNetwork -> VirtualAppliances
> > While rebooting ping to this router VM disappeared. Hence, no other
> device
> > is using the same IP.
> > But!!! when this virtual router started, then network connection to all
> > guest VMs disappeared. So, it must be something with this virtual router.
> >
> > - flakey hardware being one of
> > -+ if card in the host
> > JZ: higly unlikely
> >
> > -+ a router with bad firmware
> > JZ: also unlikely
> >
> > - of course a strange cofiguration of the software router in you host
> might
> > be the issue as well
> > JZ: I didnt do any special configuration. Just used default.
> >
> > by all I know this happening after upgrade sounds like an unhappy
> incident
> > but can't be sure.
> > The iptables restart, was this on the VirtualRouter or on the host, or
> > maybe on the guest? and the restart network?
> >
> > JZ: iptables restart on host machine. (or network restart on host)
> >
> >
> >
> > On Fri, Jul 20, 2018 at 11:14 AM Daan Hoogland <daan.hoogland@xxxxxxxxx>
> > wrote:
> >
> > > that behaviour sound familiar from a couple of cases:
> > > - an ip-address conflict.
> > > - flakey hardware being one of
> > > -+ if card in the host
> > > -+ a router with bad firmware
> > > - of course a strange cofiguration of the software router in you host
> > might
> > > be the issue as well
> > >
> > > by all I know this happening after upgrade sounds like an unhappy
> > incident
> > > but can't be sure.
> > > The iptables restart, was this on the VirtualRouter or on the host, or
> > > maybe on the guest? and the restart network?
> > >
> > >
> > > On Fri, Jul 20, 2018 at 7:43 AM, Jevgeni Zolotarjov <
> > > j.zolotarjov@xxxxxxxxx>
> > > wrote:
> > >
> > > > I updated cloudstack 4.11.0 -> 4.11.1
> > > >
> > > > Everything went OK during update, but after host reboot guest VMs
> lost
> > > > connection after few minutes of normal work.
> > > > I tried restarting network - systemctl restart network.service
> > > > then connection was restored again for few minutes
> > > >
> > > > Finally I could restore connection by restarting iptables - systemctl
> > > > restart iptables.service
> > > >
> > > > But then again guest VMs lost connection after few minutes of normal
> > > > operation.
> > > > The time of normal operation can be 5 minutes, but sometimes up to 40
> > > > minutes.
> > > >
> > > > Please help me to track the root cause and fix it
> > > >
> > > > Host OS - Centos 7.5
> > > > virtualisation - KVM
> > > >
> > >
> > >
> > >
> > > --
> > > Daan
> > >
> >
>