osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 4.11.0 -> 4.11.1 problem: Guest VMs losing connection after few minutes


jevgeni,
for sure this should have happened during the upgrade. If you still have
the logs from that day you might find the error there. a spelling error in
name or desription might have happened, for instance. anyway, congrats on
solving it.

On Fri, Jul 20, 2018 at 2:45 PM, Jevgeni Zolotarjov <j.zolotarjov@xxxxxxxxx>
wrote:

> Yes,
>
> But isn't it what is written here
> http://docs.cloudstack.apache.org/projects/cloudstack-
> release-notes/en/4.11.1.0/upgrade/upgrade-4.11.html
> ?
>
> On Fri, Jul 20, 2018 at 3:29 PM Makrand <makrandsanap@xxxxxxxxx> wrote:
>
> > I think you must have tried to register system VM template from template
> > menu from the left side manually. It will be registered as USER only in
> > that case.
> >
> > --
> > Makrand
> >
> >
> > On Fri, Jul 20, 2018 at 5:41 PM, Jevgeni Zolotarjov <
> > j.zolotarjov@xxxxxxxxx>
> > wrote:
> >
> > > Eventually I fixed the problem, without clear understanding of the root
> > > cause.
> > >
> > > I destroyed routerVM, and cloudstack recreated it. But I discovered,
> that
> > > it is version 4.11.0. Not 4.11.1!
> > > I checked and systemVM template for 4.11.1 is registered in cloudstack
> -
> > > all OK.
> > > I noticed however, that its "Type" property is USER, not SYSTEM.
> > >
> > > Then I wanted to delete template for 4.11.0, but cloudstack does not
> > offer
> > > me this option. I can only add new ones.
> > >
> > > So, I ended with manipulations in the DB itself to make template for
> > 4.11.1
> > > the only one in the system and have Type = SYSTEM.
> > >
> > > After that I destroyed again routerVM. It was recreated and it is
> 4.11.1.
> > > And now everything works fine for over an hour already.
> > >
> > > I hope, thats it.
> > >
> > > On Fri, Jul 20, 2018 at 12:10 PM ilya musayev <
> > > ilya.mailing.lists@xxxxxxxxx>
> > > wrote:
> > >
> > > > Have you tried destroying router vm and let CloudStack create new
> one ?
> > > >
> > > > On Fri, Jul 20, 2018 at 1:33 AM Jevgeni Zolotarjov <
> > > j.zolotarjov@xxxxxxxxx
> > > > >
> > > > wrote:
> > > >
> > > > > - an ip-address conflict.
> > > > >   JZ: unlikely, but not impossible. I tried to restart router VM in
> > > > > Network-Guest networks -> defaultGuestNetwork -> VirtualAppliances
> > > > > While rebooting ping to this router VM disappeared. Hence, no other
> > > > device
> > > > > is using the same IP.
> > > > > But!!! when this virtual router started, then network connection to
> > all
> > > > > guest VMs disappeared. So, it must be something with this virtual
> > > router.
> > > > >
> > > > > - flakey hardware being one of
> > > > > -+ if card in the host
> > > > > JZ: higly unlikely
> > > > >
> > > > > -+ a router with bad firmware
> > > > > JZ: also unlikely
> > > > >
> > > > > - of course a strange cofiguration of the software router in you
> host
> > > > might
> > > > > be the issue as well
> > > > > JZ: I didnt do any special configuration. Just used default.
> > > > >
> > > > > by all I know this happening after upgrade sounds like an unhappy
> > > > incident
> > > > > but can't be sure.
> > > > > The iptables restart, was this on the VirtualRouter or on the host,
> > or
> > > > > maybe on the guest? and the restart network?
> > > > >
> > > > > JZ: iptables restart on host machine. (or network restart on host)
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Jul 20, 2018 at 11:14 AM Daan Hoogland <
> > > daan.hoogland@xxxxxxxxx>
> > > > > wrote:
> > > > >
> > > > > > that behaviour sound familiar from a couple of cases:
> > > > > > - an ip-address conflict.
> > > > > > - flakey hardware being one of
> > > > > > -+ if card in the host
> > > > > > -+ a router with bad firmware
> > > > > > - of course a strange cofiguration of the software router in you
> > host
> > > > > might
> > > > > > be the issue as well
> > > > > >
> > > > > > by all I know this happening after upgrade sounds like an unhappy
> > > > > incident
> > > > > > but can't be sure.
> > > > > > The iptables restart, was this on the VirtualRouter or on the
> host,
> > > or
> > > > > > maybe on the guest? and the restart network?
> > > > > >
> > > > > >
> > > > > > On Fri, Jul 20, 2018 at 7:43 AM, Jevgeni Zolotarjov <
> > > > > > j.zolotarjov@xxxxxxxxx>
> > > > > > wrote:
> > > > > >
> > > > > > > I updated cloudstack 4.11.0 -> 4.11.1
> > > > > > >
> > > > > > > Everything went OK during update, but after host reboot guest
> VMs
> > > > lost
> > > > > > > connection after few minutes of normal work.
> > > > > > > I tried restarting network - systemctl restart network.service
> > > > > > > then connection was restored again for few minutes
> > > > > > >
> > > > > > > Finally I could restore connection by restarting iptables -
> > > systemctl
> > > > > > > restart iptables.service
> > > > > > >
> > > > > > > But then again guest VMs lost connection after few minutes of
> > > normal
> > > > > > > operation.
> > > > > > > The time of normal operation can be 5 minutes, but sometimes up
> > to
> > > 40
> > > > > > > minutes.
> > > > > > >
> > > > > > > Please help me to track the root cause and fix it
> > > > > > >
> > > > > > > Host OS - Centos 7.5
> > > > > > > virtualisation - KVM
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Daan
> > > > > >
> > > > >
> > > >
> > >
> >
>



-- 
Daan