OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Snapshots only on Primary Storage feature


I think reverting the change in 4.11.1 is probably a good idea.

*Will Stevens*
Chief Technology Officer
c 514.826.0190

<https://goo.gl/NYZ8KK>


On Fri, May 18, 2018 at 2:52 PM ilya musayev <ilya.mailing.lists@xxxxxxxxx>
wrote:

> Perhaps bring it back into 4.11.1?
>
> On Fri, May 18, 2018 at 9:28 AM Suresh Kumar Anaparti <
> sureshkumar.anaparti@xxxxxxxxx> wrote:
>
> > Si / Will,
> >
> > That is just FYI, if anyone uses VMware with that flag set to false. I'm
> > neither against the feature nor telling to rip that out.
> >
> > You are correct, the PR 2081 supports KVM and Xen as the volume snapshots
> > are directly supported on them and backup operation is not tightly
> coupled
> > with the create operation.
> >
> > -Suresh
> >
> > On Fri, May 18, 2018 at 7:38 PM, Simon Weller <sweller@xxxxxxx.invalid>
> > wrote:
> >
> > > There are plenty of features in ACS that are particular to a certain
> > > hypervisor (or hypervisor set), including VMware specific items.
> > >
> > > It was never claimed this feature worked across all hypervisors. In
> > > addition to that, the default was to leave the existing functionality
> > > exactly the way it was originally implemented and if a user wished to
> > > change the functionality they could via a global config variable.
> > >
> > > Your original spec for PR 2081 in confluence states that the PR was
> > > targeted towards KVM and Xen, so I'm confused as to why VMware is even
> > > being mentioned here.
> > >
> > >
> > > This is a major feature regression that a number of
> organizations/service
> > > providers are relying on and it wasn't called out when the PR was
> > submitted.
> > >
> > >
> > > ________________________________
> > > From: Will Stevens <wstevens@xxxxxxxxxxxx>
> > > Sent: Friday, May 18, 2018 6:12 AM
> > > To: dev@xxxxxxxxxxxxxxxxxxxxx
> > > Subject: Re: Snapshots only on Primary Storage feature
> > >
> > > Just because it does not work for VMware should not a reason to rip out
> > the
> > > functionality for other hypervisors where it is being used though.
> > >
> > > I know we also have the requirement that snapshots are not
> automatically
> > > replicated to secondary storage, so this feature is useful to us.
> > >
> > > I don't understand the rational for removing the feature just because
> it
> > > does not work on VMware.
> > >
> > > On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, <
> > > sureshkumar.anaparti@xxxxxxxxx> wrote:
> > >
> > > > Si,
> > > >
> > > > The PR# 1697 with the global setting *snapshot.backup.rightafter** -
> > > > false* doesn't
> > > > work for VMware as create snapshot never takes a snapshot in Primary
> > > pool,
> > > > it just returns the snapshot uuid. The backup snapshot does the
> > complete
> > > > job - creates a VM snapshot with the uuid, extracts and exports the
> > > target
> > > > volume to secondary. On demand backup snapshot doesn't work as there
> is
> > > no
> > > > snapshot in primary. Also, there'll be only one entry with Primary
> > store
> > > > role in snapshot_store_ref, which is the latest snapshot taken for
> that
> > > > volume.
> > > >
> > > > -Suresh
> > > >
> > > > On Fri, May 18, 2018 at 1:03 AM, Simon Weller
> <sweller@xxxxxxx.invalid
> > >
> > > > wrote:
> > > >
> > > > > The whole point of the original PR was to optionally disable this
> > > > > functionality.
> > > > >
> > > > > We don't expose views of the backup state to our customers (we have
> > our
> > > > > own customer interfaces) and it's a large waste of space for us to
> be
> > > > > backing up tons of VM images when we have a solid primary storage
> > > > > infrastructure that already has lots of resiliency.
> > > > >
> > > > >
> > > > > I guess we're going to have to revisit this again before we can
> > > consider
> > > > > rebasing on 4.11.
> > > > >
> > > > > ________________________________
> > > > > From: Suresh Kumar Anaparti <sureshkumar.anaparti@xxxxxxxxx>
> > > > > Sent: Thursday, May 17, 2018 2:21 PM
> > > > > To: dev
> > > > > Subject: Re: Snapshots only on Primary Storage feature
> > > > >
> > > > > Hi Si,
> > > > >
> > > > > No. not possible to disable the backup to secondary. It copies the
> > > volume
> > > > > snapshot to secondary in a background thread using asyncBackup
> param
> > > (set
> > > > > to true) and allows other operations during that time.
> > > > >
> > > > > I understand that the backup was on demand when any operations are
> > > > > performed on the snapshot. But, backup during that time may take
> > > > > considerable time (depending on the snapshot size and the network
> > > > > bandwidth), which can result in the job timeout and the User may
> > assume
> > > > > that it is already Backed up based on its state, unless it is
> > > documented.
> > > > >
> > > > > -Suresh
> > > > >
> > > > > On Fri, May 18, 2018 at 12:23 AM, Simon Weller
> > <sweller@xxxxxxx.invalid
> > > >
> > > > > wrote:
> > > > >
> > > > > > Suresh,
> > > > > >
> > > > > >
> > > > > > With this new merged  PR, is it possible to disable the backup to
> > > > > > secondary completely? I can't tell from the reference spec and
> > we're
> > > > not
> > > > > on
> > > > > > a 4.10/4.11 base yet.
> > > > > >
> > > > > > For the record, in the instances where a volume or template from
> > > > snapshot
> > > > > > was required, the backup image was copied on demand to secondary.
> > > > > >
> > > > > > In an ideal world, secondary storage wouldn't even be involved in
> > > most
> > > > of
> > > > > > these options, instead using the native clone features of the
> > > > underlying
> > > > > > storage.
> > > > > >
> > > > > >
> > > > > > - Si
> > > > > >
> > > > > > ________________________________
> > > > > > From: Suresh Kumar Anaparti <sureshkumar.anaparti@xxxxxxxxx>
> > > > > > Sent: Thursday, May 17, 2018 1:37 PM
> > > > > > To: dev@xxxxxxxxxxxxxxxxxxxxx
> > > > > > Cc: Nathan Johnson
> > > > > > Subject: Re: Snapshots only on Primary Storage feature
> > > > > >
> > > > > > Hi Glen / Si,
> > > > > >
> > > > > > In PR# 1697, the global setting *snapshot.backup.rightafter* if
> set
> > > to
> > > > > > true, it'll be the default behaviour and snapshot is copied to
> the
> > > > > > secondary storage. If set to false, then the snapshot state
> > > transitions
> > > > > are
> > > > > > mocked and Snapshot would be in BackedUp state even though it is
> > not
> > > > > really
> > > > > > in Secondary storage, which doesn't make sense. Also, that will
> > > enable
> > > > to
> > > > > > create a volume or template from the snapshot, which will
> obviously
> > > > fail.
> > > > > >
> > > > > > This behavior was changed with the PR
> > > > > > https://github.com/apache/cloudstack/pull/2081. There is a clear
> > > > > > separation
> > > > > > of create and backup volume snapshot operations. The global
> setting
> > > > > > *snapshot.backup.rightafter* has been removed in PR# 2081.
> > > > > >
> > > > > > -Suresh
> > > > > >
> > > > > > On Thu, May 17, 2018 at 8:40 PM, Simon Weller
> > > <sweller@xxxxxxx.invalid
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Glen,
> > > > > > >
> > > > > > >
> > > > > > > This feature was implemented in 4.9 by my colleague Nathan
> > Johnson.
> > > > > You
> > > > > > > enable it by changing the global setting
> > > snapshot.backup.rightafter
> > > > to
> > > > > > > false.
> > > > > > >
> > > > > > >
> > > > > > > The PR is reference here: https://github.com/apache/
> > > > > cloudstack/pull/1697
> > > > > > >
> > > > > > >
> > > > > > > We have the exact same use case as you, as we also use Ceph.
> > > > > > >
> > > > > > >
> > > > > > > - Si
> > > > > > >
> > > > > > >
> > > > > > > ________________________________
> > > > > > > From: Glen Baars <glen@xxxxxxxxxxxxxxxxxxxxxx>
> > > > > > > Sent: Thursday, May 17, 2018 9:46 AM
> > > > > > > To: dev@xxxxxxxxxxxxxxxxxxxxx
> > > > > > > Subject: Snapshots only on Primary Storage feature
> > > > > > >
> > > > > > >
> > > > > > > Hello Devs,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I have been thinking about a feature request and want to see
> what
> > > > > people
> > > > > > > think about the use case.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We use KVM + Ceph RBD as storage.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Currently, when a client takes a snapshot, Cloudstack takes a
> > Ceph
> > > > > > > snapshot and then uses qemu-img to export to secondary storage.
> > > This
> > > > > > > creates a full backup of the server. Clients want to use this
> as
> > a
> > > > > daily
> > > > > > > snapshot and it isn’t feasible due to the space requirements.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We would like create the snapshot only on primary storage. It
> is
> > > > > > > replicated offsite and fault tolerant. I can see that the
> > download
> > > > > > snapshot
> > > > > > > and create template features may be an issue.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I have seen the below features in the recent releases and
> > wondered
> > > if
> > > > > > this
> > > > > > > was the direction that the development was going.
> > > > > > >
> > > > > > > Separation of volume snapshot creation on primary storage and
> > > backing
> > > > > > > operation on secondary storage.
> > > > > > >
> > > > > > > Bypass secondary storage template copy/transfer for KVM.
> > > > > > >
> > > > > > > Kind regards,
> > > > > > >
> > > > > > > Glen Baars
> > > > > > >
> > > > > > > BackOnline Manager
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > T  1300 733 328 / +61 8 6102 3276
> > > > > > >
> > > > > > > NZ +64 9280 3561
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > www.timg.com<http://www.timg.com/>
> > > > > > >
> > > > > > >  [http://images.dbonline.com.au/images/fb.png]  Facebook<
> > > > > > > https://www.facebook.com/TheInformationManagementGroup>  [
> > > > > > > http://images.dbonline.com.au/images/li.png] LinkedIn<
> > > > > > http://www.linkedin.
> > > > > > > com/company/the-information-management-group?trk=hb_tab_
> > > > > > compy_id_2724246>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Watch a short video about what we do!<https://www.youtube.com/
> > > > > > > watch?v=scLGLwSIFQc>
> > > > > > >
> > > > > > > [http://images.dbonline.com.au/images/timgv3.jpg]<https://
> > > > > goo.gl/eAHLO7>
> > > > > > >
> > > > > > > This e-mail may contain confidential and/or privileged
> > > information.If
> > > > > you
> > > > > > > are not the intended recipient (or have received this e-mail in
> > > > error)
> > > > > > > please notify the sender immediately and destroy this e-mail.
> Any
> > > > > > > unauthorized copying, disclosure or distribution of the
> material
> > in
> > > > > this
> > > > > > > e-mail is strictly forbidden.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > This e-mail is intended solely for the benefit of the
> > addressee(s)
> > > > and
> > > > > > any
> > > > > > > other named recipient. It is confidential and may contain
> legally
> > > > > > > privileged or confidential information. If you are not the
> > > recipient,
> > > > > any
> > > > > > > use, distribution, disclosure or copying of this e-mail is
> > > > prohibited.
> > > > > > The
> > > > > > > confidentiality and legal privilege attached to this
> > communication
> > > is
> > > > > not
> > > > > > > waived or lost by reason of the mistaken transmission or
> delivery
> > > to
> > > > > you.
> > > > > > > If you have received this e-mail in error, please notify us
> > > > > immediately.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>