OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PROPOSE] Backup and Recovery Framework


Rafael,

I would agree with that as well as how Quadric spins up a ABD at time of
backup. I have tried both Alike and Unitrends (old PHD virtual) and ACS
removed the device before it works, but that was back on v4.3.



On Thu, Jun 7, 2018 at 11:03 AM, Rafael Weingärtner <
rafaelweingartner@xxxxxxxxx> wrote:

> I believe the framework is going to be agnostic, but we need to implement
> "driver" to deal with different vendors API, right?
>
> On Thu, Jun 7, 2018 at 5:01 PM, Todd Pigram <todd@xxxxxxxxxxxxxx> wrote:
>
> > Nicolas,
> >
> > Given you Veeam example, this framework is backup vendor independent? So
> I
> > could use Veeam for vSphere or Hyper-V or use Quadric (Alike) for
> XenServer
> > or Hyper-V. This would sure beat the way I am doing it now, inside the
> VM.
> >
> > I will be following this closely.
> >
> >
> >
> > On Thu, Jun 7, 2018 at 10:13 AM, Rafael Weingärtner <
> > rafaelweingartner@xxxxxxxxx> wrote:
> >
> > > Ok, now I start to understand.  So, we will be handling snapshots and
> > they
> > > will be sent to a third-party system.
> > >
> > > Now let’s proceed. If I configure this “feature” with external=false.
> > What
> > > does it do? Nothing? I mean, what is the purpose of this “external”
> > > parameter?
> > >
> > >
> > > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> > > Nicolas.Vazquez@xxxxxxxxxxxxx> wrote:
> > >
> > > > Its format may depend on the provider, but under the hood it is a
> > > snapshot
> > > > which can be full or incremental and it is copied and stored in a
> > backup
> > > > repository in one of the backup servers. For example, if you use
> Veeam,
> > > you
> > > > will find your backups stored as .vbk (for full backups) and .vib
> (for
> > > > incremental backups). It is the task of the backup provider to create
> > and
> > > > move these files to the backup repository.
> > > >
> > > > ________________________________
> > > > From: Rafael Weingärtner <rafaelweingartner@xxxxxxxxx>
> > > > Sent: Thursday, June 7, 2018 10:39:35 AM
> > > > To: dev
> > > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > > >
> > > > Ok, there are a lot of things, which are kind of blurry to me. So,
> > let’s
> > > > try to go step by step.
> > > >
> > > > Let me ask again. What is a backup of the VM? You mention
> > restore-point.
> > > > What is it? Is it a snapshot in the hypervisor (complete/partial
> > > snapshot)?
> > > > If it is not a snapshot,  what is it then? Are you installing some
> > agent
> > > in
> > > > the VM and then copying/mirroring the files in the VM’s file system?
> > > >
> > > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> > > > Nicolas.Vazquez@xxxxxxxxxxxxx> wrote:
> > > >
> > > > > Hi Daniel and Rafael,
> > > > >
> > > > >
> > > > > I have updated the diagram picture on the FS, please let me know if
> > the
> > > > > problem persists.
> > > > >
> > > > >
> > > > > @<mailto:daniel.herrmann@xxxxxxxxxxxxxxxx>Daniel you are right
> about
> > > > > restoring VMs, please let me update the FS. Regarding your
> question,
> > we
> > > > > have considered the backups as a first class citizens, meaning
> that a
> > > > > backup would continue existing unless its originating VM has been
> > > > > removed/expunged. This way, listBackups API method would list them
> in
> > > > order
> > > > > to allow a user to restore a deleted VM from a backup.
> > > > >
> > > > >
> > > > > @Rafael the idea of the Dummy provider is simply returning the
> > expected
> > > > > entities from a backup provider. For example, the BackupManager
> could
> > > ask
> > > > > the BackupProvider to list its backups and would expect a list of
> > > > backups.
> > > > > As the Dummy Provider implements the BackupProvider interface, it
> has
> > > its
> > > > > own implementation of listBackups() method and simply returns some
> > mock
> > > > > entities to the Backup Manager. It does not store any backup, it
> > simply
> > > > > creates them in execution time and returns them.
> > > > >
> > > > >
> > > > > From the CloudStack point of view, Backups are stored externally,
> on
> > > the
> > > > > Backup and Recovery provider servers and are handled by them. We
> keep
> > > > track
> > > > > of the existing backups in the Backup provider. Implementation may
> > > depend
> > > > > on the Backup and Recovery provider, would consist on snapshots,
> but
> > > are
> > > > > handled by the provider and not by CloudStack.
> > > > >
> > > > >
> > > > > Backup policies are created on the Backup and Recovery provider and
> > > > > provide retention options and periods as well as backup times. For
> > > > example,
> > > > > the provider may expose Golden, Silver and Bronze policies and have
> > > > > different options for each of them.
> > > > >
> > > > >
> > > > > The 'external' parameter on the listBackupPolicies API method would
> > > only
> > > > > ask the Backup and Recovery provider to list their policies if it
> is
> > > set
> > > > to
> > > > > true. If it is not set, or set to false, it just simply returns the
> > > > > existing mapped policies in CloudStack. Initially, CloudStack does
> > not
> > > > > store any backup policy, those should be imported/mapped by the
> > > > > 'importBackupPolicy' API method.
> > > > >
> > > > >
> > > > > Finally, we consider backups as restore points. As mentioned above,
> > > > > backups are stored on the Backup and Recovery provider servers, so
> we
> > > > keep
> > > > > a mapping between the CloudStack entity and the backup, which can
> be
> > > used
> > > > > to restore VMs.
> > > > >
> > > > >
> > > > > Please let me know if anything.
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > > Nicolas Vazquez
> > > > >
> > > > > ________________________________
> > > > > From: Rafael Weingärtner <rafaelweingartner@xxxxxxxxx>
> > > > > Sent: Thursday, June 7, 2018 6:55:57 AM
> > > > > To: dev
> > > > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > > > >
> > > > > Hi Nicolas,
> > > > >
> > > > > The idea is great. I have some questions though.
> > > > >
> > > > > Before getting to the questions, your pictures are broken, or at
> > least
> > > > they
> > > > > seem broken to me.
> > > > >
> > > > > How would the “dummy” backup provider work? I mean, what does it
> do?
> > > > where
> > > > > does it storage the "backup"?
> > > > > What is the difference from this Backup to a Snapshot? Is it simply
> > > > storing
> > > > > a snapshot outside the cloud structure?
> > > > > How would these “backup policies” look like? I mean, what can I
> > > > configure?
> > > > > Regarding that parameter “external” of your backup policy, how
> would
> > it
> > > > > work if we set it to false?
> > > > > What are you defining as a backup (what do you mean by backup in
> the
> > > > > context of this feature?)? I mean, is it simply a snapshot file
> that
> > > you
> > > > > store somewhere else?
> > > > >
> > > > >
> > > > > Nicolas.Vazquez@xxxxxxxxxxxxx
> > > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > > ,
> > > > > @shapeblue
> > > > >
> > > > >
> > > > >
> > > >
> > > > Nicolas.Vazquez@xxxxxxxxxxxxx
> > > > www.shapeblue.com
> > > > ,
> > > > @shapeblue
> > > >
> > > >
> > > >
> > > > > On Thu, Jun 7, 2018 at 6:05 AM, <daniel.herrmann@xxxxxxxxxxxxxxxx>
> > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > +1 for this feature request, thank you for bringing this to life!
> > > > Volume
> > > > > > snapshots are what our users currently use as backup solution,
> > which
> > > > has
> > > > > > some inherent problems such as handling large volumes and the
> > > handling
> > > > of
> > > > > > multiple volumes (consistency).
> > > > > >
> > > > > > One thought: when restoring a VM, I think there are two cases to
> > > > > consider:
> > > > > >
> > > > > > - The VM is still in CS and the user just wants to restore an old
> > > > version
> > > > > > of the VM
> > > > > > - The VM has already been deleted and the user wants to restore
> the
> > > > > > deleted VM
> > > > > >
> > > > > > As listBackups is described as " listBackups API method. List
> > > existing
> > > > > > backups for a VM". Is the second use case supported at all?
> > > > > >
> > > > > > I like the approach not to define the backup policies in CS but
> in
> > > > > > whatever tool is used in the end, as otherwise a lot of features
> > > might
> > > > be
> > > > > > lost. We as services providers need to create them in the first
> > > place,
> > > > > but
> > > > > > I think this is the better solution.
> > > > > >
> > > > > > Regards
> > > > > > Daniel
> > > > > >
> > > > > > On 06.06.18, 14:43, "Nicolas Vazquez" <
> > > Nicolas.Vazquez@xxxxxxxxxxxxx>
> > > > > > wrote:
> > > > > >
> > > > > >     Hi all,
> > > > > >
> > > > > >
> > > > > >     We would like to introduce a new framework into CloudStack
> that
> > > > will
> > > > > > allow adding Backup and Recovery providers as plugins. This
> > framework
> > > > > goal
> > > > > > is allowing users to backup their guest VMs for recovery
> purposes.
> > > > > >
> > > > > >
> > > > > >     Please find the FS on this link: https://cwiki.apache.org/
> > > > > > confluence/display/CLOUDSTACK/Backup+and+Recovery+Framework
> > > > > >
> > > > > >
> > > > > >     Regards,
> > > > > >
> > > > > >     Nicolas Vazquez
> > > > > >
> > > > > >     Nicolas.Vazquez@xxxxxxxxxxxxx
> > > > > >     www.shapeblue.com<http://www.shapeblue.com>
> > > > > >     ,
> > > > > >     @shapeblue
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Rafael Weingärtner
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Rafael Weingärtner
> > > >
> > >
> > >
> > >
> > > --
> > > Rafael Weingärtner
> > >
> >
> >
> >
> > --
> >
> >
> > Todd Pigram
> > http://about.me/ToddPigram
> > www.linkedin.com/in/toddpigram/
> > @pigram86 on twitter
> > https://plus.google.com/+ToddPigram86
> > Mobile - 216-224-5769
> >
> > PGP Public Key
> > <http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB>
> >
>
>
>
> --
> Rafael Weingärtner
>



-- 


Todd Pigram
http://about.me/ToddPigram
www.linkedin.com/in/toddpigram/
@pigram86 on twitter
https://plus.google.com/+ToddPigram86
Mobile - 216-224-5769

PGP Public Key
<http://pgp.mit.edu/pks/lookup?op=get&search=0x96B7B0F0C55933BB>