[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: advice needed: snapshot DB/table cleanup


Hi Daan, thx for the reply,

so yes I would not touch SF just for sake of magic, whatever - otherwise I
can remove them in same fashion (.

the second "is this the same" - its not -first SQL -  I delete from
snapshot_store_ref where the template is either DESTROYED or has REMOVED
date set in the main "snapshots" table. then later I just make sure (2nd
SQL) in the main "snapshot" table that if either the REMOVED or DESTROYED
is set - that I also set the missing value :) (previously all store_ref is
gone because of 1st SQL...)

As for the garbage, this has been happening from 4.5 up to now 4.8 for good
knows what reason - you remove something, and either status is set to
DESTROYED with no removal date (mostly its only for snapsbots, I don't
recall seen it on other resources) or removed date is set but state is
still READY (I actually just recently seen this and only on snaposhots -
can't be sure if this is because of ACS, or because of someone changing DB
(in case of snaps are in ERROR or ALLOCATED state - then you simply have to
alter DB, no way to cleanup via API). When you deal with CEPH and
long-running snaphosts than different gremlins can happen from time to time
- my experience at least...


Hope that makes sense (my answers)

Cheers
ANdrija

On Fri, 17 Aug 2018 at 16:40, Daan Hoogland <daan.hoogland@xxxxxxxxx> wrote:

> andrija,
>
> On Fri, Aug 17, 2018 at 11:23 AM, Andrija Panic <andrija.panic@xxxxxxxxx>
> wrote:
>
> > HI guys, hi Mike.T.,
> >
> > we have removed all NFS and CEPH storages, and are now purely running on
> > SolidFire (KVM).
> >
> > Now I want to do serious snapshot cleanup (for reason explained at the
> end
> > of email) - since "snapshots" and "snapshot_store_ref" tables are a
> > complete mess (i.e. snapshot is destroyed with/without "removed" date,
> and
> > then there are still references in snapshot_store_ref to these fully/half
> > destroyed snapshots...)
> >
> > I would like to ask for a tip - based on my common sense and experience,
> I
> > was thinking on doing something like following:
> >
> > SQL:
> > delete from snapshot_store_ref where snapshot_id in (select id from
> > snapshots where status="destroyed" or removed is NOT NULL and min_iops is
> > NULL)
> >
> why the do you want to keep the solidfire snapshots when removed?
>
>
>
> >
> > This last "min_iops is NULL" is identifier for snaps that are NOT on
> > SolidFire - I would not touch SF snapshots) - i.e. all snapshots that are
> > created from SF volumes have min_iops and max_iops values set - so I just
> > exclude them here....
> >
> > - So - above I want to remove all references for snapshots that are
> > fully/semi destroyed (status=destroyed but no removed date - or other way
> > around - those that have "removed" date but status=Ready.....)
> >
> isn't this the same as below?
>
>
>
> >
> > Then I was also thinking does it make sense, to also set (in "snapshots"
> > table) status=Destroyed where removed is NOT NULL and other way around -
> > set removed date where status=Destroyed.
> >
> isn't this the same as above?
>
>
> Also when having cleaned all snapshots that are not for solidfire I would
> first do a check as your mess should be largely cleaned by then.
> and if a few snapshots still jump out better investigate those to see if
> you can find any root cause for the failure.
>
>
> > Sorry for long question - but I had issues with some snaps referencing
> CEPH
> > (and we removed CEPH/NFS from ACS GUI) - i.e. client was unable to list
> > snaps for his account, because some volumes had snaps that were
> referencing
> > CEPH (though they are migrated to SF or deleted)...
> >
> > Thanks a lot
> >
> hope my loose hipshots help,
>
>
>
> >
> > Andrija
> >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
>
>
> --
> Daan
>


-- 

Andrija Panić