OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: rolling version upgrade, upgradesstables, and vulnerability window


Thank you very much. I couldn't find any definitive answer on that on the list or stackoverflow.

It's clear that the safest for a prod cluster is rolling version upgrade of the binary, then the upgradesstables. 

I will strongly consider cstar for the upgradesstables


On Tue, Oct 30, 2018 at 10:39 AM Alexander Dejanovski <alex@xxxxxxxxxxxxxxxxx> wrote:
Yes, as the new version can read both the old and the new sstables format.

Restrictions only apply when the cluster is in mixed versions.

On Tue, Oct 30, 2018 at 4:37 PM Carl Mueller <carl.mueller@xxxxxxxxxxxxxxx.invalid> wrote:
But the topology change restrictions are only in place while there are heterogenous versions in the cluster? All the nodes at the upgraded version with "degraded" sstables does NOT preclude topology changes or node replacement/addition?


On Tue, Oct 30, 2018 at 10:33 AM Jeff Jirsa <jjirsa@xxxxxxxxx> wrote:
Wait for 3.11.4 to be cut

I also vote for doing all the binary bounces and upgradesstables after the fact, largely because normal writes/compactions are going to naturally start upgrading sstables anyway, and there are some hard restrictions on mixed mode (e.g. schema changes won’t cross version) that can be far more impactful.



--
Jeff Jirsa


> On Oct 30, 2018, at 8:21 AM, Carl Mueller <carl.mueller@xxxxxxxxxxxxxxx.INVALID> wrote:
>
> We are about to finally embark on some version upgrades for lots of clusters, 2.1.x and 2.2.x targetting eventually 3.11.x
>
> I have seen recipes that do the full binary upgrade + upgrade sstables for 1 node before moving forward, while I've seen a 2016 vote by Jon Haddad (a TLP guy) that backs doing the binary version upgrades through the cluster on a rolling basis, then doing the upgradesstables on a rolling basis.
>
> Under what cluster conditions are streaming/node replacement precluded, that is we are vulnerable to a cloud provided dumping one of our nodes under us or hardware failure? We ain't apple, but we do have 30+ node datacenters and 80-100 node clusters.
>
> Is the node replacement and streaming only disabled while there are heterogenous cassandra versions, or until all the sstables have been upgraded in the cluster?
>
> My instincts tell me the best thing to do is to get all the cassandra nodes to the same version without the upgradesstables step through the cluster, and then roll through the upgradesstables as needed, and that upgradesstables is a node-local concern that doesn't impact streaming or node replacement or other situations since cassandra can read old version sstables and new sstables would simply be the new format.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xxxxxxxxxxxxxxxxxxxx
For additional commands, e-mail: user-help@xxxxxxxxxxxxxxxxxxxx

--
-----------------
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting