osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Roadmap for 4.0


My wish list:

* Add support for arithmetic operators (CASSANDRA-11935)
* Allow IN restrictions on column families with collections
(CASSANDRA-12654)
* Add support for + and - operations on dates (CASSANDRA-11936)
* Add the currentTimestamp, currentDate, currentTime and currentTimeUUID
functions (CASSANDRA-13132)
* Allow selecting Map values and Set elements (CASSANDRA-7396)

Those are mostly useful for timeseries data models and I guess has no
significant impact on the internals and operations so the risk of
regression is low

On Mon, Apr 2, 2018 at 4:33 PM, Jeff Jirsa <jjirsa@xxxxxxxxx> wrote:

> 9608 (java9)
>
> --
> Jeff Jirsa
>
>
> > On Apr 2, 2018, at 3:45 AM, Jason Brown <jasedbrown@xxxxxxxxx> wrote:
> >
> > The only additional tickets I'd like to mention are:
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-13971 - Automatic
> > certificate management using Vault
> > - Stefan's Vault integration work. A sub-ticket, CASSANDRA-14102,
> addresses
> > encryption at-rest, subsumes CASSANDRA-9633 (SSTable encryption) - which
> I
> > doubt I would be able to get to any time this year. It would definitely
> be
> > nice to have a clarified encryption/security story for 4.0.
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-11990 - Address rows
> rather
> > than partitions in SASI
> > - a nice update for SASI, but not critical.
> >
> > -Jason
> >
> >> On Sat, Mar 31, 2018 at 6:53 PM, Ben Bromhead <ben@xxxxxxxxxxxxxxx>
> wrote:
> >>
> >> Apologies all, I didn't realize I was responding to this discussion
> only on
> >> the @user list. One of the perils of responding to a thread that is on
> both
> >> user and dev...
> >>
> >> For context, I have included my response to Kurt's previous discussion
> on
> >> this topic as it only ended up on the user list.
> >>
> >> *After some further discussions with folks offline, I'd like to revive
> this
> >> discussion. *
> >>
> >> *As Kurt mentioned, to keep it simple I if we can simply build consensus
> >> around what is in for 4.0 and what is out. We can then start the
> process of
> >> working off a 4.0 branch towards betas and release candidates. Again as
> >> Kurt mentioned, assigning a timeline to it right now is difficult, but
> >> having a firm line in the sand around what features/patches are in, then
> >> limiting future 4.0 work to bug fixes will give folks a less nebulous
> >> target to work on. *
> >>
> >> *The other thing to mention is that once we have a 4.0 branch to work
> off,
> >> we at Instaclustr have a commitment to dogfooding the release
> candidates on
> >> our internal staging and internal production workloads before 4.0
> becomes
> >> generally available. I know other folks have similar commitments and
> simply
> >> having a 4.0 branch with a clear list of things that are in or out will
> >> allow everyone to start testing and driving towards a quality release. *
> >>
> >> *The other thing is that there are already a large number of changes
> ready
> >> for 4.0, I would suggest not recommending tickets for 4.0 that have not
> yet
> >> been finished/have outstanding work unless you are the person working
> on it
> >> (or are offering to work on it instead) and can get it ready for review
> in
> >> a timely fashion. That way we can build a more realistic working target.
> >> For other major breaking changes, there is always 5.0 or 4.1 or
> whatever we
> >> end up doing :)*
> >>
> >> Thinking further about it, I would suggest a similar process that was
> >> applied to releasing 3.0, in order to get to 4.0:
> >>
> >>   - Clean up ticket labeling. Move tickets unlikely to make it / be
> worked
> >>   on for 4.0 to something else (e.g. 4.x or whatever).
> >>   - Tickets labeled 4.0 will be the line in the sand, with some trigger
> >>   ("done") event where all features not done by a certain event will
> >> simply
> >>   move into the next release. For the 3.0 branch, this occurred after a
> >>   large review of 8099. For 4.0 it could simply be resolving all current
> >>   blockers/major tickets tagged 4.0... doesn't have to be / nor is it
> >>   something I would strongly advocate.
> >>   - Once we hit this "done" event. Cut a Cassandra-4.0 branch and start
> >>   the alpha/beta/rc cycle from that branch, with only bugfixes going
> into
> >>   it
> >>   - This, in my mind, is similar to the 3.0 approach
> >>   https://mail-archives.apache.org/mod_mbox/cassandra-dev/
> >> 201503.mbox/%3CCALdd-zjAyiTbZksMeq2LxGwLF5LPhoi_
> >> 4vsjy8JBHBRnsxH%3D8A%40mail.gmail.com%3E,
> >>   but without the subsequent tick-tock :)
> >>
> >> There are currently 3 open blockers tagged 4.0, some are old and
> probably
> >> not really blockers anymore, there are other tickets that may/should be
> >> blockers on 4.0:
> >>
> >>   - https://issues.apache.org/jira/browse/CASSANDRA-13951
> >>   - https://issues.apache.org/jira/browse/CASSANDRA-13994
> >>   - https://issues.apache.org/jira/browse/CASSANDRA-12042
> >>
> >> In terms of major tickets that I would like to see land:
> >>
> >>   - https://issues.apache.org/jira/browse/CASSANDRA-7622 Virtual Tables
> >>   - https://issues.apache.org/jira/browse/CASSANDRA-13628 Internode
> netty
> >>   - https://issues.apache.org/jira/browse/CASSANDRA-13475 Pluggable
> >> Storage
> >>   - https://issues.apache.org/jira/browse/CASSANDRA-9633 SSTable
> >> encryption
> >>
> >> Ben
> >>
> >>> On Mon, Feb 12, 2018 at 11:26 PM Jeff Jirsa <jjirsa@xxxxxxxxx> wrote:
> >>>
> >>>
> >>> Advantages of cutting a release sooner than later:
> >>> 1) The project needs to constantly progress forward. Releases are the
> >> most
> >>> visible part of that.
> >>> 2) Having a huge changelog in a release increases the likelihood of
> bugs
> >>> that take time to find.
> >>>
> >>> Advantages of a slower release:
> >>> 1) We don't do major versions often, and when we do breaking changes
> >>> (protocol, file format, etc), we should squeeze in as many as possible
> to
> >>> avoid having to roll new majors
> >>> 2) There are probably few people actually running 3.11 at scale, so
> >>> probably few people actually testing trunk.
> >>>
> >>> In terms of "big" changes I'd like to see land, the ones that come to
> >> mind
> >>> are:
> >>>
> >>> https://issues.apache.org/jira/browse/CASSANDRA-9754 - "Birch"
> (changes
> >>> file format)
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13442 - Transient
> >>> Replicas (probably adds new replication strategy or similar)
> >>> https://issues.apache.org/jira/browse/CASSANDRA-13628 - Rest of the
> >>> internode netty stuff (no idea if this changes internode stuff, but I
> bet
> >>> it's a lot easier if it lands on a major)
> >>> https://issues.apache.org/jira/browse/CASSANDRA-7622 - Virtual Tables
> >>> (selfish inclusion, probably doesn't need to be a major at all, and I
> >>> wouldn't even lose sleep if it slips, but I'd like to see it land)
> >>>
> >>> Stuff I'm ok with slipping to 4.X or 5.0, but probably needs to land
> on a
> >>> major because we'll change something big (like gossip, or the way
> schema
> >> is
> >>> passed, etc):
> >>>
> >>> https://issues.apache.org/jira/browse/CASSANDRA-9667 - Strongly
> >>> consistent membership
> >>> https://issues.apache.org/jira/browse/CASSANDRA-10699 - Strongly
> >>> consistent schema
> >>>
> >>> All that said, what I really care about is building confidence in the
> >>> release, which means an extended testing cycle. If all of those patches
> >>> landed tomorrow, I'd still expect us to be months away from a release,
> >>> because we need to bake the next major - there's too many changes to
> >> throw
> >>> out an alpha/beta/rc and hope someone actually runs it.
> >>>
> >>> I don't believe Q3/Q4 is realistic, but I may be biased (or jaded).
> It's
> >>> possible Q3/Q4 alpha/beta is realistic, but definitely not a release.
> >>>
> >>>
> >>>
> >>>
> >>> On Sun, Feb 11, 2018 at 8:29 PM, kurt greaves <kurt@xxxxxxxxxxxxxxx>
> >>> wrote:
> >>>
> >>>> Hi friends,
> >>>> *TL;DR: Making a plan for 4.0, ideally everyone interested should
> >> provide
> >>>> up to two lists, one for tickets they can contribute resources to
> >> getting
> >>>> finished, and one for features they think would be desirable for 4.0,
> >> but
> >>>> not necessarily have the resources to commit to helping with.*
> >>>>
> >>>> So we had that Roadmap for 4.0 discussion last year, but there was
> never
> >>>> a conclusion or a plan that came from it. Times getting on and the
> >> changes
> >>>> list for 4.0 is getting pretty big. I'm thinking it would probably
> make
> >>>> sense to define some goals to getting 4.0 released/have an actual
> plan.
> >> 4.0
> >>>> is already going to be quite an unwieldy release with a lot of testing
> >>>> required.
> >>>>
> >>>> Note: the following is open to discussion, if people don't like the
> plan
> >>>> feel free to speak up. But in the end it's a pretty basic plan and I
> >> don't
> >>>> think we should over-complicate it, I also don't want to end up in a
> >>>> discussion where we "make a plan to make a plan". Regardless of
> whatever
> >>>> plan we do end up following it would still be valuable to have a list
> of
> >>>> tickets for 4.0 which is the overall goal of this email - so let's not
> >> get
> >>>> too worked up on the details just yet (save that for after I
> >>>> summarise/follow up).
> >>>>
> >>>> // TODO
> >>>> I think the best way to go about this would be for us to come up with
> a
> >>>> list of JIRA's that we want included in 4.0, tag these as 4.0, and all
> >>>> other improvements as 4.x. We can then aim to release 4.0 once all the
> >> 4.0
> >>>> tagged tickets (+bug fixes/blockers) are complete.
> >>>>
> >>>> Now, the catch is that we obviously don't want to include too many
> >>>> tickets in 4.0, but at the same time we want to make sure 4.0 has an
> >>>> appealing feature set for both users/operators/developers. To minimise
> >>>> scope creep I think the following strategy will help:
> >>>>
> >>>> We should maintain two lists:
> >>>>
> >>>>   1. JIRA's that people want in 4.0 and can commit resources to
> getting
> >>>>   them implemented in 4.0.
> >>>>   2. JIRA's that people simply think would be desirable for 4.0, but
> >>>>   currently don't have anyone assigned to them or planned assignment.
> >> It
> >>>>   would probably make sense to label these with an additional tag in
> >> JIRA. *(User's
> >>>>   please feel free to point out what you want here)*
> >>>>
> >>>> From list 1 will come our source of truth for when we release 4.0.
> >> (after
> >>>> aggregating a list I will summarise and we can vote on it).
> >>>>
> >>>> List 2 would be the "hopeful" list, where stories can be picked up
> from
> >>>> if resourcing allows, or where someone comes along and decides it's
> good
> >>>> enough to work on. I guess we can also base this on a vote system if
> we
> >>>> reach the point of including some of them. (but for the moment it's
> >> purely
> >>>> to get an idea of what users actually want).
> >>>>
> >>>> Please don't refrain from listing something that's already been
> >>>> mentioned. The purpose is to get an idea of everyone's
> >> priorities/interests
> >>>> and the resources available. We will need multiple resources for each
> >>>> ticket, so anywhere we share an interest will make for a lot easier
> work
> >>>> sharing.
> >>>>
> >>>> Note that we are only talking about improvements here. Bugs will be
> >>>> treated the same as always, and major issues/regressions we'll need to
> >> fix
> >>>> prior to 4.0 anyway.
> >>>>
> >>>> TIME FRAME
> >>>> Generally I think it's a bad idea to commit to any hard deadline, but
> we
> >>>> should have some time frames in mind. My idea would be to aim for a
> Q3/4
> >>>> 2018 release, and as we go we just review the outstanding improvements
> >> and
> >>>> decide whether it's worth pushing it back or if we've got enough to
> >>>> release. I suppose keep this time frame in mind when choosing your
> >> tickets.
> >>>>
> >>>> We can aim for an earlier date (midyear?) but I figure the
> >>>> testing/validation/bugfixing period prior to release might drag on a
> >> bit so
> >>>> being a bit conservative here.
> >>>> The main goal would be to not let list 1 grow unless we're well ahead,
> >>>> and only cull from it if we're heavily over-committed or we decide the
> >>>> improvement can wait. I assume this all sounds like common sense but
> >>>> figured it's better to spell it out now.
> >>>>
> >>>>
> >>>> NEXT STEPS
> >>>> After 2 weeks/whenever the discussion dies off I'll consolidate all
> the
> >>>> tickets, relevant comments and follow up with a summary, where we can
> >>>> discuss/nitpick issues and come up with a final list to go ahead with.
> >>>>
> >>>> On a side note, in conjunction with this effort we'll obviously have
> to
> >>>> do something about validation and testing. I'll keep that out of this
> >> email
> >>>> for now, but there will be a follow up so that those of us willing to
> >> help
> >>>> validate/test trunk can avoid duplicating effort.
> >>>>
> >>>> REVIEW
> >>>> This is the list of "huge/breaking" tickets that got mentioned in the
> >>>> last roadmap discussion and their statuses. This is not terribly
> >> important
> >>>> but just so we can keep in mind what we previously talked about. I
> >> think we
> >>>> leave it up to the relevant contributors to decide whether they want
> to
> >> get
> >>>> the still open tickets into 4.0.
> >>>>
> >>>> CASSANDRA-9425 Immutable node-local schema
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9425> - Committed
> >>>> CASSANDRA-10699 Strongly consistent schema alterations
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10699> - Open, no
> >>>> discussion in quite some time.
> >>>> CASSANDRA-12229 NIO streaming
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12229> - Committed
> >>>> CASSANDRA-8457 NIO messaging
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8457> - Committed
> >>>> CASSANDRA-12345 Gossip 2.0
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12345> - Open, no
> sign
> >>>> of any action.
> >>>> CASSANDRA-9754 Make index info heap friendly for large CQL partitions
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> - In progress
> >> but
> >>>> no update in a long time.
> >>>> CASSANDRA-11559 enhanced node representation
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> - Open, no
> >>>> change since early 2016.
> >>>> CASSANDRA-6246 epaxos
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-6246> - In progress
> >> but
> >>>> no update since Feb 2017.
> >>>> CASSANDRA-7544 storage port configurable per node
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-7544> - Committed
> >>>> CASSANDRA-11115 remove thrift support
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11115> - Committed
> >>>> CASSANDRA-10857 dropping compact storage
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10857> - Committed
> >>>>
> >>>> To start us off...
> >>>> And here are my lists to get us started.
> >>>> 1.
> >>>> CASSANDRA-8460 - Tiered/Cold storage for TWCS
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8460>
> >>>> CASSANDRA-12783 - Batchlog redesign
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12783>
> >>>> CASSANDRA-11559 - Enchance node representation
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559>
> >>>>    CASSANDRA-12344 - Forward writes to replacement node with same
> >>>> address <https://issues.apache.org/jira/browse/CASSANDRA-12344>
> >>>> CASSANDRA-8119 - More expressive Consistency Levels
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8119>
> >>>> CASSANDRA-14210 - Optimise SSTables upgrade task scheduling
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-14210>
> >>>> CASSANDRA-10540 - RangeAwareCompaction
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10540>
> >>>>
> >>>>
> >>>> 2:
> >>>> CASSANDRA-10726 - Read repair inserts should not be blocking
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10726>
> >>>> CASSANDRA-9754 - Make index info heap friendly for large CQL
> partitions
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754>
> >>>> CASSANDRA-12294 - LDAP auth
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12294>
> >>>> CASSANDRA-12151 - Audit logging
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12151>
> >>>> CASSANDRA-10495 - Fix streaming with vnodes
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10495>
> >>>>
> >>>> Also, here's some handy JQL to start you off:
> >>>> project = CASSANDRA AND fixVersion in (4.x, 4.0) AND issue in
> >>>> watchedIssues() AND status != Resolved
> >>>>
> >>>>
> >>> --
> >> Ben Bromhead
> >> CTO | Instaclustr <https://www.instaclustr.com/>
> >> +1 650 284 9692
> >> Reliability at Scale
> >> Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
>
>