[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Testing 4.0 Post-Freeze

Here’s why I really like this proposal:

– It focuses our thought and effort on what it’ll take for each of us to become confident in running Apache Cassandra 4.0 for our most critical applications *prior* to cutting the dot-zero release.
– It marks a commitment we can make to our user community: delivering 4.0’s featureset with safety and stability is our top priority.
– There are many different perspectives each contributor can bring: correctness, performance, durability, automation, documentation, operational safety, etc; and many ways to gain confidence in each – perf tests, profiling, shadow writes/traffic mirroring, diff tests, replay tests, fuzz tests, fault injection, chaos engineering, and so many others. By bringing diverse methods to bear on the same class of problem (quality), we’re more likely to identify issues that could impact our users and to ship a better release.
– It’s disciplined and courageous!

Here’s why I think this won’t discourage new contributors from joining – in fact, the opposite:

– It provides a very simple step that any contributor can take toward shipping 4.0: running it.
– We raise spirits by delivering enhancements the community has been working on for two years rather than fracturing our attention.
– If members of the community are interested in post-4.0 work, they’re not blocked from making progress toward that – but it’s not the focus, and unlikely that review bandwidth would be available given that focus. I agree with Kurt’s note that nobody can be “forced" to do anything - but by committing to reserving trunk for stabilizing changes until the beta, we focus our effort on polishing and delivering the release.

On the last question:
> On another note, we are still planning on accepting/reviewing bug fixes for previous versions as well correct?

I’d expect so – and to take it a step further, it’s likely that this rigor will result in us identifying serious issues that were not regressions in 4.0 warranting backports. As with any investment in testing / validation, I’m hoping we do … but also hoping we don’t. :-)

> On Jul 3, 2018, at 7:21 PM, kurt greaves <kurt@xxxxxxxxxxxxxxx> wrote:
> tl;dr: +1 for committing to testing + bugfixing after feature freeze, but I
> can't really see how changing the branching strategy is going to have any
> impact at all.
> I really don't think it's a big deal. People will work based on whatever
> priorities they've been assigned, regardless of what you do to the
> branching. That's essentially just red tape and adding unnecessary
> complexity to an already established process. Granted, you'll have to do
> one less merge, but it should be an effortless merge, and it saves there
> being any confusion about what people should do with features. If someone
> decided to commit a feature, they could, and we wouldn't have to be all
> over every ticket making sure people don't commit things that shouldn't be
> committed.
> Cutting to the point, all we're really aiming for here is a commitment from
> the community to only dev on bugfixes, only review bugfixes, and
> testing/validation during the freeze period. That's not going to be
> enforced by a change in branching strategy, it's going to be enforced by
> the contributors themselves (and their respective priority-setters).
> The best we can do is encourage a commitment to testing and bugfixing for
> the freeze period. This is a volunteer based project after all, so we can't
> really force anyone to do anything. If contributors make proposals for
> features in the freeze period we can just tell them that there is a focus
> on 4.0 testing and it's unlikely to be reviewed until 4.0 is released at
> best.
> On another note, we are still planning on accepting/reviewing bug fixes for
> previous versions as well correct?  Just to clarify because it doesn't seem
> to be mentioned here and we don't want people getting in arguments over
> reviewing patches that only affect older releases.
> I think not having your patch reviewed for months is more
>> discouraging than following the community and helping with stability of
>> 4.0.
> Patches not getting reviewed for months on end already occurs. Changing how
> we branch isn't going to change this or make anyone feel better about it.
> The best we can do is communicate why their patches aren't getting
> reviewed, and we should be doing this on individual feature tickets that
> get raised and on the ML.
> On 4 July 2018 at 00:44, Mick Semb Wever <mick@xxxxxxxxxxxxxxxxx> wrote:
>>> We propose that between the September freeze date and beta, a new branch
>>> would not be created and trunk would only have bug fixes and performance
>>> improvements committed to it.
>> +1, like really yes!! because it's a step towards a more stable-master
>> project.
>> If trunk is focused on stabilising (which ideally it shouldn't have to, if
>> stable-master were true) then new features remaining in their respective
>> branches shouldn't be a big cost or risk (nor seen as anything negative).
>> And hopefully any additional practices/lessons learnt from during the
>> freeze period get carried on for good.
>> Although, and unfortunately, there's just not the testing infrastructure
>> available to the public to ensure feature branches are solid before
>> merging. But I struggle to see that as an excuse to only "stabilise" on
>> release branches.
>> 2¢,
>> mick