osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Testing 4.0 Post-Freeze


We have done all this for previous releases and we know it has not worked
well. So how giving it one more try is going to help here. Can someone
outline what will change for 4.0 which will make it more successful?

Not branching is one idea but we should discuss what will change now than
iterating what has already happened.

On Mon, Jul 9, 2018 at 11:25 AM Jeremy Hanna <jeremy.hanna1234@xxxxxxxxx>
wrote:

> I wholeheartedly agree with efforts to make releases more stable and the
> more contributors that add tests or test within the context of their own
> use cases, the better.  +1 non-binding to the idea of better, more complete
> testing for releases.
>
> When it comes to how to do this, I think that’s where there is
> disagreement.  I personally don’t think that branching detracts from the
> goal of stabilization.  There are a couple of personas involved here:
>
> * Longtime contributors/committers: I think it’s sufficient to generally
> ask that whatever time/effort they can contribute be towards stabilizing,
> testing, and especially testing their production scenarios to cover more
> surface area when it comes to usage edge cases.  That along with testing
> longer running things in those scenarios for other types of edge cases.
>
> * New contributors: They likely won’t know about the strategy.  They’re
> just poking around and looking at lhf tickets or tickets that they need
> done.  Those are typically low risk but at the same time we don’t want to
> compromise stability by merging those in.  Reviewing low risk stuff for
> inclusion doesn’t take a ton of time and gives them a sense that they can
> be of help and empowers them.  After they have that first experience, then
> a longer term contributor could share with them a blog post or tldr thread
> summary about the 4.0 stabilization efforts (would be nice to have one to
> point people too once we're done).  We could also start creating lhf
> tickets with stabilization and testing in mind.
>
> Unless we want to make a fundamental change to the project’s process
> (making trunk stable at all times going forward), then I don’t think
> there’s a reason why branching would detract from these efforts.  In other
> words if we’re just trying to say that we want to focus on stabilization
> for the alpha/beta/rc of 4.0, then I would prefer branching along with
> clear messaging.
>
> Jeremy
>
> > On Jul 9, 2018, at 12:43 PM, sankalp kohli <kohlisankalp@xxxxxxxxx>
> wrote:
> >
> > How is this preventing someone from working and collaborating on an
> Apache
> > Project? You can still collaborate on making 4.0 more stable.
> >
> > Why will these committers not work on making 4.0 more stable and fix
> broken
> > tests? Considering  we cannot release without test passing, how will
> these
> > features be used?
> >
> > On Mon, Jul 9, 2018 at 10:03 AM Jonathan Haddad <jon@xxxxxxxxxxxxx>
> wrote:
> >
> >> I don't see how changing the process and banning feature commits is
> >> going to be any help to the project. There may be a couple committers
> >> who are interested in committing new features.
> >>
> >> I'm a -1 on changing the branching strategy in a way that prevents
> >> people from working and collaborating on an Apache project.
> >>
> >> On Mon, Jul 9, 2018 at 9:56 AM sankalp kohli <kohlisankalp@xxxxxxxxx>
> >> wrote:
> >>>
> >>> I did not see a -1 but all +0s and a few +1s.
> >>>
> >>> On Mon, Jul 9, 2018 at 5:49 AM Josh McKenzie <jmckenzie@xxxxxxxxxx>
> >> wrote:
> >>>
> >>>> What did we land on? Sounds like we're pretty split without consensus
> >> on
> >>>> "change the old branching strategy and reject new things on trunk
> >> during
> >>>> stabilization" vs. "keep doing things the way we did but message
> >> strongly
> >>>> that we won't be reviewing new things until 4.0 is stable".
> >>>>
> >>>> On Sat, Jul 7, 2018 at 5:01 PM Sankalp Kohli <kohlisankalp@xxxxxxxxx>
> >>>> wrote:
> >>>>
> >>>>> Does anyone has any more feedback on this?
> >>>>>
> >>>>>> On Jul 4, 2018, at 05:36, Aleksey Yeshchenko <aleksey@xxxxxxxxx>
> >>>> wrote:
> >>>>>>
> >>>>>> For what it’s worth, I’m fine with both formal branching-level
> >> freeze
> >>>>> and informal ‘let people commit to trunk if they can find a
> >> reviewer’ one
> >>>>> and will support either.
> >>>>>>
> >>>>>> So long as either/both are communicated to the contributors, the
> >> only
> >>>>> difference is in where new feature work gets accumulated: will stay
> >> a bit
> >>>>> longer in personal branches in the latter way.
> >>>>>>
> >>>>>> —
> >>>>>> AY
> >>>>>>
> >>>>>> On 4 July 2018 at 01:30:40, sankalp kohli (kohlisankalp@xxxxxxxxx)
> >>>>> wrote:
> >>>>>>
> >>>>>> Having an explicit way to tell the community that we all will work
> >> on
> >>>>>> testing is better than writing a patch which will sit without
> >> review
> >>>>> for
> >>>>>> months. I think not having your patch reviewed for months is more
> >>>>>> discouraging than following the community and helping with
> >> stability of
> >>>>>> 4.0.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Jul 3, 2018 at 3:02 PM Josh McKenzie <jmckenzie@xxxxxxxxxx
> >>>
> >>>>> wrote:
> >>>>>>
> >>>>>>>>
> >>>>>>>> We propose that between the September freeze date and beta, a new
> >>>>> branch
> >>>>>>>> would not be created and trunk would only have bug fixes and
> >>>>> performance
> >>>>>>>> improvements committed to it.
> >>>>>>>
> >>>>>>>
> >>>>>>> This is more of a call to action and announcement of intent than
> >> an
> >>>>> attempt
> >>>>>>>> to
> >>>>>>>> enforce policy; we can and probably will branch off 4.0, and keep
> >>>>> trunk
> >>>>>>>> technically active.
> >>>>>>>
> >>>>>>> These are two very different statements. :) Which is it?
> >>>>>>>
> >>>>>>> On Tue, Jul 3, 2018 at 5:57 PM Aleksey Yeshchenko <
> >> aleksey@xxxxxxxxx>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> If we want to have a stable, usable 4.0.0 release out in the next
> >>>>> 6-12
> >>>>>>>> months, there needs to be a focused effort on getting it out - or
> >>>>> else
> >>>>>>>> it’ll just never happen.
> >>>>>>>>
> >>>>>>>> This is more of a call to action and announcement of intent than
> >> an
> >>>>>>>> attempt to enforce policy; we can and probably will branch off
> >> 4.0,
> >>>>> and
> >>>>>>>> keep trunk technically active. But so long as there is a critical
> >>>> mass
> >>>>> of
> >>>>>>>> active contributors who are on board with only/mostly working on
> >>>>>>> stability,
> >>>>>>>> bug fixes, and validation - both as assignees and reviewers -
> >> we’ll
> >>>>> have
> >>>>>>> a
> >>>>>>>> de-facto freeze.
> >>>>>>>>
> >>>>>>>> And I have a feeling that there is such a critical mass.
> >>>>>>>>
> >>>>>>>> —
> >>>>>>>> AY
> >>>>>>>>
> >>>>>>>> On 3 July 2018 at 22:23:38, Jeff Jirsa (jjirsa@xxxxxxxxx) wrote:
> >>>>>>>>
> >>>>>>>> I think there's value in the psychological commitment that if
> >> someone
> >>>>> has
> >>>>>>>> time to contribute, their contributions should be focused on
> >>>>> validating a
> >>>>>>>> release, not pushing future features.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Jul 3, 2018 at 1:03 PM, Jonathan Haddad <
> >> jon@xxxxxxxxxxxxx>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I agree with Josh. I don’t see how changing the convention
> >> around
> >>>>> trunk
> >>>>>>>>> will improve the process, seems like it’ll only introduce a
> >> handful
> >>>>> of
> >>>>>>>>> rollback commits when people forget.
> >>>>>>>>>
> >>>>>>>>> Other than that, it all makes sense to me.
> >>>>>>>>>
> >>>>>>>>> I’ve been working on a workload centric stress tool on and off
> >> for a
> >>>>>>>> little
> >>>>>>>>> bit in an effort to create something that will help with wider
> >>>>> adoption
> >>>>>>>> in
> >>>>>>>>> stress testing. It differs from the stress we ship by including
> >>>>> fully
> >>>>>>>>> functional stress workloads as well as a validation process. The
> >>>>> idea
> >>>>>>>> being
> >>>>>>>>> to be flexible enough to test both performance and correctness
> >> in
> >>>>> LWT
> >>>>>>>> and
> >>>>>>>>> MVs as well as other arbitrary workloads.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_thelastpickle_tlp-2Dstress&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=l_G2ByhfCyu3k9TzNVqiagdVQ8vOMJqHZvDq_JKvbiQ&s=f8gf_JCP6JRQIRkL_1R_zJOS_6gdAAsLleDr2PZHppE&e=
> >>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Jon
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jul 3, 2018 at 12:28 PM Josh McKenzie <
> >> jmckenzie@xxxxxxxxxx
> >>>>>
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Why not just branch a 4.0-rel and bugfix there and merge up
> >> while
> >>>>>>>> still
> >>>>>>>>>> accepting new features or improvements on trunk?
> >>>>>>>>>>
> >>>>>>>>>> I don't think the potential extra engagement in testing will
> >>>>> balance
> >>>>>>>> out
> >>>>>>>>>> the atrophy and discouraging contributions / community
> >> engagement
> >>>>>>>> we'd
> >>>>>>>>> get
> >>>>>>>>>> by deferring all improvements and new features in an open-ended
> >>>>> way.
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jul 3, 2018 at 1:33 PM sankalp kohli <
> >>>> kohlisankalp@xxxxxxxxx
> >>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi cassandra-dev@,
> >>>>>>>>>>>
> >>>>>>>>>>> With the goal of making Cassandra's 4.0 the most stable major
> >>>>>>>> release
> >>>>>>>>> to
> >>>>>>>>>>> date, we would like all committers of the project to consider
> >>>>>>>> joining
> >>>>>>>>> us
> >>>>>>>>>> in
> >>>>>>>>>>> dedicating their time and attention to testing, running, and
> >>>>> fixing
> >>>>>>>>>> issues
> >>>>>>>>>>> in 4.0 between the September freeze and the 4.0 beta release.
> >> This
> >>>>>>>>> would
> >>>>>>>>>>> result in a freeze of new feature development on trunk or
> >> branches
> >>>>>>>>> during
> >>>>>>>>>>> this period, instead focusing on writing, improving, and
> >> running
> >>>>>>>> tests
> >>>>>>>>> or
> >>>>>>>>>>> fixing and reviewing bugs or performance regressions found in
> >> 4.0
> >>>>>>>> or
> >>>>>>>>>>> earlier.
> >>>>>>>>>>>
> >>>>>>>>>>> How would this work?
> >>>>>>>>>>>
> >>>>>>>>>>> We propose that between the September freeze date and beta, a
> >> new
> >>>>>>>>> branch
> >>>>>>>>>>> would not be created and trunk would only have bug fixes and
> >>>>>>>>> performance
> >>>>>>>>>>> improvements committed to it. At the same time we do not want
> >> to
> >>>>>>>>>> discourage
> >>>>>>>>>>> community contributions. Not all contributors can be expected
> >> to
> >>>>> be
> >>>>>>>>> aware
> >>>>>>>>>>> of such a decision or may be new to the project. In cases
> >> where
> >>>>> new
> >>>>>>>>>>> features are contributed during this time, the contributor
> >> can be
> >>>>>>>>>> informed
> >>>>>>>>>>> of the current status of the release process, be encouraged to
> >>>>>>>>> contribute
> >>>>>>>>>>> to testing or bug fixing, and have their feature reviewed
> >> after
> >>>>> the
> >>>>>>>>> beta
> >>>>>>>>>> is
> >>>>>>>>>>> reached.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> What happens when beta is reached?
> >>>>>>>>>>>
> >>>>>>>>>>> Ideally, contributors who have made significant contributions
> >> to
> >>>>>>>> the
> >>>>>>>>>>> release will stick around to continue testing between beta and
> >>>>>>>> final
> >>>>>>>>>>> release. Any additional folks who continue this focus would
> >> also
> >>>>> be
> >>>>>>>>>> greatly
> >>>>>>>>>>> appreciated.
> >>>>>>>>>>>
> >>>>>>>>>>> What about before the freeze?
> >>>>>>>>>>>
> >>>>>>>>>>> Testing new features is of course important. This isn't meant
> >> to
> >>>>>>>>>> discourage
> >>>>>>>>>>> development – only to enable us to focus on testing and
> >> hardening
> >>>>>>>> 4.0
> >>>>>>>>> to
> >>>>>>>>>>> deliver Cassandra's most stable major release. We would like
> >> to
> >>>>> see
> >>>>>>>>>>> adoption of 4.0 happen much more quickly than its predecessor.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for considering this proposal,
> >>>>>>>>>>> Sankalp Kohli
> >>>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Jon Haddad
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=l_G2ByhfCyu3k9TzNVqiagdVQ8vOMJqHZvDq_JKvbiQ&s=paSngQpMm3DhoWay8lDuWEYELVOrti8evQeT1LodXdY&e=
> >>>>>
> >>>>>>>>
> >>>>>>>>> twitter: rustyrazorblade
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> >>>>> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
> >>>>
> >>
> >>
> >>
> >> --
> >> Jon Haddad
> >> http://www.rustyrazorblade.com
> >> twitter: rustyrazorblade
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> >> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
>
>