osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Testing 4.0 Post-Freeze


Sure...thats an addition to the 3 states I was thinking(-1,+/-0 and +1) :)

On Mon, Jul 9, 2018 at 12:49 PM Josh McKenzie <jmckenzie@xxxxxxxxxx> wrote:

> >
> > I did not see a -1 but all +0s and a few +1s.
>
> It's more accurate to say you have quite a few -0's, some +0's, and a few
> +1's, probably some -0.5's if you're into that.
>
> On Mon, Jul 9, 2018 at 2:53 PM Jeremy Hanna <jeremy.hanna1234@xxxxxxxxx>
> wrote:
>
> > What I’m saying is that for new contributors, not branching has a cost
> and
> > I think there are other ways to organize efforts.
> >
> > One of the suggestions was for each organization to test their own use
> > cases in the stabilization process.  I don’t think that’s been done very
> > effectively previously.  Most organizations only do any sort of
> significant
> > testing when they’re about to put their clusters on the line - i.e. once
> a
> > version has several post GA point releases.  That’s logical and no one
> > wants to be the first one to production.  However, if people were to
> agree
> > to do some form of testing pre-release, then I think that would be a step
> > in the right direction.  In the early days of the project I tried to do
> > this in a small way by testing the Hadoop support for every release
> (major
> > and minor) since it wasn’t on everyone’s radar but was important to me.
> > Then I would vote with a non-binding vote and described what was tested.
> >
> > > On Jul 9, 2018, at 1:30 PM, sankalp kohli <kohlisankalp@xxxxxxxxx>
> > wrote:
> > >
> > > We have done all this for previous releases and we know it has not
> worked
> > > well. So how giving it one more try is going to help here. Can someone
> > > outline what will change for 4.0 which will make it more successful?
> > >
> > > Not branching is one idea but we should discuss what will change now
> than
> > > iterating what has already happened.
> > >
> > > On Mon, Jul 9, 2018 at 11:25 AM Jeremy Hanna <
> jeremy.hanna1234@xxxxxxxxx
> > >
> > > wrote:
> > >
> > >> I wholeheartedly agree with efforts to make releases more stable and
> the
> > >> more contributors that add tests or test within the context of their
> own
> > >> use cases, the better.  +1 non-binding to the idea of better, more
> > complete
> > >> testing for releases.
> > >>
> > >> When it comes to how to do this, I think that’s where there is
> > >> disagreement.  I personally don’t think that branching detracts from
> the
> > >> goal of stabilization.  There are a couple of personas involved here:
> > >>
> > >> * Longtime contributors/committers: I think it’s sufficient to
> generally
> > >> ask that whatever time/effort they can contribute be towards
> > stabilizing,
> > >> testing, and especially testing their production scenarios to cover
> more
> > >> surface area when it comes to usage edge cases.  That along with
> testing
> > >> longer running things in those scenarios for other types of edge
> cases.
> > >>
> > >> * New contributors: They likely won’t know about the strategy.
> They’re
> > >> just poking around and looking at lhf tickets or tickets that they
> need
> > >> done.  Those are typically low risk but at the same time we don’t want
> > to
> > >> compromise stability by merging those in.  Reviewing low risk stuff
> for
> > >> inclusion doesn’t take a ton of time and gives them a sense that they
> > can
> > >> be of help and empowers them.  After they have that first experience,
> > then
> > >> a longer term contributor could share with them a blog post or tldr
> > thread
> > >> summary about the 4.0 stabilization efforts (would be nice to have one
> > to
> > >> point people too once we're done).  We could also start creating lhf
> > >> tickets with stabilization and testing in mind.
> > >>
> > >> Unless we want to make a fundamental change to the project’s process
> > >> (making trunk stable at all times going forward), then I don’t think
> > >> there’s a reason why branching would detract from these efforts.  In
> > other
> > >> words if we’re just trying to say that we want to focus on
> stabilization
> > >> for the alpha/beta/rc of 4.0, then I would prefer branching along with
> > >> clear messaging.
> > >>
> > >> Jeremy
> > >>
> > >>> On Jul 9, 2018, at 12:43 PM, sankalp kohli <kohlisankalp@xxxxxxxxx>
> > >> wrote:
> > >>>
> > >>> How is this preventing someone from working and collaborating on an
> > >> Apache
> > >>> Project? You can still collaborate on making 4.0 more stable.
> > >>>
> > >>> Why will these committers not work on making 4.0 more stable and fix
> > >> broken
> > >>> tests? Considering  we cannot release without test passing, how will
> > >> these
> > >>> features be used?
> > >>>
> > >>> On Mon, Jul 9, 2018 at 10:03 AM Jonathan Haddad <jon@xxxxxxxxxxxxx>
> > >> wrote:
> > >>>
> > >>>> I don't see how changing the process and banning feature commits is
> > >>>> going to be any help to the project. There may be a couple
> committers
> > >>>> who are interested in committing new features.
> > >>>>
> > >>>> I'm a -1 on changing the branching strategy in a way that prevents
> > >>>> people from working and collaborating on an Apache project.
> > >>>>
> > >>>> On Mon, Jul 9, 2018 at 9:56 AM sankalp kohli <
> kohlisankalp@xxxxxxxxx>
> > >>>> wrote:
> > >>>>>
> > >>>>> I did not see a -1 but all +0s and a few +1s.
> > >>>>>
> > >>>>> On Mon, Jul 9, 2018 at 5:49 AM Josh McKenzie <jmckenzie@xxxxxxxxxx
> >
> > >>>> wrote:
> > >>>>>
> > >>>>>> What did we land on? Sounds like we're pretty split without
> > consensus
> > >>>> on
> > >>>>>> "change the old branching strategy and reject new things on trunk
> > >>>> during
> > >>>>>> stabilization" vs. "keep doing things the way we did but message
> > >>>> strongly
> > >>>>>> that we won't be reviewing new things until 4.0 is stable".
> > >>>>>>
> > >>>>>> On Sat, Jul 7, 2018 at 5:01 PM Sankalp Kohli <
> > kohlisankalp@xxxxxxxxx>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Does anyone has any more feedback on this?
> > >>>>>>>
> > >>>>>>>> On Jul 4, 2018, at 05:36, Aleksey Yeshchenko <aleksey@xxxxxxxxx
> >
> > >>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> For what it’s worth, I’m fine with both formal branching-level
> > >>>> freeze
> > >>>>>>> and informal ‘let people commit to trunk if they can find a
> > >>>> reviewer’ one
> > >>>>>>> and will support either.
> > >>>>>>>>
> > >>>>>>>> So long as either/both are communicated to the contributors, the
> > >>>> only
> > >>>>>>> difference is in where new feature work gets accumulated: will
> stay
> > >>>> a bit
> > >>>>>>> longer in personal branches in the latter way.
> > >>>>>>>>
> > >>>>>>>> —
> > >>>>>>>> AY
> > >>>>>>>>
> > >>>>>>>> On 4 July 2018 at 01:30:40, sankalp kohli (
> kohlisankalp@xxxxxxxxx
> > )
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Having an explicit way to tell the community that we all will
> work
> > >>>> on
> > >>>>>>>> testing is better than writing a patch which will sit without
> > >>>> review
> > >>>>>>> for
> > >>>>>>>> months. I think not having your patch reviewed for months is
> more
> > >>>>>>>> discouraging than following the community and helping with
> > >>>> stability of
> > >>>>>>>> 4.0.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Jul 3, 2018 at 3:02 PM Josh McKenzie <
> > jmckenzie@xxxxxxxxxx
> > >>>>>
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> We propose that between the September freeze date and beta, a
> > new
> > >>>>>>> branch
> > >>>>>>>>>> would not be created and trunk would only have bug fixes and
> > >>>>>>> performance
> > >>>>>>>>>> improvements committed to it.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> This is more of a call to action and announcement of intent
> than
> > >>>> an
> > >>>>>>> attempt
> > >>>>>>>>>> to
> > >>>>>>>>>> enforce policy; we can and probably will branch off 4.0, and
> > keep
> > >>>>>>> trunk
> > >>>>>>>>>> technically active.
> > >>>>>>>>>
> > >>>>>>>>> These are two very different statements. :) Which is it?
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Jul 3, 2018 at 5:57 PM Aleksey Yeshchenko <
> > >>>> aleksey@xxxxxxxxx>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> If we want to have a stable, usable 4.0.0 release out in the
> > next
> > >>>>>>> 6-12
> > >>>>>>>>>> months, there needs to be a focused effort on getting it out -
> > or
> > >>>>>>> else
> > >>>>>>>>>> it’ll just never happen.
> > >>>>>>>>>>
> > >>>>>>>>>> This is more of a call to action and announcement of intent
> than
> > >>>> an
> > >>>>>>>>>> attempt to enforce policy; we can and probably will branch off
> > >>>> 4.0,
> > >>>>>>> and
> > >>>>>>>>>> keep trunk technically active. But so long as there is a
> > critical
> > >>>>>> mass
> > >>>>>>> of
> > >>>>>>>>>> active contributors who are on board with only/mostly working
> on
> > >>>>>>>>> stability,
> > >>>>>>>>>> bug fixes, and validation - both as assignees and reviewers -
> > >>>> we’ll
> > >>>>>>> have
> > >>>>>>>>> a
> > >>>>>>>>>> de-facto freeze.
> > >>>>>>>>>>
> > >>>>>>>>>> And I have a feeling that there is such a critical mass.
> > >>>>>>>>>>
> > >>>>>>>>>> —
> > >>>>>>>>>> AY
> > >>>>>>>>>>
> > >>>>>>>>>> On 3 July 2018 at 22:23:38, Jeff Jirsa (jjirsa@xxxxxxxxx)
> > wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> I think there's value in the psychological commitment that if
> > >>>> someone
> > >>>>>>> has
> > >>>>>>>>>> time to contribute, their contributions should be focused on
> > >>>>>>> validating a
> > >>>>>>>>>> release, not pushing future features.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Jul 3, 2018 at 1:03 PM, Jonathan Haddad <
> > >>>> jon@xxxxxxxxxxxxx>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> I agree with Josh. I don’t see how changing the convention
> > >>>> around
> > >>>>>>> trunk
> > >>>>>>>>>>> will improve the process, seems like it’ll only introduce a
> > >>>> handful
> > >>>>>>> of
> > >>>>>>>>>>> rollback commits when people forget.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Other than that, it all makes sense to me.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I’ve been working on a workload centric stress tool on and
> off
> > >>>> for a
> > >>>>>>>>>> little
> > >>>>>>>>>>> bit in an effort to create something that will help with
> wider
> > >>>>>>> adoption
> > >>>>>>>>>> in
> > >>>>>>>>>>> stress testing. It differs from the stress we ship by
> including
> > >>>>>>> fully
> > >>>>>>>>>>> functional stress workloads as well as a validation process.
> > The
> > >>>>>>> idea
> > >>>>>>>>>> being
> > >>>>>>>>>>> to be flexible enough to test both performance and
> correctness
> > >>>> in
> > >>>>>>> LWT
> > >>>>>>>>>> and
> > >>>>>>>>>>> MVs as well as other arbitrary workloads.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_thelastpickle_tlp-2Dstress&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=l_G2ByhfCyu3k9TzNVqiagdVQ8vOMJqHZvDq_JKvbiQ&s=f8gf_JCP6JRQIRkL_1R_zJOS_6gdAAsLleDr2PZHppE&e=
> > >>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Jon
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Jul 3, 2018 at 12:28 PM Josh McKenzie <
> > >>>> jmckenzie@xxxxxxxxxx
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Why not just branch a 4.0-rel and bugfix there and merge up
> > >>>> while
> > >>>>>>>>>> still
> > >>>>>>>>>>>> accepting new features or improvements on trunk?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I don't think the potential extra engagement in testing will
> > >>>>>>> balance
> > >>>>>>>>>> out
> > >>>>>>>>>>>> the atrophy and discouraging contributions / community
> > >>>> engagement
> > >>>>>>>>>> we'd
> > >>>>>>>>>>> get
> > >>>>>>>>>>>> by deferring all improvements and new features in an
> > open-ended
> > >>>>>>> way.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Jul 3, 2018 at 1:33 PM sankalp kohli <
> > >>>>>> kohlisankalp@xxxxxxxxx
> > >>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi cassandra-dev@,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> With the goal of making Cassandra's 4.0 the most stable
> major
> > >>>>>>>>>> release
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>> date, we would like all committers of the project to
> consider
> > >>>>>>>>>> joining
> > >>>>>>>>>>> us
> > >>>>>>>>>>>> in
> > >>>>>>>>>>>>> dedicating their time and attention to testing, running,
> and
> > >>>>>>> fixing
> > >>>>>>>>>>>> issues
> > >>>>>>>>>>>>> in 4.0 between the September freeze and the 4.0 beta
> release.
> > >>>> This
> > >>>>>>>>>>> would
> > >>>>>>>>>>>>> result in a freeze of new feature development on trunk or
> > >>>> branches
> > >>>>>>>>>>> during
> > >>>>>>>>>>>>> this period, instead focusing on writing, improving, and
> > >>>> running
> > >>>>>>>>>> tests
> > >>>>>>>>>>> or
> > >>>>>>>>>>>>> fixing and reviewing bugs or performance regressions found
> in
> > >>>> 4.0
> > >>>>>>>>>> or
> > >>>>>>>>>>>>> earlier.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> How would this work?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We propose that between the September freeze date and
> beta, a
> > >>>> new
> > >>>>>>>>>>> branch
> > >>>>>>>>>>>>> would not be created and trunk would only have bug fixes
> and
> > >>>>>>>>>>> performance
> > >>>>>>>>>>>>> improvements committed to it. At the same time we do not
> want
> > >>>> to
> > >>>>>>>>>>>> discourage
> > >>>>>>>>>>>>> community contributions. Not all contributors can be
> expected
> > >>>> to
> > >>>>>>> be
> > >>>>>>>>>>> aware
> > >>>>>>>>>>>>> of such a decision or may be new to the project. In cases
> > >>>> where
> > >>>>>>> new
> > >>>>>>>>>>>>> features are contributed during this time, the contributor
> > >>>> can be
> > >>>>>>>>>>>> informed
> > >>>>>>>>>>>>> of the current status of the release process, be encouraged
> > to
> > >>>>>>>>>>> contribute
> > >>>>>>>>>>>>> to testing or bug fixing, and have their feature reviewed
> > >>>> after
> > >>>>>>> the
> > >>>>>>>>>>> beta
> > >>>>>>>>>>>> is
> > >>>>>>>>>>>>> reached.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> What happens when beta is reached?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Ideally, contributors who have made significant
> contributions
> > >>>> to
> > >>>>>>>>>> the
> > >>>>>>>>>>>>> release will stick around to continue testing between beta
> > and
> > >>>>>>>>>> final
> > >>>>>>>>>>>>> release. Any additional folks who continue this focus would
> > >>>> also
> > >>>>>>> be
> > >>>>>>>>>>>> greatly
> > >>>>>>>>>>>>> appreciated.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> What about before the freeze?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Testing new features is of course important. This isn't
> meant
> > >>>> to
> > >>>>>>>>>>>> discourage
> > >>>>>>>>>>>>> development – only to enable us to focus on testing and
> > >>>> hardening
> > >>>>>>>>>> 4.0
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>> deliver Cassandra's most stable major release. We would
> like
> > >>>> to
> > >>>>>>> see
> > >>>>>>>>>>>>> adoption of 4.0 happen much more quickly than its
> > predecessor.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for considering this proposal,
> > >>>>>>>>>>>>> Sankalp Kohli
> > >>>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> Jon Haddad
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>
> >
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=l_G2ByhfCyu3k9TzNVqiagdVQ8vOMJqHZvDq_JKvbiQ&s=paSngQpMm3DhoWay8lDuWEYELVOrti8evQeT1LodXdY&e=
> > >>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> twitter: rustyrazorblade
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > ---------------------------------------------------------------------
> > >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> > >>>>>>> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
> > >>>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Jon Haddad
> > >>>>
> >
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rustyrazorblade.com&d=DwIFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=qK2RkRAsGtixYf0IgKlRBYLfTrXyOKED9OOTyMVvDf4&m=4uhQot00l-PuEymqD360tNN_Nhl7Uy9JH3ch4SiWc3I&s=cKnTet4KURi1yMqHlSk4FGIQJIP9jys3E8-6zfDjoNo&e=
> > >>>> twitter: rustyrazorblade
> > >>>>
> > >>>>
> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> > >>>> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
> > >>>>
> > >>>>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> > >> For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
> > >>
> > >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxxxxxx
> > For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxxxxxx
>