osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Drop 0. from the version


I start to think that Druid versioning doesn't need to be semantic and
doesn't need a distinction between major and minor. A release is just a
release, like IntelliJ's 2018.1, 2018.2, 2018.3. Every release could have
some extensions API broken. If there is an operations breaking change, we
could commit to support rollback strategies e. g. for 1 year of releases
after that, that means in the next four releases (if we stick to 3-month
cadence).

On Fri, 21 Dec 2018 at 19:10, Charles Allen <crallen@xxxxxxxxxx> wrote:

> If I'm greedily honest, I don't want to maintain that many backport
> channels. I'd rather have "If you want XYZ backport for version 14, then
> you have to take the latest minor version for 14" and have a policy to
> where someone can upgrade from 14.x --> 14.latest with (hopefully) no
> config changes.
>
>
>
>
> On Fri, Dec 21, 2018 at 9:03 AM David Glasser <glasser@xxxxxxxxxxxxxxxxx>
> wrote:
>
> > One nice advantage to moving out of 0.x is that it frees up a digit on
> the
> > right side to more cleanly differentiate between "minor release (a random
> > assortment of bug fixes, small features, etc)" and "patch release
> > (literally the minimum delta to give you a security fix or high impact
> bug
> > fix)".
> >
> > --dave
> >
> > On Fri, Dec 21, 2018 at 8:58 AM Gian Merlino <gian@xxxxxxxxxx> wrote:
> >
> > > I'm not too fussy around whether we do a 1.0 or simply drop the 0. and
> > have
> > > it be a 14.0 or 15.0 or 16.0 or wherever we are at the time we do it. I
> > > also like the quarterly cadence of release-from-master we had before we
> > got
> > > blocked on the ASF transition, and would like to pick that back up
> again
> > > (with the next branch cut from master at the end of January, since we
> did
> > > the 0.13.0 branch cut in late October).
> > >
> > > Seems to me that good points of discussion are, what should we use as
> the
> > > rule for incrementing the major version? Do we do what we've been doing
> > > (incrementing whenever there's either an incompatible change in
> extension
> > > APIs, or in query APIs, or when necessary to preserve the ability to
> > always
> > > be able to roll forward/back one major release). Or do we do something
> > else
> > > (Roman seems to be suggesting dropping extension APIs from
> > consideration).
> > >
> > > And also, what does 1.0 or 14.0 or 15.0 or what-have-you mean to us? Is
> > it
> > > something that should be tied to ASF graduation? Completeness of
> vision?
> > > Stability of APIs or operational characteristics? Something else? You
> are
> > > right that it is sort of a marketing/mentality thing, so it's an
> > > opportunity for us to declare that we feel Druid has reached some
> > > milestone. My feeling at this time is probably ASF graduation or
> > > completeness of vision (see my earlier mail for thoughts there) are the
> > > ones that make most sense to me.
> > >
> > > On Fri, Dec 21, 2018 at 10:41 AM Charles Allen <crallen@xxxxxxxxxx>
> > wrote:
> > >
> > > > Is there any feeling in the community that the logic behind the
> > releases
> > > > needs to change?
> > > >
> > > > If so then I think we should discuss what that release cadence needs
> to
> > > > look like.
> > > >
> > > > If not then dropping the 0. prefix is a marketing / mental item. Kind
> > of
> > > > like the 3.x->4.x Linux kernel upgrade. If this is the case then
> would
> > we
> > > > even want to go with 1.x? I think Roman's proposal would work fine in
> > > this
> > > > case. Where we just call it Apache Druid 14 (or 15 or whatever it is
> > when
> > > > we get there) and just keep the same logic for when we release stuff,
> > > which
> > > > has been something like:
> > > >
> > > > For a X.Y release, going to a X.? release should be very straight
> > forward
> > > > for anyone running stock Druid.
> > > > For a X.Y release, going to a (X+1).? or from a (X+1).? back to an
> X.Y
> > > > release should be feasible. It might require running a tool supported
> > by
> > > > the community.
> > > > For a X.Y release, going to an (X+2).? or an (X-2).? is not
> supported.
> > > Some
> > > > things that will not have tools might have warning logs printed that
> > the
> > > > functionality will change (should we change these to alerts?)
> > > >
> > > > If this sounds reasonable then jumping straight to Apache Druid 14 on
> > the
> > > > first official apache release would make a lot of sense.
> > > >
> > > > Cheers,
> > > > Charles Allen
> > > >
> > > >
> > > > On Thu, Dec 20, 2018 at 11:07 PM Gian Merlino <gian@xxxxxxxxxx>
> wrote:
> > > >
> > > > > I think it's a good point. Culturally we have been willing to break
> > > > > extension APIs for relatively small benefits. But we have generally
> > > been
> > > > > unwilling to make breaking changes on the operations side quite so
> > > > > liberally. Also, most cluster operators don't have their own custom
> > > > > extensions, in my experience. So it does make sense to
> differentiate
> > > > them.
> > > > > I'm not sure how it makes sense to differentiate them, though. It
> > could
> > > > be
> > > > > done through the version number (only increment the major version
> for
> > > > > operations breaking changes) or it could be done through an
> > "upgrading"
> > > > > guide in the documentation (increment the major version for
> > operations
> > > or
> > > > > extension breaking changes, but, have a guide that tells people
> which
> > > > > versions have operations breaking changes to aid in upgrades).
> > > > >
> > > > > Coming back to the question in the subject of your mail: IMO, for
> > > > > "graduation" out of 0.x, we should talk as a community about what
> > that
> > > > > means to us. It is a milestone that on the one hand, doesn't mean
> > much,
> > > > but
> > > > > on the other hand, can be deeply symbolic. Some things that it has
> > > meant
> > > > to
> > > > > other projects:
> > > > >
> > > > > 1) Production readiness. Obviously Druid is well past this. If this
> > is
> > > > what
> > > > > dropping the 0. means, then we should do it immediately.
> > > > >
> > > > > 2) Belief that the APIs have become relatively stable. Like you
> said,
> > > the
> > > > > extension APIs don't seem particularly close to stable, but maybe
> > > that's
> > > > > okay. However, the pace of breaking changes on the operations and
> > query
> > > > > side for non-experimental features has been relatively calm for the
> > > past
> > > > > couple of years, so if we focus on that then we can make a case
> here.
> > > > >
> > > > > 3) Completeness of vision. This one is the most interesting to me.
> I
> > > > > suspect that different people in the community have different
> visions
> > > for
> > > > > Druid. It is also the kind of project that may never truly be
> > complete
> > > in
> > > > > vision (in principle, the platform could become a competitive data
> > > > > warehouse, search engine, etc, …). For what it's worth, my vision
> of
> > > > Druid
> > > > > for the next year at least involves robust stream ingestion being a
> > > first
> > > > > class ingestion method (Kafka / Kinesis indexing service style) and
> > SQL
> > > > > being a first class query language. These are both, today, still
> > > > > experimental features. So are lookups. All of these 3 features,
> from
> > > > what I
> > > > > can see, are quite popular amongst Druid users despite being
> > > > experimental.
> > > > > For a 'completeness of vision' based 1.0 I would want to lift all
> of
> > > > those
> > > > > out of experimental status and, for SQL in particular, to have its
> > > > > functionality rounded out a bit more (to support the native query
> > > > features
> > > > > it doesn't currently support, like multi-value dimensions,
> > > datasketches,
> > > > > etc).
> > > > >
> > > > > 4) Marketing / timing. Like, doing a 1.0 around the time we
> graduate
> > > from
> > > > > the Incubator. Not sure how much this really matters, but projects
> do
> > > it
> > > > > sometimes.
> > > > >
> > > > > Another question is, how often do we intend to rev the version? At
> > the
> > > > rate
> > > > > we're going, we rev 2-3 major versions a year. Would we intend to
> > keep
> > > > that
> > > > > up, or slow it down by making more of an effort to avoid breaking
> > > > changes?
> > > > >
> > > > > On Thu, Dec 20, 2018 at 2:17 PM Roman Leventov <
> > leventov.ru@xxxxxxxxx>
> > > > > wrote:
> > > > >
> > > > > > It may also make sense to distinguish "operations" breaking
> changes
> > > > from
> > > > > > API breaking changes. Operations breaking changes establish the
> > > minimum
> > > > > > cadence of Druid cluster upgrades, that allow rolling Druid
> > versions
> > > > back
> > > > > > and forward. I. e. it's related to segment format, the format of
> > the
> > > > data
> > > > > > kept in ZooKeeper and the SQL database, or events such as
> stopping
> > > > > support
> > > > > > of ZooKeeper for certain things (e. g. forcing using of HTTP
> > > > > > announcements). So Druid cluster operators cannot update Druid
> from
> > > > > version
> > > > > > X to version Z skipping the version Y, if both Y and Z have some
> > > > > operations
> > > > > > breaking changes. (Any such changes should support rollback
> options
> > > at
> > > > > > least until the next version with operations breaking changes.)
> > > > > >
> > > > > > API breaking changes are just changes in Druid extensions APIs.
> > Druid
> > > > > > cluster operators could skip any number of releases with such
> > > breaking
> > > > > > changes, as long as their extension's code is updated for the
> > latest
> > > > > > version of API.
> > > > > >
> > > > > > On Thu, 20 Dec 2018 at 20:03, Roman Leventov <
> leventov@xxxxxxxxxx>
> > > > > wrote:
> > > > > >
> > > > > > > It doesn't seem to me that Druid API is going to stabilize in
> the
> > > > near
> > > > > > > future (if ever), because there are so many extension points
> and
> > > > > > something
> > > > > > > is broken in every release. On the other hand, Druid is not
> > Hadoop
> > > or
> > > > > > > Spark, which have applications API. Druid API for extensions,
> not
> > > > > > > applications. It is used by people who are closer to Druid
> > > > development
> > > > > > and
> > > > > > > fixing their extensions is routine.
> > > > > > >
> > > > > > > With that, I think it make sense to drop "0." from the Druid
> > > version
> > > > > and
> > > > > > > call it Druid 14, Druid 15, etc.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>