Re: Splitting the repo
The purpose is that we have a monolithic core today mostly providing
The idea is to have something more API oriented with interface/SPI.
Our users would then be able to pick the part of the core they want,
resulting with lighter artifacts, and for us, it gives a more flexible
On 10/10/2018 10:26, Robert Bradshaw wrote:
> My question was not whether we should split the repo, but why? (Dividing
> things into more (or fewer) modules withing a single repo is a separate
> question.) Maybe I'm just not following what you mean by "more API
> oriented." It would force stabler APIs.
> On Wed, Oct 10, 2018 at 10:18 AM Jean-Baptiste Onofré <jb@xxxxxxxxxxxx
> <mailto:jb@xxxxxxxxxxxx>> wrote:
> +1, even I think we could split the core even deeper.
> I discussed with Luke and Reuven to introduce core-sql, core-schema,
> core-sdf, ...
> It's not a huge effort, and would allow us to move forward on Beam "more
> API oriented" approach.
> On 10/10/2018 10:12, Robert Bradshaw wrote:
> > Hi everyone,
> > While IMHO it's too early to even be able to split the repo, it's
> not to
> > early to talk about it, and I wanted to spin this off to keep the
> > thread focused.
> > In particular, I am trying to figure out exactly what is hoped to be
> > gained by splitting things up. In my experience, a single project that
> > spans multiple repos has always come with excessive overhead and pain.
> > Of note, we recently merged the website and dataflow-worker into the
> > main repo *exactly* to avoid this pain (though the latter was
> > particularly bad due to one of the repos being private).
> > If need be, I don't see any reason we can't have a single repo with
> > directories
> > model/
> > website/
> > java/
> > go/
> > ...
> > possibly even with their own build system (unified only through a
> > top-level "build everything" script that descends into each subdir and
> > runs the appropriate command). I'm not saying we should do this (there
> > is value in having a single consistent build system, etc.) but it's
> > possible. We could probably even make separate releases out of this
> > single repo (if we wanted, though given that our releases are
> > rather than feature-based, I don't see much advantage here).
> > Also, there was the comment.
> > On Wed, Oct 10, 2018 at 7:35 AM Romain Manni-Bucau
> > <rmannibucau@xxxxxxxxx <mailto:rmannibucau@xxxxxxxxx>
> <mailto:rmannibucau@xxxxxxxxx <mailto:rmannibucau@xxxxxxxxx>>> wrote:
> >> Side note: beam portability would be saner if added on top of others
> > than the opposite which is done today.
> > I think you brought this up before, Romain. I'm still trying to
> wrap my
> > head around what you mean here. Could you elaborate what such a
> > structure would look like?
> Jean-Baptiste Onofré
> jbonofre@xxxxxxxxxx <mailto:jbonofre@xxxxxxxxxx>
> Talend - http://www.talend.com
Talend - http://www.talend.com