[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Splitting the repo

On Wed, Oct 10, 2018 at 10:35 AM Romain Manni-Bucau <rmannibucau@xxxxxxxxx> wrote:
Also we can get a more adapted build tool by area and not break the repo for each build. Go and python build always need a git clean for java users which is a big issue so let's build each subproject - that is what beam is today - as they should with an adapted tool.

If this is the case, that should be fixed. I can't remember the last time I did a git clean, so clearly things are not working as well for you as for I. 
It requires very few validations byt it is trivial to add unit tests to ensure it is not broken on these contact points.

Le mer. 10 oct. 2018 11:29, Jean-Baptiste Onofré <jb@xxxxxxxxxxxx> a écrit :
The purpose is that we have a monolithic core today mostly providing
abstract classes.

The idea is to have something more API oriented with interface/SPI.

Our users would then be able to pick the part of the core they want,
resulting with lighter artifacts, and for us, it gives a more flexible


On 10/10/2018 10:26, Robert Bradshaw wrote:
> My question was not whether we should split the repo, but why? (Dividing
> things into more (or fewer) modules withing a single repo is a separate
> question.) Maybe I'm just not following what you mean by "more API
> oriented." It would force stabler APIs. 
> On Wed, Oct 10, 2018 at 10:18 AM Jean-Baptiste Onofré <jb@xxxxxxxxxxxx
> <mailto:jb@xxxxxxxxxxxx>> wrote:
>     Hi,
>     +1, even I think we could split the core even deeper.
>     I discussed with Luke and Reuven to introduce core-sql, core-schema,
>     core-sdf, ...
>     It's not a huge effort, and would allow us to move forward on Beam "more
>     API oriented" approach.
>     Regards
>     JB
>     On 10/10/2018 10:12, Robert Bradshaw wrote:
>     > Hi everyone,
>     >
>     > While IMHO it's too early to even be able to split the repo, it's
>     not to
>     > early to talk about it, and I wanted to spin this off to keep the
>     other
>     > thread focused.
>     >
>     > In particular, I am trying to figure out exactly what is hoped to be
>     > gained by splitting things up. In my experience, a single project that
>     > spans multiple repos has always come with excessive overhead and pain.
>     > Of note, we recently merged the website and dataflow-worker into the
>     > main repo *exactly* to avoid this pain (though the latter was
>     > particularly bad due to one of the repos being private).
>     >
>     > If need be, I don't see any reason we can't have a single repo with
>     > directories
>     >
>     > model/
>     > website/
>     > java/
>     > go/
>     > ...
>     >
>     > possibly even with their own build system (unified only through a
>     > top-level "build everything" script that descends into each subdir and
>     > runs the appropriate command). I'm not saying we should do this (there
>     > is value in having a single consistent build system, etc.) but it's
>     > possible. We could probably even make separate releases out of this
>     > single repo (if we wanted, though given that our releases are
>     time-based
>     > rather than feature-based, I don't see much advantage here).
>     >
>     > Also, there was the comment.
>     >
>     > On Wed, Oct 10, 2018 at 7:35 AM Romain Manni-Bucau
>     > <rmannibucau@xxxxxxxxx <mailto:rmannibucau@xxxxxxxxx>
>     <mailto:rmannibucau@xxxxxxxxx <mailto:rmannibucau@xxxxxxxxx>>> wrote:
>     >>
>     >> Side note: beam portability would be saner if added on top of others
>     > than the opposite which is done today.
>     >
>     > I think you brought this up before, Romain. I'm still trying to
>     wrap my
>     > head around what you mean here. Could you elaborate what such a
>     > structure would look like? 
>     --
>     Jean-Baptiste Onofré
>     jbonofre@xxxxxxxxxx <mailto:jbonofre@xxxxxxxxxx>
>     http://blog.nanthrax.net
>     Talend - http://www.talend.com

Jean-Baptiste Onofré
Talend - http://www.talend.com