osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Splitting the repo


The purpose is that we have a monolithic core today mostly providing
abstract classes.

The idea is to have something more API oriented with interface/SPI.

Our users would then be able to pick the part of the core they want,
resulting with lighter artifacts, and for us, it gives a more flexible
approach.

Regards
JB

On 10/10/2018 10:26, Robert Bradshaw wrote:
> My question was not whether we should split the repo, but why? (Dividing
> things into more (or fewer) modules withing a single repo is a separate
> question.) Maybe I'm just not following what you mean by "more API
> oriented." It would force stabler APIs. 
> 
> On Wed, Oct 10, 2018 at 10:18 AM Jean-Baptiste Onofré <jb@xxxxxxxxxxxx
> <mailto:jb@xxxxxxxxxxxx>> wrote:
> 
>     Hi,
> 
>     +1, even I think we could split the core even deeper.
> 
>     I discussed with Luke and Reuven to introduce core-sql, core-schema,
>     core-sdf, ...
> 
>     It's not a huge effort, and would allow us to move forward on Beam "more
>     API oriented" approach.
> 
>     Regards
>     JB
> 
>     On 10/10/2018 10:12, Robert Bradshaw wrote:
>     > Hi everyone,
>     >
>     > While IMHO it's too early to even be able to split the repo, it's
>     not to
>     > early to talk about it, and I wanted to spin this off to keep the
>     other
>     > thread focused.
>     >
>     > In particular, I am trying to figure out exactly what is hoped to be
>     > gained by splitting things up. In my experience, a single project that
>     > spans multiple repos has always come with excessive overhead and pain.
>     > Of note, we recently merged the website and dataflow-worker into the
>     > main repo *exactly* to avoid this pain (though the latter was
>     > particularly bad due to one of the repos being private).
>     >
>     > If need be, I don't see any reason we can't have a single repo with
>     > directories
>     >
>     > model/
>     > website/
>     > java/
>     > go/
>     > ...
>     >
>     > possibly even with their own build system (unified only through a
>     > top-level "build everything" script that descends into each subdir and
>     > runs the appropriate command). I'm not saying we should do this (there
>     > is value in having a single consistent build system, etc.) but it's
>     > possible. We could probably even make separate releases out of this
>     > single repo (if we wanted, though given that our releases are
>     time-based
>     > rather than feature-based, I don't see much advantage here).
>     >
>     > Also, there was the comment.
>     >
>     > On Wed, Oct 10, 2018 at 7:35 AM Romain Manni-Bucau
>     > <rmannibucau@xxxxxxxxx <mailto:rmannibucau@xxxxxxxxx>
>     <mailto:rmannibucau@xxxxxxxxx <mailto:rmannibucau@xxxxxxxxx>>> wrote:
>     >>
>     >> Side note: beam portability would be saner if added on top of others
>     > than the opposite which is done today.
>     >
>     > I think you brought this up before, Romain. I'm still trying to
>     wrap my
>     > head around what you mean here. Could you elaborate what such a
>     > structure would look like? 
> 
>     -- 
>     Jean-Baptiste Onofré
>     jbonofre@xxxxxxxxxx <mailto:jbonofre@xxxxxxxxxx>
>     http://blog.nanthrax.net
>     Talend - http://www.talend.com
> 

-- 
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com