osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Proposal] Create new sub project for journaled messaging


Hi JB,

the idea is not to replace kafka. Instead we want to provide an API that
can abstract kafka as well as other journal based pub sub systems.
So actually one implementation of the api would bridge to kafka. We also
plan a mongo and a in memory impl.

One use case for the api would be a new EventAdmin with history. Another is
logging that ensures no messages are lost at startup even if the real
backend appears later.

Christian

Am Sa., 22. Dez. 2018 um 13:13 Uhr schrieb Jean-Baptiste Onofré <
jb@xxxxxxxxxxxx>:

> Hi Christian,
>
> I'm not sure I got the scope.
>
> Is it a new pub/sub system like Kafka or Google PubSub (or even
> ActiveMQ/AmazonMQ) ?
> Is it something like eventadmin on cloud (like we do in Cellar with
> Hazelcast) ?
>
> If it's just a pure pub/sub, I don't see a huge value compared to Kafka.
>
> About a spec, if we only have one impl of the spec, and the spec is
> mainly "focused" on the current impl, not sure it makes sense as well.
>
> Thoughts ?
>
> Regards
> JB
>
> On 22/12/2018 09:38, Christian Schneider wrote:
> > Some Background
> >
> > At Adobe I and two colleagues Timothee Maret and Alexei Krainiouk work on
> > making the AEM content publishing fit for the cloud. It is about
> > transporting content from author instances that are only visible to a
> > customer to outside facing instances that handle the load. For this
> effort
> > it is important to have a reliable pub/sub messaging with support of a
> > history where each consumer can decide where to start consuming from.
> >
> > We found that Apache Kafka is a good fit function wise but a little heavy
> > regarding deployment and management. So to be more versatile we thought
> > about encapsulating the messaging using an API and providing
> > implementations for different backends.
> > Unfortunately there is no existing API that solves this.
> >
> > Use case and API sketch
> >
> > So we created a description of the use cases as well as a sketch for a
> > minimal API.
> > See https://github.com/cschneider/journaled-events/blob/master/README.md
> >
> > Proposal
> >
> > We would like to develop this API as well as implementations as a sub
> > project at Apache Aries.
> > The main reason for choosing Aries is that David told us that there is
> > interest in a OSGi spec about a similar purpose. So we might also bring
> > this into an OSGi spec.
> > Alexei and Timothee are not yet committers at Aries. I think we can work
> > with them using the normal contribution model for the start and make them
> > committers after a few PRs.
> >
> > So what do you think?
> > As a first step I would like to discuss if the Aries PMC is interested in
> > giving this effort a home.  After that we can discuss the actual API as
> > well as possible usages and backends.
> >
> > Cheers,
> > Christian
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@xxxxxxxxxx
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>


-- 
-- 
Christian Schneider
http://www.liquid-reality.de

Computer Scientist
http://www.adobe.com