osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Proposal] Create new sub project for journaled messaging


Your help is more than welcome.

I will start a new thread to discuss the interface and adapters/impls.

Christian

Am Sa., 22. Dez. 2018 um 15:11 Uhr schrieb Jean-Baptiste Onofré <
jb@xxxxxxxxxxxx>:

> I see, thanks for the details.
>
> If you think there's several use cases and adoption, I agree with a new
> sub-project and spec.
>
> I would be more than happy to help, I'm working a lot with Kafka and
> Zookeeper recently (especially new features coming in ActiveMQ).
>
> Regards
> JB
>
> On 22/12/2018 14:46, Christian Schneider wrote:
> > Decanter is based in EventAdmin. I like the nice decoupling EventAdmin
> > gives us for decantr but it is limited by EventAdmin's deficiencies.
> > For example any Events sent before a subscriber arrives are lost.
> > The main advantage of the API compared to simple pub/sub is that you can
> go
> > back in the history (like in kafka).
> >
> > So I think either the API directly or an EventAdmin based on the jounaled
> > messaging API could be very valuable for Decanter to make sure no startup
> > messages are lost.
> >
> > Christian
> >
> > Am Sa., 22. Dez. 2018 um 14:34 Uhr schrieb Jean-Baptiste Onofré <
> > jb@xxxxxxxxxxxx>:
> >
> >> OK, it's clear now.
> >>
> >> It sounds a bit like Decanter (EventAdmin sent to a storage backend),
> no ?
> >>
> >> Regards
> >> JB
> >>
> >> On 22/12/2018 14:30, Christian Schneider wrote:
> >>> Hi JB,
> >>>
> >>> the idea is not to replace kafka. Instead we want to provide an API
> that
> >>> can abstract kafka as well as other journal based pub sub systems.
> >>> So actually one implementation of the api would bridge to kafka. We
> also
> >>> plan a mongo and a in memory impl.
> >>>
> >>> One use case for the api would be a new EventAdmin with history.
> Another
> >> is
> >>> logging that ensures no messages are lost at startup even if the real
> >>> backend appears later.
> >>>
> >>> Christian
> >>>
> >>> Am Sa., 22. Dez. 2018 um 13:13 Uhr schrieb Jean-Baptiste Onofré <
> >>> jb@xxxxxxxxxxxx>:
> >>>
> >>>> Hi Christian,
> >>>>
> >>>> I'm not sure I got the scope.
> >>>>
> >>>> Is it a new pub/sub system like Kafka or Google PubSub (or even
> >>>> ActiveMQ/AmazonMQ) ?
> >>>> Is it something like eventadmin on cloud (like we do in Cellar with
> >>>> Hazelcast) ?
> >>>>
> >>>> If it's just a pure pub/sub, I don't see a huge value compared to
> Kafka.
> >>>>
> >>>> About a spec, if we only have one impl of the spec, and the spec is
> >>>> mainly "focused" on the current impl, not sure it makes sense as well.
> >>>>
> >>>> Thoughts ?
> >>>>
> >>>> Regards
> >>>> JB
> >>>>
> >>>> On 22/12/2018 09:38, Christian Schneider wrote:
> >>>>> Some Background
> >>>>>
> >>>>> At Adobe I and two colleagues Timothee Maret and Alexei Krainiouk
> work
> >> on
> >>>>> making the AEM content publishing fit for the cloud. It is about
> >>>>> transporting content from author instances that are only visible to a
> >>>>> customer to outside facing instances that handle the load. For this
> >>>> effort
> >>>>> it is important to have a reliable pub/sub messaging with support of
> a
> >>>>> history where each consumer can decide where to start consuming from.
> >>>>>
> >>>>> We found that Apache Kafka is a good fit function wise but a little
> >> heavy
> >>>>> regarding deployment and management. So to be more versatile we
> thought
> >>>>> about encapsulating the messaging using an API and providing
> >>>>> implementations for different backends.
> >>>>> Unfortunately there is no existing API that solves this.
> >>>>>
> >>>>> Use case and API sketch
> >>>>>
> >>>>> So we created a description of the use cases as well as a sketch for
> a
> >>>>> minimal API.
> >>>>> See
> >> https://github.com/cschneider/journaled-events/blob/master/README.md
> >>>>>
> >>>>> Proposal
> >>>>>
> >>>>> We would like to develop this API as well as implementations as a sub
> >>>>> project at Apache Aries.
> >>>>> The main reason for choosing Aries is that David told us that there
> is
> >>>>> interest in a OSGi spec about a similar purpose. So we might also
> bring
> >>>>> this into an OSGi spec.
> >>>>> Alexei and Timothee are not yet committers at Aries. I think we can
> >> work
> >>>>> with them using the normal contribution model for the start and make
> >> them
> >>>>> committers after a few PRs.
> >>>>>
> >>>>> So what do you think?
> >>>>> As a first step I would like to discuss if the Aries PMC is
> interested
> >> in
> >>>>> giving this effort a home.  After that we can discuss the actual API
> as
> >>>>> well as possible usages and backends.
> >>>>>
> >>>>> Cheers,
> >>>>> Christian
> >>>>>
> >>>>
> >>>> --
> >>>> Jean-Baptiste Onofré
> >>>> jbonofre@xxxxxxxxxx
> >>>> http://blog.nanthrax.net
> >>>> Talend - http://www.talend.com
> >>>>
> >>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@xxxxxxxxxx
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@xxxxxxxxxx
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>


-- 
-- 
Christian Schneider
http://www.liquid-reality.de

Computer Scientist
http://www.adobe.com