osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Proposal] Create new sub project for journaled messaging


I see, thanks for the details.

If you think there's several use cases and adoption, I agree with a new
sub-project and spec.

I would be more than happy to help, I'm working a lot with Kafka and
Zookeeper recently (especially new features coming in ActiveMQ).

Regards
JB

On 22/12/2018 14:46, Christian Schneider wrote:
> Decanter is based in EventAdmin. I like the nice decoupling EventAdmin
> gives us for decantr but it is limited by EventAdmin's deficiencies.
> For example any Events sent before a subscriber arrives are lost.
> The main advantage of the API compared to simple pub/sub is that you can go
> back in the history (like in kafka).
> 
> So I think either the API directly or an EventAdmin based on the jounaled
> messaging API could be very valuable for Decanter to make sure no startup
> messages are lost.
> 
> Christian
> 
> Am Sa., 22. Dez. 2018 um 14:34 Uhr schrieb Jean-Baptiste Onofré <
> jb@xxxxxxxxxxxx>:
> 
>> OK, it's clear now.
>>
>> It sounds a bit like Decanter (EventAdmin sent to a storage backend), no ?
>>
>> Regards
>> JB
>>
>> On 22/12/2018 14:30, Christian Schneider wrote:
>>> Hi JB,
>>>
>>> the idea is not to replace kafka. Instead we want to provide an API that
>>> can abstract kafka as well as other journal based pub sub systems.
>>> So actually one implementation of the api would bridge to kafka. We also
>>> plan a mongo and a in memory impl.
>>>
>>> One use case for the api would be a new EventAdmin with history. Another
>> is
>>> logging that ensures no messages are lost at startup even if the real
>>> backend appears later.
>>>
>>> Christian
>>>
>>> Am Sa., 22. Dez. 2018 um 13:13 Uhr schrieb Jean-Baptiste Onofré <
>>> jb@xxxxxxxxxxxx>:
>>>
>>>> Hi Christian,
>>>>
>>>> I'm not sure I got the scope.
>>>>
>>>> Is it a new pub/sub system like Kafka or Google PubSub (or even
>>>> ActiveMQ/AmazonMQ) ?
>>>> Is it something like eventadmin on cloud (like we do in Cellar with
>>>> Hazelcast) ?
>>>>
>>>> If it's just a pure pub/sub, I don't see a huge value compared to Kafka.
>>>>
>>>> About a spec, if we only have one impl of the spec, and the spec is
>>>> mainly "focused" on the current impl, not sure it makes sense as well.
>>>>
>>>> Thoughts ?
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 22/12/2018 09:38, Christian Schneider wrote:
>>>>> Some Background
>>>>>
>>>>> At Adobe I and two colleagues Timothee Maret and Alexei Krainiouk work
>> on
>>>>> making the AEM content publishing fit for the cloud. It is about
>>>>> transporting content from author instances that are only visible to a
>>>>> customer to outside facing instances that handle the load. For this
>>>> effort
>>>>> it is important to have a reliable pub/sub messaging with support of a
>>>>> history where each consumer can decide where to start consuming from.
>>>>>
>>>>> We found that Apache Kafka is a good fit function wise but a little
>> heavy
>>>>> regarding deployment and management. So to be more versatile we thought
>>>>> about encapsulating the messaging using an API and providing
>>>>> implementations for different backends.
>>>>> Unfortunately there is no existing API that solves this.
>>>>>
>>>>> Use case and API sketch
>>>>>
>>>>> So we created a description of the use cases as well as a sketch for a
>>>>> minimal API.
>>>>> See
>> https://github.com/cschneider/journaled-events/blob/master/README.md
>>>>>
>>>>> Proposal
>>>>>
>>>>> We would like to develop this API as well as implementations as a sub
>>>>> project at Apache Aries.
>>>>> The main reason for choosing Aries is that David told us that there is
>>>>> interest in a OSGi spec about a similar purpose. So we might also bring
>>>>> this into an OSGi spec.
>>>>> Alexei and Timothee are not yet committers at Aries. I think we can
>> work
>>>>> with them using the normal contribution model for the start and make
>> them
>>>>> committers after a few PRs.
>>>>>
>>>>> So what do you think?
>>>>> As a first step I would like to discuss if the Aries PMC is interested
>> in
>>>>> giving this effort a home.  After that we can discuss the actual API as
>>>>> well as possible usages and backends.
>>>>>
>>>>> Cheers,
>>>>> Christian
>>>>>
>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@xxxxxxxxxx
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>>>
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@xxxxxxxxxx
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
> 
> 

-- 
Jean-Baptiste Onofré
jbonofre@xxxxxxxxxx
http://blog.nanthrax.net
Talend - http://www.talend.com