osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Calcite Streaming and Mongo


That will be cool. It can also be used as an example for other projects to
use Calcite streaming. I would imagine you need to translate the SQL
through the calcite logical plan to a set of pipeline operators and
pipeline stages in mongo. You can refer to projects like Flink, Samza or
Storm for referenece. Thanks.

Shuyi

On Wed, Oct 3, 2018 at 12:16 PM Michael Mior <mmior@xxxxxxxxxx> wrote:

> That would be great and an awesome example of an adapter using streaming. I
> haven't looked closely at Mongo's change streams, but you'd have to see if
> the semantics of the stream are suitable for exposing existing tables also
> as streams or if it would make more sense to separate streams and tables.
>
> --
> Michael Mior
> mmior@xxxxxxxxxx
>
>
> Le mar. 2 oct. 2018 à 20:43, Andrei Sereda <andrei@xxxxxxxxx> a écrit :
>
> > Hello,
> >
> > Since version 3.6 mongo introduced collection change streams
> > <https://docs.mongodb.com/v3.6/changeStreams/> which allow to observe
> > real-time changes of a collection. This looks very accommodating to
> calcite
> > streams <https://calcite.apache.org/docs/stream.html> which provides
> more
> > complex windowing and grouping operations.
> >
> > Do you think it makes sense to support streams in Mongo Adapter ? Would
> you
> > be interested in such feature ?
> >
> > Thanks in advance,
> > Andrei.
> >
>


-- 
"So you have to trust that the dots will somehow connect in your future."