Re: Calcite Streaming and Mongo
That will be cool. It can also be used as an example for other projects to
use Calcite streaming. I would imagine you need to translate the SQL
through the calcite logical plan to a set of pipeline operators and
pipeline stages in mongo. You can refer to projects like Flink, Samza or
Storm for referenece. Thanks.
On Wed, Oct 3, 2018 at 12:16 PM Michael Mior <mmior@xxxxxxxxxx> wrote:
> That would be great and an awesome example of an adapter using streaming. I
> haven't looked closely at Mongo's change streams, but you'd have to see if
> the semantics of the stream are suitable for exposing existing tables also
> as streams or if it would make more sense to separate streams and tables.
> Michael Mior
> Le mar. 2 oct. 2018 à 20:43, Andrei Sereda <andrei@xxxxxxxxx> a écrit :
> > Hello,
> > Since version 3.6 mongo introduced collection change streams
> > <https://docs.mongodb.com/v3.6/changeStreams/> which allow to observe
> > real-time changes of a collection. This looks very accommodating to
> > streams <https://calcite.apache.org/docs/stream.html> which provides
> > complex windowing and grouping operations.
> > Do you think it makes sense to support streams in Mongo Adapter ? Would
> > be interested in such feature ?
> > Thanks in advance,
> > Andrei.
"So you have to trust that the dots will somehow connect in your future."