OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Druid as a OLTP solution


Shushant,

Gian is definitely correct that Druid is not designed for OLTP.

However the scenario you describe does not sound like OLTP. You talk of querying data, but you don’t mention inserting/updating individual records with low latency, which is a hallmark of OLTP, and something that Druid is not good at.

One thing Druid is very good at is near-realtime OLAP. If you have a transactional system and a means (such as Kafka) to flow the inserts to that transactional system into Druid with low latency, then Druid can perform queries on a snapshot of your transactional system that is only slightly delayed from reality. If that is your scenario, it is worth looking more closely at Druid.

Julian


> On Aug 19, 2018, at 6:30 PM, Gian Merlino <gian@xxxxxxxxxx> wrote:
> 
> Hi Shushant,
> 
> Druid is definitely not intended for OLTP processing. It's generally meant
> for storing, querying, and analyzing event streams. Check out
> http://druid.io/ for some more info about what Druid is good at.
> 
> On Sat, Aug 18, 2018 at 10:19 PM Shushant Arora <shushantarora09@xxxxxxxxx>
> wrote:
> 
>> Hi
>> 
>> I have a requirement where I have a large scale data and each record has a
>> mandatory visitorId field and one or more optional dimension
>> and records of same visitorId can come after few days time gap.So I have
>> records in system like
>> visitorId1,Timestamp1,Dim1=val1
>> visitorId1,Timestamp1+20days,Dim2=val2
>> 
>> Now I have a requirement to count distinct visitorIds where Dim1=val1 and
>> Dim2=val2(i.,e any row of same visitor has Dim1=val1 and any row of same
>> vsisitor has Dim2=val2).
>> 
>> 
>> Can I do grpupBy visitorId and filter on 2 dimensions Dim=val1 and
>> Dim2=val2 both being in separate row and count on visitorId .
>> Since druid needs Timestamp in each event so I used currentTs for that.
>> Will this return result as 1.
>> 
>> Or druid is not for OLTP processimg?
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: dev-help@xxxxxxxxxxxxxxxx