osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [EXTERNAL] Re: Tracing in cassandra


So, querying for a status column that is not part of the partition key will not scale – especially if there are only a few status codes. Every node in the cluster will be reading all the data in the table and trying to send to the coordinator node.

 

With some possibly erroneous assumptions on what you are trying to do, I would suggest looking at something more like:

 

Create table active_sessions

Primary key ((user, session),session_start_time DESC)

(perhaps using a TTL to expire data)

 

(And if you need the historical information)

Create table inactive_sessions

Primary key ((user, session), session_end_time DESC)

(also consider adding a time bucket to the primary key, if this will get large)

 

Much depends on volume, how you actually use the data, how long it can last, etc. But the place to start is to have a table for each query upon which you can execute at least a partition key query. Table scans in Cassandra (alone) are not the right access plan. (Spark is one way around this for certain analytics use cases.)

 

Sean Durity

lord of the (C*) rings (Staff Systems Engineer – Cassandra)

MTC 2250

#cassandra - for the latest news and updates

 

From: Abdul Patel <abd786.ap@xxxxxxxxx>
Sent: Friday, October 12, 2018 12:41 PM
To: user@xxxxxxxxxxxxxxxxxxxx
Subject: Re: [EXTERNAL] Re: Tracing in cassandra

 

Sean,

 

here it is :

CREATE TABLE Keyspave.tblname (

    user_id bigint,

    session_id text,

    application_guid text,

    last_access_time timestamp,

    login_time timestamp,

    status int,

    terminated_by text,

    update_time timestamp,

    PRIMARY KEY (user_id, session_id)

) WITH CLUSTERING ORDER BY (session_id ASC)

 

also they see timeouts with limit 11 as well, so is it better to remove with limit option ? or whats best to query such schema?

 

On Fri, Oct 12, 2018 at 11:05 AM Durity, Sean R <SEAN_R_DURITY@xxxxxxxxxxxxx> wrote:

Cross-partition = multiple partitions

 

Simple example:

Create table customer (

Customerid int,

Name text,

Lastvisit date,

Phone text,

Primary key (customerid) );

 

Query

Select customerid from customer limit 5000;

 

The query is asking for 5000 different partitions to be selected across the cluster. This is a very EXPENSIVE query for Cassandra, especially as the number of nodes goes up. Typically, you want to query a single partition. Read timeouts are usually caused by queries that are selecting many partitions or a very large partition. That is why a schema for the involved table could help.

 

 

Sean Durity

 

From: Abdul Patel <abd786.ap@xxxxxxxxx>
Sent: Friday, October 12, 2018 10:04 AM
To: user@xxxxxxxxxxxxxxxxxxxx
Subject: [EXTERNAL] Re: Tracing in cassandra

 

Cpuld you elaborate cross partition query?

On Friday, October 12, 2018, Durity, Sean R <SEAN_R_DURITY@xxxxxxxxxxxxx> wrote:

I suspect you are doing a cross-partition query, which will not scale well (as you can see). What is the schema for the table involved?

 

 

Sean Durity

 

From: Abdul Patel <abd786.ap@xxxxxxxxx>
Sent: Thursday, October 11, 2018 5:54 PM
To: adam@xxxxxxxxxxxxxxx
Cc: user@xxxxxxxxxxxxxxxxxxxx
Subject: [EXTERNAL] Re: Tracing in cassandra

 

Query :

SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; 

-Error: ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}

 

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bd7c0-cd9e-11e8-8e99-15807bff4dfd |                                                                    Parsing SELECT * FROM keysoace.tablenameWHERE user_id = 390797583 LIMIT 5000; | 10.54.145.32 |           4020 |                   Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70bfed0-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                               Preparing statement | 10.54.145.32 |           5065 |                   Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c25e0-cd9e-11e8-8e99-15807bff4dfd |                                                                                                         Executing single-partition query on roles | 10.54.145.32 |           6171 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf0-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                      Acquiring sstable references | 10.54.145.32 |           6362 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf1-cd9e-11e8-8e99-15807bff4dfd |                                                                         Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 |           6641 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf2-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                     Key cache hit for sstable 346 | 10.54.145.32 |           6955 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c4cf3-cd9e-11e8-8e99-15807bff4dfd |                                                                                                          Bloom filter allows skipping sstable 347 | 10.54.145.32 |           7202 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7400-cd9e-11e8-8e99-15807bff4dfd |                                                                                                         Merged data from memtables and 2 sstables | 10.54.145.32 |           7386 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7401-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                 Read 1 live and 0 tombstone cells | 10.54.145.32 |           7519 |                                   ReadStage-2

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7402-cd9e-11e8-8e99-15807bff4dfd |                                                                                                         Executing single-partition query on roles | 10.54.145.32 |           7826 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7403-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                      Acquiring sstable references | 10.54.145.32 |           7924 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7404-cd9e-11e8-8e99-15807bff4dfd |                                                                         Skipped 0/2 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 |           8060 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7405-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                     Key cache hit for sstable 346 | 10.54.145.32 |           8137 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7406-cd9e-11e8-8e99-15807bff4dfd |                                                                                                          Bloom filter allows skipping sstable 347 | 10.54.145.32 |           8187 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c7407-cd9e-11e8-8e99-15807bff4dfd |                                                                                                         Merged data from memtables and 2 sstables | 10.54.145.32 |           8318 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70c9b10-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                 Read 1 live and 0 tombstone cells | 10.54.145.32 |           8941 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc220-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                              Read-repair DC_LOCAL | 10.54.145.32 |           9468 |                   Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc221-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                   reading data from /10.54.145.31 | 10.54.145.32 |           9754 |                   Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70cc222-cd9e-11e8-8e99-15807bff4dfd |                                                                                                       Executing single-partition query on session | 10.54.145.32 |          10310 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce930-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                 reading digest from /10.54.101.56 | 10.54.145.32 |          10343 |                   Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce931-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                      Acquiring sstable references | 10.54.145.32 |          10356 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce932-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                   Key cache hit for sstable 68642 | 10.54.145.32 |          10445 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce933-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                 reading digest from /10.54.101.57 | 10.54.145.32 |          10480 |                   Native-Transport-Requests-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce934-cd9e-11e8-8e99-15807bff4dfd |                                                                                                        Bloom filter allows skipping sstable 68397 | 10.54.145.32 |          10496 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce935-cd9e-11e8-8e99-15807bff4dfd |                                                                         Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.32 |          10547 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce936-cd9e-11e8-8e99-15807bff4dfd |                                                                                                             Sending READ message to /10.54.145.31 | 10.54.145.32 |          10575 | MessagingService-Outgoing-/10.54.145.31-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce937-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                   Key cache hit for sstable 68655 | 10.54.145.32 |          10631 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce938-cd9e-11e8-8e99-15807bff4dfd |                                                                                                             Sending READ message to /10.54.101.56 | 10.54.145.32 |          10695 | MessagingService-Outgoing-/10.54.101.56-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce939-cd9e-11e8-8e99-15807bff4dfd |                                                                                                             Sending READ message to /10.54.101.57 | 10.54.145.32 |          10823 | MessagingService-Outgoing-/10.54.101.57-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93a-cd9e-11e8-8e99-15807bff4dfd |                                                                                                         Merged data from memtables and 3 sstables | 10.54.145.32 |          10863 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ce93b-cd9e-11e8-8e99-15807bff4dfd |                                                                                                                Read 13 live and 0 tombstone cells | 10.54.145.32 |          10914 |                                   ReadStage-4

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                          READ message received from /10.54.145.32 | 10.54.145.31 |            883 |       MessagingService-Incoming-/10.54.145.32

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dac80-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                          READ message received from /10.54.145.32 | 10.54.101.57 |            803 |       MessagingService-Incoming-/10.54.145.32

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dd390-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                          READ message received from /10.54.145.32 | 10.54.101.56 |            961 |       MessagingService-Incoming-/10.54.145.32

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                       Executing single-partition query on session | 10.54.145.31 |           3208 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa0-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                       Executing single-partition query on session | 10.54.101.57 |           3258 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                                      Acquiring sstable references | 10.54.145.31 |           3483 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70dfaa1-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                                      Acquiring sstable references | 10.54.101.57 |           3549 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                                   Key cache hit for sstable 74713 | 10.54.145.31 |           3732 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                                   Key cache hit for sstable 85409 | 10.54.101.57 |           3978 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b0-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                       Executing single-partition query on session | 10.54.101.56 |           3497 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                                   Key cache hit for sstable 74445 | 10.54.145.31 |           3893 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b1-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                        Bloom filter allows skipping sstable 85201 | 10.54.101.57 |           4193 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                        Bloom filter allows skipping sstable 74383 | 10.54.145.31 |           3975 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b2-cd9e-11e8-ba8b-bbf19da89f90 |                                                                         Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.57 |           4354 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                         Skipped 0/3 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.145.31 |           4142 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e21b3-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                                   Key cache hit for sstable 85418 | 10.54.101.57 |           4512 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                         Merged data from memtables and 3 sstables | 10.54.101.57 |           4989 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c0-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                                      Acquiring sstable references | 10.54.101.56 |           3751 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                                Read 13 live and 0 tombstone cells | 10.54.101.57 |           5336 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c1-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                                  Key cache hit for sstable 101147 | 10.54.101.56 |           4091 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e48c2-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                                Read 13 live and 0 tombstone cells | 10.54.101.57 |           5586 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd0-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                               Enqueuing response to /10.54.145.32 | 10.54.101.57 |           5745 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e6fd1-cd9e-11e8-ba8b-bbf19da89f90 |                                                                                                 Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.57 |           6473 | MessagingService-Outgoing-/10.54.145.32-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-8e99-15807bff4dfd |                                                                                              REQUEST_RESPONSE message received from /10.54.101.57 | 10.54.145.32 |             57 |       MessagingService-Incoming-/10.54.101.57

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                         Merged data from memtables and 3 sstables | 10.54.145.31 |           6600 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e0-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                                  Key cache hit for sstable 100896 | 10.54.101.56 |           6496 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-8e99-15807bff4dfd |                                                                                                            Processing response from /10.54.101.57 | 10.54.145.32 |            587 |                        RequestResponseStage-1

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e1-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                                Read 13 live and 0 tombstone cells | 10.54.145.31 |           6892 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70e96e2-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                               Enqueuing response to /10.54.145.32 | 10.54.145.31 |           7038 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-8e99-15807bff4dfd |                                                                                              REQUEST_RESPONSE message received from /10.54.145.31 | 10.54.145.32 |             15 |       MessagingService-Incoming-/10.54.145.31

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-9ad7-9f2cb164b2ba |                                                                                                 Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.145.31 |           7641 | MessagingService-Outgoing-/10.54.145.32-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf0-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                       Bloom filter allows skipping sstable 100887 | 10.54.101.56 |           6652 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-8e99-15807bff4dfd |                                                                                                            Processing response from /10.54.145.31 | 10.54.145.32 |             95 |                        RequestResponseStage-1

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf1-cd9e-11e8-bed7-f13ab36bfa4c |                                                                         Skipped 0/4 non-slice-intersecting sstables, included 0 due to tombstones | 10.54.101.56 |           6828 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ebdf2-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                                  Key cache hit for sstable 101149 | 10.54.101.56 |           7005 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f3320-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                         Merged data from memtables and 4 sstables | 10.54.101.56 |           9897 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8140-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                                Read 13 live and 0 tombstone cells | 10.54.101.56 |          11969 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8141-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                                Read 13 live and 0 tombstone cells | 10.54.101.56 |          12067 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70f8142-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                               Enqueuing response to /10.54.145.32 | 10.54.101.56 |          12361 |                                   ReadStage-3

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70fcf60-cd9e-11e8-bed7-f13ab36bfa4c |                                                                                                 Sending REQUEST_RESPONSE message to /10.54.145.32 | 10.54.101.56 |          14161 | MessagingService-Outgoing-/10.54.145.32-Small

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff670-cd9e-11e8-8e99-15807bff4dfd |                                                                                              REQUEST_RESPONSE message received from /10.54.101.56 | 10.54.145.32 |             51 |       MessagingService-Incoming-/10.54.101.56

e70ac650-cd9e-11e8-8e99-15807bff4dfd | e70ff671-cd9e-11e8-8e99-15807bff4dfd |      




The information in this Internet Email is confidential and may be legally privileged. It is intended solely for the addressee. Access to this Email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed to our clients any opinions or advice contained in this Email are subject to the terms and conditions expressed in any applicable governing The Home Depot terms of business or client engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy and content of this attachment and for any damages or losses arising from any inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-mail message or its attachment.