OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: compaction_throughput: Difference between 0 (unthrottled) and large value


Thomas,

This post from Ryan Svihla has a few notes in it that may or may not be useful to you:

>If you read the original throttling Jira you can see that there is a hurry up and wait component to unthrottled compaction (CASSANDRA-2156- Compaction Throttling). Ultimately you will saturate your IO in bursts, backing up other processes and making different bottlenecks spike up a long the way, potentially causing something OTHER than compaction to get so far behind that the server becomes unresponsive (such as GC).

via https://medium.com/@foundev/how-i-tune-cassandra-compaction-7c16fb0b1d99

On Mon, Jun 11, 2018 at 12:05 AM, Steinmaurer, Thomas <thomas.steinmaurer@xxxxxxxxxxxxx> wrote:

Sorry, should have first looked at the source code. In case of 0, it is set to Double.MAX_VALUE.

 

Thomas

 

From: Steinmaurer, Thomas [mailto:thomas.steinmaurer@dynatrace.com]
Sent: Montag, 11. Juni 2018 08:53
To: user@xxxxxxxxxxxxxxxxxxxx
Subject: compaction_throughput: Difference between 0 (unthrottled) and large value

 

Hello,

 

on a 3 node loadtest cluster with very capable machines (32 physical cores, 512G RAM, 20T storage (26 disk RAID)), I’m trying to max out compaction, thus currently testing with:

 

concurrent_compactors: 16

compaction_throughput_mb_per_sec: 0

 

With our simulated incoming load + compaction etc., the Linux volume shows ~ 20 Mbyte/s Read IO + 50 Mbyte/s Write IO in AVG, constantly.

 

 

Setting throughput to 0 should mean unthrottled, right? Is this really unthrottled from a throughput perspective and then is basically limited by disk capabilities only? Or should it be better set to a very high value instead of 0. Is there any semantical difference here?

 

 

Thanks,

Thomas

 

The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313

The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313



--
Joshua Galbraith | Senior Software Engineer | New Relic
C: 907-209-1208 | jgalbraith@xxxxxxxxxxxx