[jira] [Created] (FLINK-9442) Flink Scaling not working
Yow Siew Wai created FLINK-9442:
Summary: Flink Scaling not working
Issue Type: Bug
Reporter: Yow Siew Wai
We are in the middle of testing scaling ability of Flink. But we found that scaling not working, no matter increase more slot or increase number of Task Manager. We would expect a linear, if not close-to-linear scaling performance but the result even show degradation. Appreciated any comments.
-Just a simple pass through test,
- auto gen source 3mil records, each 1000k in size, parallelism=1
- source pass into next map operator, which just return the same record, and sent counter to statsD, parallelism is in cases = 2,4,6
- 3 TM, total 6 slots(2/TM) each JM/TM has 32 vCPU, 100GB memory
- 2 slots: 26 seconds, 3mil/26=115k TPS
- 4 slots: 23 seconds, 3mil/23=130k TPS
- 6 slots: 22 seconds, 3mil/22=136k TPS
As shown the scaling is almost nothing. Any clue? Thanks.
This message was sent by Atlassian JIRA