I agree with Vino, that you should check if the watermark is
progressing for all subtasks, if you are using event time
semantics. If this is not the problem it would help if you could
share the code of your job. By the way have you tried reproducing
the problem with collection source?
On 10/10/18 08:44, 潘 功森 wrote:
Cause default state
size in one hour is too small，and the max
window size is 24 hours, so I used 500M.
MemoryStateBackend stateBackend = new MemoryStateBackend(MAX_STATE_SIZE);//500M
And I found
Irrespective of the configured maximal state
size, the state cannot be larger than the akka frame size.
So I add a config in flink-comf.yaml:
What else do I have to pay attention to?
I saw the exception image you provided. Based on the
exception message, it seems you used the default max state
You can specify the max state size to override the
default value. Try :
MemoryStateBackend stateBackend = new
Please note that you need to reserve enough memory for
Please have a
look about my last mail.
When the cached
window data is too large, how?
Did you mean "computer momery" referring to
The Flink window mechanism is internally based
on State, and this is done for fault tolerance.
If you introduce external storage, it will
break its design and bring other problems.
"ram to cache the distinct data
about sliding window" means I used computer
momery not the third part db to cache the
data need used in window.
“the data need used in window”
：such as the sliding
window is 1 hour, and I need to count the
distinct users, I need to cache the user id
about one hour.
Cause there’re no related
vino yang <yanghua1127@xxxxxxxxx>
发送时间: Wednesday, October 10, 2018
主题: Re: No data issued by flink window
after a few hours
Can you explain what "ram to cache
the distinct data about sliding window"
The information you provide is too
small and will not help others to help
you analyze the problem and provide
In addition, regarding the usage of
Flink related issues, please only send
mail to the user mailing list.
The dev mailing list is mainly used
to discuss development related issues.
I used flink window, and when the job
begins, we could get the results of
windiow.But there’re no results issued after
a few hours.
I found the job is still running and no
errors, and the data not used window all can
By the way, I used Flink 1.3.2 and ram
to cache the distinct data about sliding