OSDir

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: ConcurrentModificationException while accessing managed keyed state


Hi,

Sorry guys for the delay. I was trying to reproduce the complete error on my local machine but could not get it though. I will try again with actual traffic and let you the exception Stacktrace.

For now, I have the following details available to me.

Flink version: 1.4.1
State backend: Heap

I am not creating a custom thread to operate the state.

Code snippet : https://gist.github.com/garvitlnmiit/10db9d4b6eb41135332fba13d908e36c

I will reply back with the Stacktrace soon.

Thanks,



On Sat, Jun 2, 2018 at 6:49 PM, aitozi <gjying1314@xxxxxxxxx> wrote:
Hi Garvit Sharma,

Flink run with per parallel with a single thread. Can you show a little code
about how you use the keyed state in processFunction

Garvit Sharma wrote
> Hi,
>
> I have a use case where I am keeping the keyed state in ProcessFunction.
>
> Key: Integer personId;
>
> /**
>  * The data type stored in the state
>  */public class PersonDetails {
>     public long count;
>     public long lastModified;}
>
>
> I have encountered a lot of ConcurrentModificationException.
>
> I thought Flink processes all the operators on a keyed stream in a single
> thread. It seems like operators being accessed through multiple threads.
>
> If I get such exception then the data coming from Kafka would be consumed
> without making an update to the internal state. Making me lose the data.
>
> Please help me in handling the case according to my use case.
>
> Thanks,
>
> --
>
> Garvit Sharma
> github.com/garvitlnmiit/
>
> No Body is a Scholar by birth, its only hard work and strong determination
> that makes him master.





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/



--

Garvit Sharma
github.com/garvitlnmiit/

No Body is a Scholar by birth, its only hard work and strong determination that makes him master.