osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: ConcurrentModificationException while accessing managed keyed state


Hi ,Garvit Sharma

Just read your code snippet, I think it look ok. May need your reproduce
senior and exception stackstrace then.

Best wishes
Aitozi 


Garvit Sharma wrote
> Hi,
> 
> Sorry guys for the delay. I was trying to reproduce the complete error on
> my local machine but could not get it though. I will try again with actual
> traffic and let you the exception Stacktrace.
> 
> For now, I have the following details available to me.
> 
> Flink version: 1.4.1
> State backend: Heap
> 
> I am not creating a custom thread to operate the state.
> 
> Code snippet :
> https://gist.github.com/garvitlnmiit/10db9d4b6eb41135332fba13d908e36c
> 
> I will reply back with the Stacktrace soon.
> 
> Thanks,
> 
> 
> 
> On Sat, Jun 2, 2018 at 6:49 PM, aitozi <

> gjying1314@

> > wrote:
> 
>> Hi Garvit Sharma,
>>
>> Flink run with per parallel with a single thread. Can you show a little
>> code
>> about how you use the keyed state in processFunction
>>
>> Garvit Sharma wrote
>> > Hi,
>> >
>> > I have a use case where I am keeping the keyed state in
>> ProcessFunction.
>> >
>> > Key: Integer personId;
>> >
>> > /**
>> >  * The data type stored in the state
>> >  */public class PersonDetails {
>> >     public long count;
>> >     public long lastModified;}
>> >
>> >
>> > I have encountered a lot of ConcurrentModificationException.
>> >
>> > I thought Flink processes all the operators on a keyed stream in a
>> single
>> > thread. It seems like operators being accessed through multiple
>> threads.
>> >
>> > If I get such exception then the data coming from Kafka would be
>> consumed
>> > without making an update to the internal state. Making me lose the
>> data.
>> >
>> > Please help me in handling the case according to my use case.
>> >
>> > Thanks,
>> >
>> > --
>> >
>> > Garvit Sharma
>> > github.com/garvitlnmiit/
>> >
>> > No Body is a Scholar by birth, its only hard work and strong
>> determination
>> > that makes him master.
>>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-flink-user-mailing-list-archive.2336050.
>> n4.nabble.com/
>>
> 
> 
> 
> -- 
> 
> Garvit Sharma
> github.com/garvitlnmiit/
> 
> No Body is a Scholar by birth, its only hard work and strong determination
> that makes him master.





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/