OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Akka Http used in custom RichSourceFunction


Hi,

Please take a look on https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/debugging_classloading.html . Especially check if you are using child-first class loading config. If it doesn’t help, probably you should shade your akka dependency. 

What is the full exception? Is it thrown when YOURS code tries to shutdown, or when FLINK’s code tries to shutdown?

Piotrek

On 24 May 2018, at 14:38, Niels van Kaam <niels@xxxxxxxxxxx> wrote:

Hi All,

I wrote a custom source function (RichSourceFunction) which connects to a web socket using the Akka Http Library. The job using this source runs fine on a local environment until upon shutdown I see the following error in the log: "Exception in thread "main" java.lang.NoSuchMethodError: akka.actor.ActorSystem.shutdown()V"

My impression is the issue is caused by a version conflict between flink's akka dependency and my own one (due to akka http). This seems to be related to this issue: https://issues.apache.org/jira/browse/FLINK-9240

Can I somehow avoid this conflict?
If not, does this mean I should avoid using Akka (or at least other versions than Flink's) within my sources/sinks?
Or can I safely catch and ignore the error? 

My dependencies are:
Flink: 1.4.2
akka-actor: 2.5.12
akka-stream: 2.5.12
akka-http: 10.1.1

Thank you for your help!

Cheers,
Niels