OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

VerifyError when running Python streaming job


Hi,

I'm running into errors when trying to run a Flink streaming program. 

Running the WordCount example from the docs fails with this error:

java.lang.VerifyError: (class: site$py, method: _Helper$26 signature: (Lorg/python/core/PyFrame;Lorg/python/core/ThreadState;)Lorg/python/core/PyObject;) Incompatible argument to function
	at java.lang.Class.getDeclaredConstructors0(Native Method)
	at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
	at java.lang.Class.getConstructor0(Class.java:3075)
	at java.lang.Class.getConstructor(Class.java:1825)
	at org.python.core.BytecodeLoader.makeCode(BytecodeLoader.java:70)
	at org.python.core.util.importer.getModuleCode(importer.java:217)
	at org.python.core.util.importer.importer_load_module(importer.java:95)
	at org.python.modules.zipimport.zipimporter.zipimporter_load_module(zipimporter.java:163)
	at org.python.modules.zipimport.zipimporter$zipimporter_load_module_exposer.__call__(Unknown Source)
	at org.python.core.PyBuiltinMethodNarrow.__call__(PyBuiltinMethodNarrow.java:46)
	at org.python.core.imp.loadFromLoader(imp.java:587)
	at org.python.core.imp.find_module(imp.java:537)
	at org.python.core.imp.import_next(imp.java:840)
	at org.python.core.imp.import_first(imp.java:861)
	at org.python.core.imp.load(imp.java:716)
	at org.python.core.Py.importSiteIfSelected(Py.java:1558)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:116)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:94)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:71)
	at org.apache.flink.streaming.python.util.InterpreterUtils.initPythonInterpreter(InterpreterUtils.java:122)
	at org.apache.flink.streaming.python.util.InterpreterUtils.deserializeFunction(InterpreterUtils.java:73)
	at org.apache.flink.streaming.python.api.functions.AbstractPythonUDF.open(AbstractPythonUDF.java:51)
	at org.apache.flink.streaming.python.api.functions.PythonFlatMapFunction.open(PythonFlatMapFunction.java:51)
	at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
	at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
	at org.apache.flink.streaming.api.operators.StreamFlatMap.open(StreamFlatMap.java:43)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
	at java.lang.Thread.run(Thread.java:748)

The only job I can get to run is one that reads from a KafkaConsumer010 and outputs the result without applying any operations (if there is a Filter, Map, etc. it will crash with the same error). However, removing all the operations from the WordCount example doesn't fix the issue there.

I'm running Flink 1.6-RC, I've also tried 1.6-SNAPSHOT which made no difference.

Any help would be much appreciated.

Thanks,

Joe Malt

Software Engineer Intern
Yelp