OSDir


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 'mvn verify' fails at flink-hadoop-fs


Generally, any test that uses HDFS will fail on Windows. We've disabled most of them, but some slip through from time to time.

Note that we do not provide any guarantees for all tests passing on Windows.

On 10.07.2018 16:28, NEKRASSOV, ALEXEI wrote:
I'm running 'mvn clean verify' on Windows with no Hadoop libraries installed, and the build fails (see below).
What's the solution? Is there a switch to skip Hadoop-related tests?
Or I need to install Hadoop libraries?

Thanks,
Alex


Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.726 sec <<< FAILURE! - in org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest
org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest  Time elapsed: 1.726 sec  <<< ERROR!
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
         at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
         at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:570)
         at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:996)
         at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:484)
         at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:293)
         at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:891)
         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:638)
         at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:503)
         at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:559)
         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:724)
         at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:708)
         at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1358)
         at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:996)
         at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:867)
         at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:702)
         at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:374)
         at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:355)
         at org.apache.flink.runtime.fs.hdfs.HdfsBehaviorTest.createHDFS(HdfsBehaviorTest.java:65)

Running org.apache.flink.runtime.fs.hdfs.HdfsKindTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.081 sec - in org.apache.flink.runtime.fs.hdfs.HdfsKindTest
Running org.apache.flink.runtime.fs.hdfs.LimitedConnectionsConfigurationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec - in org.apache.flink.runtime.fs.hdfs.LimitedConnectionsConfigurationTest

Results :

Tests in error:
   HdfsBehaviorTest.createHDFS:65 ▒ UnsatisfiedLink org.apache.hadoop.io.nativeio...

Tests run: 24, Failures: 0, Errors: 1, Skipped: 1

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] force-shading ...................................... SUCCESS [  2.335 s]
[INFO] flink .............................................. SUCCESS [ 29.794 s]
[INFO] flink-annotations .................................. SUCCESS [  2.198 s]
[INFO] flink-shaded-hadoop ................................ SUCCESS [  0.226 s]
[INFO] flink-shaded-hadoop2 ............................... SUCCESS [ 11.015 s]
[INFO] flink-shaded-hadoop2-uber .......................... SUCCESS [ 16.343 s]
[INFO] flink-shaded-yarn-tests ............................ SUCCESS [ 13.653 s]
[INFO] flink-shaded-curator ............................... SUCCESS [  1.386 s]
[INFO] flink-test-utils-parent ............................ SUCCESS [  0.191 s]
[INFO] flink-test-utils-junit ............................. SUCCESS [  3.318 s]
[INFO] flink-metrics ...................................... SUCCESS [  0.212 s]
[INFO] flink-metrics-core ................................. SUCCESS [  3.502 s]
[INFO] flink-core ......................................... SUCCESS [01:30 min]
[INFO] flink-java ......................................... SUCCESS [01:31 min]
[INFO] flink-queryable-state .............................. SUCCESS [  0.186 s]
[INFO] flink-queryable-state-client-java .................. SUCCESS [  4.099 s]
[INFO] flink-filesystems .................................. SUCCESS [  0.198 s]
[INFO] flink-hadoop-fs .................................... FAILURE [  8.672 s]