osdir.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Read all json files from a hdfs partition folder


Hi,

I wanted to read all json files from hdfs with partition folder.

public static void main(String[] args) {


StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();


//path

//hdfs://localhost:8020/data/ingestion/ingestion.raw.product/2018/12/05/23


DataStream<String> df = env.readTextFile("hdfs://localhost:8020/data/ingestion/ingestion.raw.product");

try {

df.print();

env.execute("dfg");

} catch (Exception e) {

e.printStackTrace();

}


}