0 votes
1 view
in AWS by (19.2k points)

I want to read the file from the Hadoop File System.

In order to achieve the correct path of the file, I need hostname and port address of the hdfs.

so finally my path of the file will look something like

Path path = new Path("hdfs://")

Now I want to know to extract the HostName = "" & port: 9000?

Basically, I want to access the FileSystem on Amazon EMR but, when I use

 FileSystem fs = FileSystem.get(getConf());

I get

You possibly called FileSystem.get(conf) when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your path

So I decided to use URI. (I have to use URI) but I am not sure how to access the URI.

1 Answer

0 votes
by (44.5k points)

These are the two ways to solve your error:


String infile = "file.txt";

Path ofile = new Path(infile);

FileSystem fs = ofile.getFileSystem(getConf());


Configuration conf = getConf();

System.out.println("fs.default.name : - " + conf.get("fs.default.name"));

// It prints uri  as : hdfs:// or something

String uri = conf.get("fs.default.name");

FileSystem fs = FileSystem.get(uri,getConf());

Related questions

0 votes
0 answers
0 votes
1 answer
Welcome to Intellipaat Community. Get your technical queries answered by top developers !