国产探花免费观看_亚洲丰满少妇自慰呻吟_97日韩有码在线_资源在线日韩欧美_一区二区精品毛片,辰东完美世界有声小说,欢乐颂第一季,yy玄幻小说排行榜完本

首頁 > 編程 > Java > 正文

java連接hdfs ha和調(diào)用mapreduce jar示例

2019-11-26 15:34:42
字體:
供稿:網(wǎng)友

Java API 連接 HDFS HA

復(fù)制代碼 代碼如下:

public static void main(String[] args) {
  Configuration conf = new Configuration();
  conf.set("fs.defaultFS", "hdfs://hadoop2cluster");
  conf.set("dfs.nameservices", "hadoop2cluster");
  conf.set("dfs.ha.namenodes.hadoop2cluster", "nn1,nn2");
  conf.set("dfs.namenode.rpc-address.hadoop2cluster.nn1", "10.0.1.165:8020");
  conf.set("dfs.namenode.rpc-address.hadoop2cluster.nn2", "10.0.1.166:8020");
  conf.set("dfs.client.failover.proxy.provider.hadoop2cluster",        "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");
  FileSystem fs = null;
  try {
     fs = FileSystem.get(conf);
     FileStatus[] list = fs.listStatus(new Path("/"));
     for (FileStatus file : list) {
       System.out.println(file.getPath().getName());
      }
  } catch (IOException e) {
     e.printStackTrace();
  } finally{
      try {
        fs.close();
      } catch (IOException e) {
        e.printStackTrace();
      }
  }
}

Java API調(diào)用MapReduce程序

復(fù)制代碼 代碼如下:

String[] args = new String[24];
args[0] = “/usr/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar";
args[1] = "wordcount";
args[2] = "-D";
args[3] = "yarn.resourcemanager.address=10.0.1.165:8032";
args[4] = "-D";
args[5] = "yarn.resourcemanager.scheduler.address=10.0.1.165:8030";
args[6] = "-D";
args[7] = "fs.defaultFS=hdfs://hadoop2cluster/";
args[8] = "-D";
args[9] = "dfs.nameservices=hadoop2cluster";
args[10] = "-D";
args[11] = "dfs.ha.namenodes.hadoop2cluster=nn1,nn2";
args[12] = "-D";
args[13] = "dfs.namenode.rpc-address.hadoop2cluster.nn1=10.0.1.165:8020";
args[14] = "-D";
args[15] = "dfs.namenode.rpc-address.hadoop2cluster.nn2=10.0.1.166:8020";
args[16] = "-D";
args[17] = "dfs.client.failover.proxy.provider.hadoop2cluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider";
args[18] = "-D";
args[19] = "fs.hdfs.impl=org.apache.hadoop.hdfs.DistributedFileSystem";
args[20] = "-D";
args[21] = "mapreduce.framework.name=yarn";
args[22] = "/input";
args[23] = "/out01";
RunJar.main(args);

發(fā)表評(píng)論 共有條評(píng)論
用戶名: 密碼:
驗(yàn)證碼: 匿名發(fā)表
主站蜘蛛池模板: 新疆| 七台河市| 罗定市| 镇巴县| 宜城市| 凤翔县| 黔西县| 柳江县| 海丰县| 南投市| 苍梧县| 普格县| 内乡县| 张北县| 株洲市| 新丰县| 通河县| 垣曲县| 壶关县| 和田县| 汾阳市| 突泉县| 姜堰市| 平定县| 电白县| 揭西县| 五家渠市| 郎溪县| 邵阳县| 芜湖县| 衡阳市| 石门县| 抚顺市| 甘肃省| 蛟河市| 乳源| 珲春市| 沁源县| 肥城市| 桐城市| 沈阳市|