雲帆大數據學院_hdfs和YARN的啓動方式

YARN的Shell操作與管理

7.1啓動YARN

YARN有2個守護線程:ResourceManager、NodeManager。

[hadoop@localhost hadoop-2.2.0]$sbin/yarn-daemon.sh start resourcemanager

[hadoop@localhost hadoop-2.2.0]$sbin/yarn-daemon.sh start nodemanager

7.2YARN Web管理界面

YARN管理地址:

ResourceManager: 主機名:8088 。本環境中爲:http://hadoop-yarn.dragon.org:8088

NameNodeManager:主機名:8042 。本環境中爲:http://hadoop-yarn.dragon.org:8042



7.3運行MapRecuce程序


(1)  $HADOOP_HOME/share/hadoop/mapreduce目錄下有很多例子程序:

Anexample program must be given as the first argument.

Validprogram names are:

  aggregatewordcount: An Aggregate basedmap/reduce program that counts the words in the input files.

  aggregatewordhist: An Aggregate basedmap/reduce program that computes the histogram of the words in the input files.

  bbp: A map/reduce program that usesBailey-Borwein-Plouffe to compute exact digits of Pi.

  dbcount: An example job that count thepageview counts from a database.

  distbbp: A map/reduce program that uses aBBP-type formula to compute exact bits of Pi.

  grep: A map/reduce program that counts thematches of a regex in the input.

  join: A job that effects a join over sorted,equally partitioned datasets

  multifilewc: A job that counts words fromseveral files.

  pentomino: A map/reduce tile laying programto find solutions to pentomino problems.

  pi: A map/reduce program that estimates Piusing a quasi-Monte Carlo method.

  randomtextwriter: A map/reduce program thatwrites 10GB of random textual data per node.

  randomwriter: A map/reduce program thatwrites 10GB of random data per node.

  secondarysort: An example defining asecondary sort to the reduce.

  sort: A map/reduce program that sorts thedata written by the random writer.

  sudoku: A sudoku solver.

  teragen: Generate data for the terasort

  terasort: Run the terasort

  teravalidate: Checking results of terasort

  wordcount: A map/reduce program that countsthe words in the input files.

  wordmean: A map/reduce program that countsthe average length of the words in the input files.

  wordmedian: A map/reduce program that countsthe median length of the words in the input files.

  wordstandarddeviation: A map/reduce programthat counts the standard deviation of the length of the words in the inputfiles.

(2) 如何運行這些程序

運行這些例子通過$HADOOP_HOME/bin/yarn jar 命令執行,如:下面的例子是執行pi函數:

[root@hadoop-yarn hadoop-2.2.0]# bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar

說明:紅色部分:命令    藍色部分:jar文件所在的路徑


發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章