按照Hadoop官網start Hadoop cluster ，參考鏈接http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html
執行bin/hadoop命令後的顯示：
然後執行官網的操作驗證：
執行如下命令後的回顯：
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0.jar grep input output 'dfs[a-z.]+'
bin etc include input lib libexec LICENSE.txt NOTICE.txt README.txt sbin share
[root@localhost hadoop-2.6.5]# bin/hadoop jarshare/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar grep input output'dfs[a-z]+'
18/03/09 10:16:11 INFOConfiguration.deprecation: session.id is deprecated. Instead, usedfs.metrics.session-id
18/03/09 10:16:11 INFO jvm.JvmMetrics:Initializing JVM Metrics with processName=JobTracker, sessionId=
18/03/09 10:16:13 WARN mapreduce.JobResourceUploader:No job jar file set. User classes maynot be found. See Job or Job#setJar(String).
18/03/09 10:16:13 INFOinput.FileInputFormat: Total input paths to process : 8
18/03/09 10:16:13 INFOmapreduce.JobSubmitter: number of splits:8
18/03/09 10:16:14 INFOmapreduce.JobSubmitter: Submitting tokens for job: job_local1688762170_0001
18/03/09 10:16:15 INFO mapreduce.Job: Theurl to track the job: http://localhost:8080/
18/03/09 10:16:15 INFO mapreduce.Job:Running job: job_local1688762170_0001
18/03/09 10:16:15 INFOmapred.LocalJobRunner: OutputCommitter set in config null
18/03/09 10:16:15 INFOmapred.LocalJobRunner: OutputCommitter isorg.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
18/03/09 10:16:15 INFOmapred.LocalJobRunner: Waiting for map tasks
18/03/09 10:16:15 INFOmapred.LocalJobRunner: Starting task: attempt_local1688762170_0001_m_000000_0
18/03/09 10:16:15 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:15 INFO mapred.MapTask:Processing split: file:/home/hewy/hadoop/hadoop-2.6.5/input/hadoop-policy.xml:0+9683
18/03/09 10:16:16 INFO mapreduce.Job: Jobjob_local1688762170_0001 running in uber mode : false
18/03/09 10:16:16 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:16 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:16 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:16 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:16 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:16 INFO mapreduce.Job: map 0% reduce 0%
18/03/09 10:16:16 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:16 INFOmapred.LocalJobRunner:
18/03/09 10:16:16 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:16 INFO mapred.MapTask:Spilling map output
18/03/09 10:16:16 INFO mapred.MapTask:bufstart = 0; bufend = 17; bufvoid = 104857600
18/03/09 10:16:16 INFO mapred.MapTask:kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
18/03/09 10:16:16 INFO mapred.MapTask:Finished spill 0
18/03/09 10:16:16 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000000_0 is done. And is in the process ofcommitting
18/03/09 10:16:16 INFOmapred.LocalJobRunner: map
18/03/09 10:16:16 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000000_0' done.
18/03/09 10:16:16 INFOmapred.LocalJobRunner: Finishing task: attempt_local1688762170_0001_m_000000_0
18/03/09 10:16:16 INFOmapred.LocalJobRunner: Starting task: attempt_local1688762170_0001_m_000001_0
18/03/09 10:16:16 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:16 INFO mapred.MapTask:Processing split: file:/home/hewy/hadoop/hadoop-2.6.5/input/kms-site.xml:0+5511
18/03/09 10:16:17 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:17 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:17 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:17 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:17 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:17 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:17 INFOmapred.LocalJobRunner:
18/03/09 10:16:17 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:17 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000001_0 is done. And is in the process ofcommitting
18/03/09 10:16:17 INFO mapred.LocalJobRunner:map
18/03/09 10:16:17 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000001_0' done.
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Finishing task: attempt_local1688762170_0001_m_000001_0
18/03/09 10:16:17 INFO mapred.LocalJobRunner:Starting task: attempt_local1688762170_0001_m_000002_0
18/03/09 10:16:17 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:17 INFO mapred.MapTask:Processing split: file:/home/hewy/hadoop/hadoop-2.6.5/input/capacity-scheduler.xml:0+4436
18/03/09 10:16:17 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:17 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:17 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:17 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:17 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:17 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:17 INFO mapred.LocalJobRunner:
18/03/09 10:16:17 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:17 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000002_0 is done. And is in the process ofcommitting
18/03/09 10:16:17 INFOmapred.LocalJobRunner: map
18/03/09 10:16:17 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000002_0' done.
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Finishing task: attempt_local1688762170_0001_m_000002_0
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Starting task: attempt_local1688762170_0001_m_000003_0
18/03/09 10:16:17 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:17 INFO mapred.MapTask:Processing split: file:/home/hewy/hadoop/hadoop-2.6.5/input/kms-acls.xml:0+3523
18/03/09 10:16:17 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:17 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:17 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:17 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:17 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:17 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:17 INFOmapred.LocalJobRunner:
18/03/09 10:16:17 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:17 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000003_0 is done. And is in the process ofcommitting
18/03/09 10:16:17 INFO mapred.LocalJobRunner:map
18/03/09 10:16:17 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000003_0' done.
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Finishing task: attempt_local1688762170_0001_m_000003_0
18/03/09 10:16:17 INFO mapred.LocalJobRunner:Starting task: attempt_local1688762170_0001_m_000004_0
18/03/09 10:16:17 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:17 INFO mapred.MapTask:Processing split: file:/home/hewy/hadoop/hadoop-2.6.5/input/hdfs-site.xml:0+775
18/03/09 10:16:17 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:17 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:17 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:17 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:17 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:17 INFO mapreduce.Job: map 100% reduce 0%
18/03/09 10:16:17 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:17 INFOmapred.LocalJobRunner:
18/03/09 10:16:17 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:17 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000004_0 is done. And is in the process ofcommitting
18/03/09 10:16:17 INFOmapred.LocalJobRunner: map
18/03/09 10:16:17 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000004_0' done.
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Finishing task: attempt_local1688762170_0001_m_000004_0
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Starting task: attempt_local1688762170_0001_m_000005_0
18/03/09 10:16:17 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:17 INFO mapred.MapTask:Processing split: file:/home/hewy/hadoop/hadoop-2.6.5/input/core-site.xml:0+774
18/03/09 10:16:17 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:17 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:17 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:17 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:17 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:17 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:17 INFOmapred.LocalJobRunner:
18/03/09 10:16:17 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:17 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000005_0 is done. And is in the process ofcommitting
18/03/09 10:16:17 INFOmapred.LocalJobRunner: map
18/03/09 10:16:17 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000005_0' done.
18/03/09 10:16:17 INFO mapred.LocalJobRunner:Finishing task: attempt_local1688762170_0001_m_000005_0
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Starting task: attempt_local1688762170_0001_m_000006_0
18/03/09 10:16:17 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:17 INFO mapred.MapTask:Processing split: file:/home/hewy/hadoop/hadoop-2.6.5/input/yarn-site.xml:0+690
18/03/09 10:16:17 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:17 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:17 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:17 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:17 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:17 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:17 INFOmapred.LocalJobRunner:
18/03/09 10:16:17 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:17 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000006_0 is done. And is in the process ofcommitting
18/03/09 10:16:17 INFOmapred.LocalJobRunner: map
18/03/09 10:16:17 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000006_0' done.
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Finishing task: attempt_local1688762170_0001_m_000006_0
18/03/09 10:16:17 INFOmapred.LocalJobRunner: Starting task: attempt_local1688762170_0001_m_000007_0
18/03/09 10:16:17 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:17 INFO mapred.MapTask:Processing split:file:/home/hewy/hadoop/hadoop-2.6.5/input/httpfs-site.xml:0+620
18/03/09 10:16:18 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:18 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:18 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:18 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:18 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:18 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:18 INFOmapred.LocalJobRunner:
18/03/09 10:16:18 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:18 INFO mapred.Task:Task:attempt_local1688762170_0001_m_000007_0 is done. And is in the process ofcommitting
18/03/09 10:16:18 INFOmapred.LocalJobRunner: map
18/03/09 10:16:18 INFO mapred.Task: Task'attempt_local1688762170_0001_m_000007_0' done.
18/03/09 10:16:18 INFO mapred.LocalJobRunner:Finishing task: attempt_local1688762170_0001_m_000007_0
18/03/09 10:16:18 INFOmapred.LocalJobRunner: map task executor complete.
18/03/09 10:16:18 INFOmapred.LocalJobRunner: Waiting for reduce tasks
18/03/09 10:16:18 INFOmapred.LocalJobRunner: Starting task: attempt_local1688762170_0001_r_000000_0
18/03/09 10:16:18 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:18 INFO mapred.ReduceTask:Using ShuffleConsumerPlugin:org.apache.hadoop.mapreduce.task.reduce.Shuffle@573c189a
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: MergerManager: memoryLimit=363285696,maxSingleShuffleLimit=90821424, mergeThreshold=239768576, ioSortFactor=10,memToMemMergeOutputsThreshold=10
18/03/09 10:16:18 INFO reduce.EventFetcher:attempt_local1688762170_0001_r_000000_0 Thread started: EventFetcher forfetching Map Completion Events
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of map attempt_local1688762170_0001_m_000003_0decomp: 2 len: 6 to MEMORY
18/03/09 10:16:18 INFOreduce.InMemoryMapOutput: Read 2 bytes from map-output forattempt_local1688762170_0001_m_000003_0
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2,inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->2
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of mapattempt_local1688762170_0001_m_000000_0 decomp: 21 len: 25 to MEMORY
18/03/09 10:16:18 INFO reduce.InMemoryMapOutput:Read 21 bytes from map-output for attempt_local1688762170_0001_m_000000_0
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(NativeMethod)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 21,inMemoryMapOutputs.size() -> 2, commitMemory -> 2, usedMemory ->23
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of mapattempt_local1688762170_0001_m_000006_0 decomp: 2 len: 6 to MEMORY
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(NativeMethod)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
atorg.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 INFOreduce.InMemoryMapOutput: Read 2 bytes from map-output forattempt_local1688762170_0001_m_000006_0
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2,inMemoryMapOutputs.size() -> 3, commitMemory -> 23, usedMemory ->25
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of mapattempt_local1688762170_0001_m_000005_0 decomp: 2 len: 6 to MEMORY
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(NativeMethod)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
atorg.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 INFOreduce.InMemoryMapOutput: Read 2 bytes from map-output forattempt_local1688762170_0001_m_000005_0
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2,inMemoryMapOutputs.size() -> 4, commitMemory -> 25, usedMemory ->27
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of mapattempt_local1688762170_0001_m_000002_0 decomp: 2 len: 6 to MEMORY
18/03/09 10:16:18 INFOreduce.InMemoryMapOutput: Read 2 bytes from map-output forattempt_local1688762170_0001_m_000002_0
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2,inMemoryMapOutputs.size() -> 5, commitMemory -> 27, usedMemory ->29
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of map attempt_local1688762170_0001_m_000001_0decomp: 2 len: 6 to MEMORY
18/03/09 10:16:18 INFOreduce.InMemoryMapOutput: Read 2 bytes from map-output forattempt_local1688762170_0001_m_000001_0
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2,inMemoryMapOutputs.size() -> 6, commitMemory -> 29, usedMemory ->31
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(NativeMethod)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(NativeMethod)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of mapattempt_local1688762170_0001_m_000007_0 decomp: 2 len: 6 to MEMORY
18/03/09 10:16:18 INFOreduce.InMemoryMapOutput: Read 2 bytes from map-output forattempt_local1688762170_0001_m_000007_0
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2,inMemoryMapOutputs.size() -> 7, commitMemory -> 31, usedMemory ->33
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(NativeMethod)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
atorg.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
atorg.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 INFO reduce.LocalFetcher:localfetcher#1 about to shuffle output of mapattempt_local1688762170_0001_m_000004_0 decomp: 2 len: 6 to MEMORY
18/03/09 10:16:18 INFOreduce.InMemoryMapOutput: Read 2 bytes from map-output forattempt_local1688762170_0001_m_000004_0
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2,inMemoryMapOutputs.size() -> 8, commitMemory -> 33, usedMemory ->35
18/03/09 10:16:18 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
atorg.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:18 INFO reduce.EventFetcher:EventFetcher is interrupted.. Returning
18/03/09 10:16:18 INFOmapred.LocalJobRunner: 8 / 8 copied.
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: finalMerge called with 8 in-memory map-outputs and 0on-disk map-outputs
18/03/09 10:16:18 INFO mapred.Merger:Merging 8 sorted segments
18/03/09 10:16:18 INFO mapred.Merger: Downto the last merge-pass, with 1 segments left of total size: 10 bytes
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: Merged 8 segments, 35 bytes to disk to satisfy reducememory limit
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: Merging 1 files, 25 bytes from disk
18/03/09 10:16:18 INFOreduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
18/03/09 10:16:18 INFO mapred.Merger:Merging 1 sorted segments
18/03/09 10:16:18 INFO mapred.Merger: Downto the last merge-pass, with 1 segments left of total size: 10 bytes
18/03/09 10:16:18 INFOmapred.LocalJobRunner: 8 / 8 copied.
18/03/09 10:16:18 INFOConfiguration.deprecation: mapred.skip.on is deprecated. Instead, usemapreduce.job.skiprecords
18/03/09 10:16:18 INFO mapred.Task:Task:attempt_local1688762170_0001_r_000000_0 is done. And is in the process ofcommitting
18/03/09 10:16:18 INFOmapred.LocalJobRunner: 8 / 8 copied.
18/03/09 10:16:18 INFO mapred.Task: Taskattempt_local1688762170_0001_r_000000_0 is allowed to commit now
18/03/09 10:16:18 INFO output.FileOutputCommitter:Saved output of task 'attempt_local1688762170_0001_r_000000_0' tofile:/home/hewy/hadoop/hadoop-2.6.5/grep-temp-744353686/_temporary/0/task_local1688762170_0001_r_000000
18/03/09 10:16:18 INFOmapred.LocalJobRunner: reduce > reduce
18/03/09 10:16:18 INFO mapred.Task: Task'attempt_local1688762170_0001_r_000000_0' done.
18/03/09 10:16:18 INFOmapred.LocalJobRunner: Finishing task: attempt_local1688762170_0001_r_000000_0
18/03/09 10:16:18 INFOmapred.LocalJobRunner: reduce task executor complete.
18/03/09 10:16:19 INFO mapreduce.Job: map 100% reduce 100%
18/03/09 10:16:19 INFO mapreduce.Job: Jobjob_local1688762170_0001 completed successfully
18/03/09 10:16:19 INFO mapreduce.Job:Counters: 33
File System Counters
FILE: Number of bytesread=234656
FILE: Number of byteswritten=2301095
FILE: Number of readoperations=0
FILE: Number of large readoperations=0
FILE: Number of writeoperations=0
Map-Reduce Framework
Map input records=745
Map output records=1
Map output bytes=17
Map output materializedbytes=67
Input split bytes=973
Combine input records=1
Combine output records=1
Reduce input groups=1
Reduce shuffle bytes=67
Reduce input records=1
Reduce output records=1
Spilled Records=2
Shuffled Maps =8
Failed Shuffles=0
Merged Map outputs=8
GC time elapsed (ms)=346
CPU time spent (ms)=0
Physical memory (bytes)snapshot=0
Virtual memory (bytes)snapshot=0
Total committed heap usage(bytes)=1417719808
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=26012
File Output Format Counters
Bytes Written=123
18/03/09 10:16:19 INFO jvm.JvmMetrics:Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - alreadyinitialized
18/03/09 10:16:19 WARNmapreduce.JobResourceUploader: No job jar file set. User classes may not be found. See Job orJob#setJar(String).
18/03/09 10:16:20 INFOinput.FileInputFormat: Total input paths to process : 1
18/03/09 10:16:20 INFO mapreduce.JobSubmitter:number of splits:1
18/03/09 10:16:20 INFOmapreduce.JobSubmitter: Submitting tokens for job: job_local65761480_0002
18/03/09 10:16:20 INFO mapreduce.Job: Theurl to track the job: http://localhost:8080/
18/03/09 10:16:20 INFO mapreduce.Job:Running job: job_local65761480_0002
18/03/09 10:16:20 INFOmapred.LocalJobRunner: OutputCommitter set in config null
18/03/09 10:16:20 INFOmapred.LocalJobRunner: OutputCommitter isorg.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
18/03/09 10:16:20 INFOmapred.LocalJobRunner: Waiting for map tasks
18/03/09 10:16:20 INFOmapred.LocalJobRunner: Starting task: attempt_local65761480_0002_m_000000_0
18/03/09 10:16:20 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:20 INFO mapred.MapTask:Processing split:file:/home/hewy/hadoop/hadoop-2.6.5/grep-temp-744353686/part-r-00000:0+111
18/03/09 10:16:20 INFO mapred.MapTask:(EQUATOR) 0 kvi 26214396(104857584)
18/03/09 10:16:20 INFO mapred.MapTask:mapreduce.task.io.sort.mb: 100
18/03/09 10:16:20 INFO mapred.MapTask: softlimit at 83886080
18/03/09 10:16:20 INFO mapred.MapTask:bufstart = 0; bufvoid = 104857600
18/03/09 10:16:20 INFO mapred.MapTask:kvstart = 26214396; length = 6553600
18/03/09 10:16:20 INFO mapred.MapTask: Mapoutput collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
18/03/09 10:16:20 INFOmapred.LocalJobRunner:
18/03/09 10:16:20 INFO mapred.MapTask:Starting flush of map output
18/03/09 10:16:20 INFO mapred.MapTask:Spilling map output
18/03/09 10:16:20 INFO mapred.MapTask:bufstart = 0; bufend = 17; bufvoid = 104857600
18/03/09 10:16:20 INFO mapred.MapTask:kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
18/03/09 10:16:20 INFO mapred.MapTask:Finished spill 0
18/03/09 10:16:20 INFO mapred.Task:Task:attempt_local65761480_0002_m_000000_0 is done. And is in the process ofcommitting
18/03/09 10:16:20 INFOmapred.LocalJobRunner: map
18/03/09 10:16:20 INFO mapred.Task: Task 'attempt_local65761480_0002_m_000000_0'done.
18/03/09 10:16:20 INFOmapred.LocalJobRunner: Finishing task: attempt_local65761480_0002_m_000000_0
18/03/09 10:16:20 INFOmapred.LocalJobRunner: map task executor complete.
18/03/09 10:16:20 INFO mapred.LocalJobRunner:Waiting for reduce tasks
18/03/09 10:16:20 INFOmapred.LocalJobRunner: Starting task: attempt_local65761480_0002_r_000000_0
18/03/09 10:16:20 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
18/03/09 10:16:20 INFO mapred.ReduceTask:Using ShuffleConsumerPlugin:org.apache.hadoop.mapreduce.task.reduce.Shuffle@6750b41c
18/03/09 10:16:20 INFOreduce.MergeManagerImpl: MergerManager: memoryLimit=363285696,maxSingleShuffleLimit=90821424, mergeThreshold=239768576, ioSortFactor=10,memToMemMergeOutputsThreshold=10
18/03/09 10:16:20 INFO reduce.EventFetcher:attempt_local65761480_0002_r_000000_0 Thread started: EventFetcher for fetchingMap Completion Events
18/03/09 10:16:20 INFO reduce.LocalFetcher:localfetcher#2 about to shuffle output of mapattempt_local65761480_0002_m_000000_0 decomp: 21 len: 25 to MEMORY
18/03/09 10:16:20 INFOreduce.InMemoryMapOutput: Read 21 bytes from map-output forattempt_local65761480_0002_m_000000_0
18/03/09 10:16:20 INFOreduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 21,inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->21
18/03/09 10:16:20 WARN io.ReadaheadPool:Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(NativeMethod)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)
atorg.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18/03/09 10:16:20 INFO reduce.EventFetcher:EventFetcher is interrupted.. Returning
18/03/09 10:16:20 INFOmapred.LocalJobRunner: 1 / 1 copied.
18/03/09 10:16:20 INFOreduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0on-disk map-outputs
18/03/09 10:16:20 INFO mapred.Merger:Merging 1 sorted segments
18/03/09 10:16:20 INFO mapred.Merger: Downto the last merge-pass, with 1 segments left of total size: 11 bytes
18/03/09 10:16:20 INFOreduce.MergeManagerImpl: Merged 1 segments, 21 bytes to disk to satisfy reducememory limit
18/03/09 10:16:20 INFOreduce.MergeManagerImpl: Merging 1 files, 25 bytes from disk
18/03/09 10:16:20 INFOreduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce
18/03/09 10:16:20 INFO mapred.Merger:Merging 1 sorted segments
18/03/09 10:16:20 INFO mapred.Merger: Downto the last merge-pass, with 1 segments left of total size: 11 bytes
18/03/09 10:16:20 INFOmapred.LocalJobRunner: 1 / 1 copied.
18/03/09 10:16:20 INFO mapred.Task:Task:attempt_local65761480_0002_r_000000_0 is done. And is in the process ofcommitting
18/03/09 10:16:20 INFOmapred.LocalJobRunner: 1 / 1 copied.
18/03/09 10:16:20 INFO mapred.Task: Taskattempt_local65761480_0002_r_000000_0 is allowed to commit now
18/03/09 10:16:20 INFOoutput.FileOutputCommitter: Saved output of task'attempt_local65761480_0002_r_000000_0' tofile:/home/hewy/hadoop/hadoop-2.6.5/output/_temporary/0/task_local65761480_0002_r_000000
18/03/09 10:16:20 INFOmapred.LocalJobRunner: reduce > reduce
18/03/09 10:16:20 INFO mapred.Task: Task'attempt_local65761480_0002_r_000000_0' done.
18/03/09 10:16:20 INFOmapred.LocalJobRunner: Finishing task: attempt_local65761480_0002_r_000000_0
18/03/09 10:16:20 INFO mapred.LocalJobRunner:reduce task executor complete.
18/03/09 10:16:21 INFO mapreduce.Job: Jobjob_local65761480_0002 running in uber mode : false
18/03/09 10:16:21 INFO mapreduce.Job: map 100% reduce 100%
18/03/09 10:16:21 INFO mapreduce.Job: Jobjob_local65761480_0002 completed successfully
18/03/09 10:16:21 INFO mapreduce.Job:Counters: 33
File System Counters
FILE: Number of bytesread=66594
FILE: Number of byteswritten=1013122
FILE: Number of readoperations=0
FILE: Number of large readoperations=0
FILE: Number of writeoperations=0
Map-Reduce Framework
Map input records=1
Map output records=1
Map output bytes=17
Map output materializedbytes=25
Input split bytes=133
Combine input records=0
Combine output records=0
Reduce input groups=1
Reduce shuffle bytes=25
Reduce input records=1
Reduce output records=1
Spilled Records=2
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=33
CPU time spent (ms)=0
Physical memory (bytes)snapshot=0
Virtual memory (bytes)snapshot=0
Total committed heap usage(bytes)=270032896
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=123
File Output Format Counters
Bytes Written=23
[root@localhost hadoop-2.6.5]#
然後執行cat output/*顯示：
只顯示上圖中圈住的第二行 1 dfsadmin,因爲
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0.jar grep input output 'dfs[a-z.]+'的意思是：把input中的所有文件作爲輸入，在這些文件中查找匹配正則‘dfs[a-z.]*’的詞並統計個數。正則‘dfs[a-z.]*’的意思匹配是一dfs開頭，後面有n個任意字母的詞。而我第一遍測試的時候顯示只有一個匹配的詞dfsadmin，刪除output目錄（最好刪除了第一遍測試生成的output文件夾，因爲我看其他大神的博客他們第一次測試失敗的時候生成了這個文件夾，再次測試又失敗，提示這個文件夾已經存在。原文鏈接http://www.xuebuyuan.com/2563083.html），講input中任意文件中添加一個dfshewy(符合正則條件的詞)，再次測試就顯示上圖中的內容：有2個詞匹配正則條件。