Hadoop 編程
MapReduce - 編程
處理
- select：直接分析輸入數據，取出需要的字段數據即可
- where: 也是對輸入數據處理的過程中進行處理，判斷是否需要該數據
- aggregation:min, max, sum
- group by: 通過Reducer實現
- sort
- join: map join, reduce join
Third-Party Libraries
export LIBJARS=$MYLIB/commons-lang-2.3.jar, hadoop jar prohadoop-0.0.1-SNAPSHOT.jar org.aspress.prohadoop.c3. WordCountUsingToolRunner -libjars $LIBJARS
hadoop jar prohadoop-0.0.1-SNAPSHOT-jar-with-dependencies.jar org.aspress.prohadoop.c3. WordCountUsingToolRunner The dependent libraries are now included inside the application JAR file
一般還是上面的好，指定依賴可以利用Public Cache，如果是包含依賴，則每次都需要拷貝