Mac環境下Hadoop與Eclipse的集成

在這一節中我們來說一下應該怎樣把Hadoop集成到Eclipse中的,在官方中提供的有相關插件來集成eclipse的,但都是對Hadoop1.x的集成,下面我們進行自己適合版本的插件的生成。

如圖:

爲了能夠自定義插件,我們需要hadoop2.x的插件的源碼包,hadoop2x-eclipse-plugin-master.zip這個源碼包在網上不太容易找到,在這裏提供下載。

1、解壓下載後的源碼包

2、cd src/contrib/eclipse-plugin,進入到eclipse-plugin目錄

3、刪除掉這個目錄下的ivy和ivy.xml這兩個文件

4、修改build.xml文件,如下修改property的前5處配置

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
 
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at
 
       http://www.apache.org/licenses/LICENSE-2.0
 
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
 
<project default="jar" name="eclipse-plugin">
 
    <property name="jdk.home" value="/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home />
    <property name="hadoop.version" value="2.2.0" />
    <property name="hadoop.home" value="/usr/local/soft/hadoop-2.2.0" />
    <property name="eclipse.version" value="3.8" />
    <property name="eclipse.home" value="/Applications/adt-bundle/eclipse" />
    <property name="root" value="${basedir}" />
    <property file="${root}/build.properties" />
    <property name="name" value="${ant.project.name}" />
    <property name="src.dir" location="${root}/src" />
    <property name="build.contrib.dir" location="${root}/build/contrib" />
    <property name="build.dir" location="${build.contrib.dir}/${name}" />
    <property name="build.classes" location="${build.dir}/classes" />
    <property name="javac.deprecation" value="off" />
    <property name="javac.debug" value="on" />
    <property name="build.encoding" value="UTF-8" />
     
    <path id="eclipse-sdk-jars">
        <fileset dir="${eclipse.home}/plugins/">
            <include name="org.eclipse.ui*.jar" />
        <include name="org.eclipse.jdt*.jar" />
        <include name="org.eclipse.core*.jar" />
        <include name="org.eclipse.equinox*.jar" />
        <include name="org.eclipse.debug*.jar" />
        <include name="org.eclipse.osgi*.jar" />
        <include name="org.eclipse.swt*.jar" />
        <include name="org.eclipse.jface*.jar" />
        <include name="org.eclipse.team.cvs.ssh2*.jar" />
        <include name="com.jcraft.jsch*.jar" />
            </fileset>
    </path>
     
    <path id="project-jars">
        <fileset file="${build.dir}/lib/*.jar" />
    </path>
     
    <target name="init" unless="skip.contrib">
        <echo message="contrib: ${name}" />
        <mkdir dir="${build.dir}" />
        <mkdir dir="${build.classes}" />
        <mkdir dir="${build.dir}/lib" />
        <copy todir="${build.dir}/lib/" verbose="true">
            <fileset dir="${hadoop.home}/share/hadoop/mapreduce">
            <include name="hadoop*.jar" />
            <exclude name="*test*" />
            <exclude name="*example*" />
        </fileset>
        <fileset dir="${hadoop.home}/share/hadoop/common">
            <include name="hadoop*.jar" />
            <exclude name="*test*" />
            <exclude name="*example*" />
        </fileset>
        <fileset dir="${hadoop.home}/share/hadoop/hdfs">
            <include name="hadoop*.jar" />
            <exclude name="*test*" />
            <exclude name="*example*" />
        </fileset>
        <fileset dir="${hadoop.home}/share/hadoop/yarn">
            <include name="hadoop*.jar" />
            <exclude name="*test*" />
            <exclude name="*example*" />
        </fileset>
        <fileset dir="${hadoop.home}/share/hadoop/common/lib">
            <include name="protobuf-java-*.jar" />
            <include name="log4j-*.jar" />
            <include name="commons-cli-*.jar" />
            <include name="commons-collections-*.jar" />
            <include name="commons-configuration-*.jar" />
            <include name="commons-lang-*.jar" />
            <include name="jackson-core-asl-*.jar" />
            <include name="jackson-mapper-asl-*.jar" />
            <include name="slf4j-log4j12-*.jar" />
            <include name="slf4j-api-*.jar" />
            <include name="guava-*.jar" />
            <include name="hadoop-annotations-*.jar" />
            <include name="hadoop-auth-*.jar" />
            <include name="commons-cli-*.jar" />
            <include name="netty-*.jar" />
        </fileset>
        </copy>
    </target>
    <target name="compile" depends="init" unless="skip.contrib">
        <echo message="contrib: ${name}" />
        <javac fork="true" executable="${jdk.home}/bin/javac" encoding="${build.encoding}" srcdir="${src.dir}" includes="**/*.java" destdir="${build.classes}" debug="${javac.debug}" deprecation="${javac.deprecation}" includeantruntime="on">
            <classpath refid="eclipse-sdk-jars" />
            <classpath refid="project-jars" />
        </javac>
    </target>
        <target name="jar" depends="compile" unless="skip.contrib">
        <pathconvert property="mf.classpath" pathsep=",lib/">
        <path refid="project-jars" />
            <flattenmapper />
        </pathconvert>
        <jar jarfile="${build.dir}/hadoop-${hadoop.version}-eclipse-${eclipse.version}-plugin.jar" manifest="${root}/META-INF/MANIFEST.MF">
        <manifest>
            <attribute name="Bundle-ClassPath" value="classes/,lib/${mf.classpath}" />
        </manifest>
        <fileset dir="${build.dir}" includes="classes/ lib/" />
        <fileset dir="${root}" includes="resources/ plugin.xml" />
        </jar>
    </target>
    <target name="clean">
        <echo message="contrib: ${name}" />
        <delete dir="${build.dir}" />
    </target>
</project>

5、在這個build.xml文件所在的目錄中運行ant jar


6、把生成的插件命名爲hadoop-eclipse-plugin-2.2.0.jar,並複製到eclipse安裝目錄的plugins中,並重新打開eclipse

7、配置eclipse中的hadoop


發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章