当前位置: 首页 > hadoop, 分布式系统 > 正文

windows7 hadoop 1.2.1 eclipse pulgin 生成

关键字:
1 星2 星3 星4 星5 星 (2 次投票, 评分: 5.00, 总分: 5)
Loading ... Loading ...
baidu_share
文章目录

一.编译eclipse-plugin

1.获取hadoop 1.2.1源码:http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz,解压缩至目录。

2.在 Eclipse 中选择 File—>Import—>Existing Projects into Workspace 导入已有项目,路径为: D:\hq\sdk\hadoop-1.2.1\src\contrib\eclipse-plugin,其默认的项目名称是 MapReduceTools。

3.导入项目后,发现项目出现错误。提示hadoop-core.jar没有,在项目中新建lib目录,把D:\hq\sdk\hadoop-1.2.1\hadoop-core-1.2.1.jar拷贝到lib目录下。并把hadoop-core-1.2.1.jar更名为hadoop-core.jar。右键点击”hadoop-core-1.2.1.jar”项目,选择
Properties—>Java Build Path—>Libraries,把hadoop-core.jar添加上。

4.把D:\hq\sdk\hadoop-1.2.1\src\contrib\build-contrib.xml拷贝到MapReduceTools项目下,和build.xml同级目录。

5.修改build.xml

5.1. 修改build-contrib.xml位置。以前的位置为:

<import file="../build-contrib.xml"/>

修改为:

<import file="build-contrib.xml"/>

5.2.增加hadoop 存放目录,eclipse存放目录,hadoop版本号。

<property name="hadoop.root" location="d:/hq/sdk/hadoop-1.2.1"/>
<property name="eclipse.home" location="d:/hq/sdk/eclipse" />
<property name="version" value="1.2.1"/>

5.3.定义hadoop jar的位置

	<path id="hadoop-jars">
	  <fileset dir="${hadoop.root}/">
	    <include name="hadoop-*.jar"/>
	  </fileset>
	</path>

5.4.修改path id=”classpath”,增加path id=”hadoop-jars”

	<path id="classpath">
	  <pathelement location="${build.classes}"/>
	  <pathelement location="${hadoop.root}/build/classes"/>
	  <path refid="eclipse-sdk-jars"/>
	  <path refid="hadoop-jars"/>
	</path>

5.5.修改target name=”jar”

	<target name="jar" depends="compile" unless="skip.contrib">
	  <mkdir dir="${build.dir}/lib"/>
	  <copy file="${hadoop.root}/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-cli-1.2.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-lang-2.4.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-configuration-1.6.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar" todir="${build.dir}/lib" verbose="true"/>
	  <jar jarfile="${build.dir}/hadoop-${name}-${version}.jar" manifest="${root}/META-INF/MANIFEST.MF">
	    <fileset dir="${build.dir}" includes="classes/ lib/"/>
	    <fileset dir="${root}" includes="resources/ plugin.xml"/>
	  </jar>
	</target>

修改完成后的build.xml配置为:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
 
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at
 
       http://www.apache.org/licenses/LICENSE-2.0
 
   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
 
<project default="jar" name="eclipse-plugin">
<!-- 
  <import file="../build-contrib.xml"/>
-->
	<import file="build-contrib.xml"/>
	<property name="hadoop.root" location="d:/hq/sdk/hadoop-1.2.1"/>
	<property name="eclipse.home" location="d:/hq/sdk/eclipse" />
	<property name="version" value="1.2.1"/>
 
	<path id="hadoop-jars">
	  <fileset dir="${hadoop.root}/">
	    <include name="hadoop-*.jar"/>
	  </fileset>
	</path>
 
	<path id="classpath">
	  <pathelement location="${build.classes}"/>
	  <pathelement location="${hadoop.root}/build/classes"/>
	  <path refid="eclipse-sdk-jars"/>
	  <path refid="hadoop-jars"/>
	</path>
  <path id="eclipse-sdk-jars">
    <fileset dir="${eclipse.home}/plugins/">
      <include name="org.eclipse.ui*.jar"/>
      <include name="org.eclipse.jdt*.jar"/>
      <include name="org.eclipse.core*.jar"/>
      <include name="org.eclipse.equinox*.jar"/>
      <include name="org.eclipse.debug*.jar"/>
      <include name="org.eclipse.osgi*.jar"/>
      <include name="org.eclipse.swt*.jar"/>
      <include name="org.eclipse.jface*.jar"/>
 
      <include name="org.eclipse.team.cvs.ssh2*.jar"/>
      <include name="com.jcraft.jsch*.jar"/>
    </fileset> 
  </path>
 
  <!-- Override classpath to include Eclipse SDK jars -->
 <!--
  <path id="classpath">
    <pathelement location="${build.classes}"/>
    <pathelement location="${hadoop.root}/build/classes"/>
    <path refid="eclipse-sdk-jars"/>
  </path>
-->
  <!-- Skip building if eclipse.home is unset. -->
  <target name="check-contrib" unless="eclipse.home">
    <property name="skip.contrib" value="yes"/>
    <echo message="eclipse.home unset: skipping eclipse plugin"/>
  </target>
 
 <target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib">
    <echo message="contrib: ${name}"/>
    <javac
     encoding="${build.encoding}"
     srcdir="${src.dir}"
     includes="**/*.java"
     destdir="${build.classes}"
     includeAntRuntime="false"
     debug="${javac.debug}"
     deprecation="${javac.deprecation}">
     <classpath refid="classpath"/>
    </javac>
  </target>
 
  <!-- Override jar target to specify manifest -->
  <!--
  <target name="jar" depends="compile" unless="skip.contrib">
    <mkdir dir="${build.dir}/lib"/>
    <copy file="${hadoop.root}/build/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>
    <copy file="${hadoop.root}/build/ivy/lib/Hadoop/common/commons-cli-${commons-cli.version}.jar"  todir="${build.dir}/lib" verbose="true"/>
    <jar
      jarfile="${build.dir}/hadoop-${name}-${version}.jar"
      manifest="${root}/META-INF/MANIFEST.MF">
      <fileset dir="${build.dir}" includes="classes/ lib/"/>
      <fileset dir="${root}" includes="resources/ plugin.xml"/>
    </jar>
  </target>
  -->
	<target name="jar" depends="compile" unless="skip.contrib">
	  <mkdir dir="${build.dir}/lib"/>
	  <copy file="${hadoop.root}/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-cli-1.2.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-lang-2.4.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-configuration-1.6.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar" todir="${build.dir}/lib" verbose="true"/>
	  <copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar" todir="${build.dir}/lib" verbose="true"/>
	  <jar jarfile="${build.dir}/hadoop-${name}-${version}.jar" manifest="${root}/META-INF/MANIFEST.MF">
	    <fileset dir="${build.dir}" includes="classes/ lib/"/>
	    <fileset dir="${root}" includes="resources/ plugin.xml"/>
	  </jar>
	</target>
</project>

5.6.修改 META-INF/MANIFEST.MF

Bundle-ClassPath: classes/,lib/hadoop-core.jar,lib/commons-cli-1.2.jar,lib/commons-httpclient-3.0.1.jar,lib/jackson-core-asl-1.8.8.jar,lib/jackson-mapper-asl-1.8.8.jar,lib/commons-configuration-1.6.jar,lib/commons-lang-2.4.jar

6.利用ant进行编译。在MapReduceTools项目下,找到build.xml,右键点击,选择”Run as” ->”Ant Build”.在编译成功后,D:\hq\sdk\hadoop-1.2.1\build\contrib\eclipse-plugin\hadoop-eclipse-plugin-1.2.1.jar.把hadoop-eclipse-plugin-1.2.1.jar拷贝到eclipse\plugins目录下。重启eclipse。点击”Open Perspective”,就会看见”Map/Reduce”
map-reduce

7.新建New Hadoop Location
new-hadoop-location
8.定义一个Hadoop Location
define-hadoop-location
Location name: 这里可以随便写一个名字
Map/Reduce Master :
host : 这里写 你在 mapred-site.xml 中设置的地址
(如果你是伪分布式的,这里设置的可能是 localhost:9001)
(用ifconfig查看一下你的虚拟机IP地址,这里的localhost换成虚拟机的IP就OK)
Port: 9001
DFS Master : host:这里是你在core-site.xml 中设置的地址(同上)
port :9000
上面第二个选项我的是默认的设置。 然后 finish 即可。

二. ant 编译警告排除

执行ant编译时,总会出现如下的警告:
[javac] D:\hq\sdk\hadoop-1.2.1\src\contrib\eclipse-plugin\build.xml:77: warning: ‘includeantruntime’ was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
虽然不影响编译,但还是解决才安心。其实解决方法也很简单,只需要根据提示在javac任务中添加includeAntRuntime=”false”属性即可。例如:
修改前:

    <javac srcdir="${srcDir}" destdir="${binDir}" />

修改后:

    <javac srcdir="${srcDir}" destdir="${binDir}" includeAntRuntime="false" />

本文固定链接: http://www.chepoo.com/windows7-hadoop-1-2-1-eclipse-pulgin-compiler.html | IT技术精华网

windows7 hadoop 1.2.1 eclipse pulgin 生成:目前有1 条留言

  1. 沙发
    :

    不错的文章,内容博学多才.禁止此消息:nolinkok@163.com

    [回复]

发表评论