关于如何编译Flume 0.9.4可以参见本博客的《Flume-0.9.4源码编译及一些编译出错解决方法》和《Flume-0.9.4源码编译依赖的thrift插件安装》
　　最近我公司准备将Flume 0.94升级到Flume-ng 1.4，之前在本博客《Flume-0.9.4和Hbase-0.96整合(1)》、《Flume-0.9.4和Hbase-0.96整合(2)》和《Flume-0.9.4配置Hbase sink》等序列文章中涉及到的都是Flume 0.9.4，在之前开发的各种插件得重新编写到支持Flume-ng 1.4，所以今天来谈谈如何基于Hadoop 2.2.0、Hbase 0.96.0编译Flume-ng 1.4.0。之前特别忙，所以都没时间总结。编译Flume-ng的步骤如下：

一、去Flume官方网站下载相应的src

$ wget http://mirror.bit.edu.cn/apache/flume/1.4.0/apache-flume-1.4.0-src.tar.gz
$ tar -zxf apache-flume-1.4.0-src.tar.gz
$ cd apache-flume-1.4.0-src

二、修改相应的pom文件

　　1、修改Hadoop、Hbase相应的版本
　　apache-flume-1.4.0-src是基于Hadoop 1.0.1、Hbase 0.92.1或者Hadoop 2.0.0-alpha、Hbase 0.94.2编写的，而公司用到的Hadoop是2.2.0，Hbase是0.96，所以我们需要修改pom文件如下：

<properties>
        <hadoop.version>2.0.0-alpha</hadoop.version>
        <hbase.version>0.94.2</hbase.version>
        <hadoop.common.artifact.id>hadoop-common</hadoop.common.artifact.id>
        <thrift.version>0.8.0</thrift.version>
</properties>

修改成

<properties>
        <hadoop.version>2.2.0</hadoop.version>
        <hbase.version>0.96.0-hadoop2</hbase.version>
        <hadoop.common.artifact.id>hadoop-common</hadoop.common.artifact.id>
        <thrift.version>0.8.0</thrift.version>
 </properties>

　　2、修改protobuf的版本
　　Flume-ng的用到的protobuf是2.4.1，而目前的Hbase、Hadoop都是2.5.0，所以我们也得修改：

<dependency>
        <groupId>com.google.protobuf</groupId>
        <artifactId>protobuf-java</artifactId>
        <scope>compile</scope>
        <version>2.4.1</version>
</dependency>

修改为

<dependency>
        <groupId>com.google.protobuf</groupId>
        <artifactId>protobuf-java</artifactId>
        <scope>compile</scope>
        <version>2.5.0</version>
</dependency>

　　3、修改flume-ng-sinks中Hbase和Hadoop相应的test
将flume-ng-sinks/flume-hdfs-sink/pom.xml和flume-ng-sinks/flume-ng-hbase-sink/pom.xml文件中的下面依赖注释掉：

<dependency>
          <groupId>org.apache.hadoop</groupId>
          <artifactId>hadoop-test</artifactId>
          <scope>test</scope>
</dependency>

和

<dependency>
      <groupId>org.apache.hbase</groupId>
      <artifactId>hbase</artifactId>
      <classifier>tests</classifier>
      <scope>test</scope>
</dependency>

否则你会在编译的时候遇到下面的错误

[ERROR] Failed to execute goal on project flume-hdfs-sink: Could not 
resolve dependencies for project org.apache.flume.flume-ng-sinks:
flume-hdfs-sink:jar:1.4.0: Could not find artifact 
org.apache.hadoop:hadoop-test:jar:2.0.0-alpha in XXX -> [Help 1]

或者

[ERROR] Failed to execute goal on project flume-ng-hbase-sink: 
Could not resolve dependencies for project org.apache.flume.flume-ng-sinks:
flume-ng-hbase-sink:jar:1.4.0: The following artifacts could 
not be resolved: org.apache.hbase:hbase:jar:0.96.0-hadoop2, 
org.apache.hbase:hbase:jar:tests:0.96.0-hadoop2: 
Could not find artifact org.apache.hbase:hbase:jar:0.96.0-hadoop2 in 
XXXX -> [Help 1]

　　4、修改flume-ng-sinks/flume-ng-morphline-solr-sink的版本
　　flume-ng-morphline-solr-sink的默认为0.4.0，但是目前已经不能从Maven上下载了，所以我们在这修改为如下：

<cdk.version>0.5.0</cdk.version>

否则你会在编译的时候遇到下面的错误：

[ERROR] Failed to execute goal on project flume-ng-morphline-solr-sink: 
Could not resolve dependencies for project org.apache.flume.flume-ng-sinks:
flume-ng-morphline-solr-sink:jar:1.4.0: The following artifacts could 
not be resolved: com.cloudera.cdk:cdk-morphlines-solr-cell:jar:0.4.0, 
com.cloudera.cdk:cdk-morphlines-avro:jar:0.4.0, 
com.cloudera.cdk:cdk-morphlines-twitter:jar:0.4.0, 
com.cloudera.cdk:cdk-morphlines-tika-core:jar:0.4.0, 
com.cloudera.cdk:cdk-morphlines-tika-decompress:jar:0.4.0,
com.cloudera.cdk:cdk-morphlines-solr-core:jar:tests:0.4.0: 
Failure to find com.cloudera.cdk:cdk-morphlines-solr-cell:jar:0.4.0 in
https://repository.cloudera.com/artifactory/cloudera-repos was cached in 
the local repository, resolution will not be reattempted until the update 
interval of cdh.repo has elapsed or updates are forced -> [Help 1]

三、编译Flume源码

$ mvn install -Phadoop-2 -DskipTests -Dtar

编译完成之后，将会出现下面的编译结果：

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Flume ...................................... SUCCESS [1.303s]
[INFO] Flume NG SDK ...................................... SUCCESS [2.811s]
[INFO] Flume NG Configuration ............................ SUCCESS [0.558s]
[INFO] Flume NG Core ..................................... SUCCESS [2.343s]
[INFO] Flume NG Sinks .................................... SUCCESS [0.097s]
[INFO] Flume NG HDFS Sink ................................ SUCCESS [1.602s]
[INFO] Flume NG IRC Sink ................................. SUCCESS [0.610s]
[INFO] Flume NG Channels ................................. SUCCESS [0.067s]
[INFO] Flume NG JDBC channel ............................. SUCCESS [0.637s]
[INFO] Flume NG file-based channel ....................... SUCCESS [0.758s]
[INFO] Flume NG Node ..................................... SUCCESS [0.687s]
[INFO] Flume NG Embedded Agent ........................... SUCCESS [1.063s]
[INFO] Flume NG HBase Sink ............................... SUCCESS [2.976s]
[INFO] Flume NG ElasticSearch Sink ....................... SUCCESS [1.014s]
[INFO] Flume NG Morphline Solr Sink ...................... SUCCESS [3.125s]
[INFO] Flume Sources ..................................... SUCCESS [0.043s]
[INFO] Flume Scribe Source ............................... SUCCESS [0.865s]
[INFO] Flume JMS Source .................................. SUCCESS [1.013s]
[INFO] Flume legacy Sources .............................. SUCCESS [0.046s]
[INFO] Flume legacy Avro source .......................... SUCCESS [1.466s]
[INFO] Flume legacy Thrift Source ........................ SUCCESS [1.035s]
[INFO] Flume NG Clients .................................. SUCCESS [0.090s]
[INFO] Flume NG Log4j Appender ........................... SUCCESS [4.080s]
[INFO] Flume NG Tools .................................... SUCCESS [1.965s]
[INFO] Flume NG distribution ............................. SUCCESS [4:25.620s]
[INFO] Flume NG Integration Tests ........................ SUCCESS [2.003s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:58.453s
[INFO] Finished at: Tue May 13 22:51:20 CST 2014
[INFO] Final Memory: 127M/708M

最后将会在flume-ng-dist/target目录下产生以下的文件：

drwxr-xr-x 3 wyp wyp     4096 May 13 22:48 apache-flume-1.4.0-bin
-rw-r--r-- 1 wyp wyp 59087227 May 13 22:49 apache-flume-1.4.0-bin.tar.gz
drwxr-xr-x 3 wyp wyp     4096 May 13 22:49 apache-flume-1.4.0-src
-rw-r--r-- 1 wyp wyp  2496952 May 13 22:51 apache-flume-1.4.0-src.tar.gz
drwxr-xr-x 2 wyp wyp     4096 May 13 22:47 archive-tmp
drwxr-xr-x 3 wyp wyp     4096 May 13 22:47 maven-shared-archive-resources
-rw-r--r-- 1 wyp wyp     1005 May 13 22:51 rat.txt

apache-flume-1.4.0-bin里面的东西我们可以直接拷贝出来，直接用了，关于如何使用flume-ng，我将会在后面的文章介绍，敬请关注本博客！