标签：INFO SUCCESS flink Flink1.9 hadoop 7.0 源码 centos7.2 shaded
编译步骤参照官网：https://ci.apache.org/projects/flink/flink-docs-release-1.9/flinkDev/building.html
0.按照官网：In addition you need Maven 3 and a JDK (Java Development Kit). Flink requires at least Java 8 to build.配置好前置条件。
1.github获取flink 最新代码
git clone https://github.com/apache/flink
2.切换到flink 1.9分支
git checkout release-1.9
3.查看自己的maven版本
[hadoop@hadoop002 ~]$ mvn -v
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-05T03:00:29+08:00)
Maven home: /home/hadoop/app/apache-maven-3.6.1
Java version: 1.8.0_161, vendor: Oracle Corporation, runtime: /usr/java/jdk1.8.0_161/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-327.el7.x86_64", arch: "amd64", family: "unix"
[hadoop@hadoop002 ~]$
如果版本是 3.1.x and 3.2.x 则直接 mvn clean install -DskipTests
如果版本是 3.3.x，则采用如下步骤
mvn clean install -DskipTests -Dfast
cd flink-dist
mvn clean install
4.如要在其他第三方厂家使用flink 打包，例如要在cdh ,Hortonworks 平台上使用，我们需要指定hadoop 版本进行打包。那么要先对flink 的前置依赖 flink-shaded-hadoop-2 进行打包，在进行编译。
新建一个目录，从git上获取flink-shaded 的源码
git clone https://github.com/apache/flink-shaded.git
根据自己缺少的版本切换对应的代码分支，这里我缺少的是7.0版本的flink-shaded-hadoop-2
git checkout release-7.0
修改flink-shaded项目中的pom.xml 这里修改是为了加入cdh 等中央仓库，否则编译对应版本可能找不到cdh 相关的包。
在pom.xml文件中的<profiles></profiles>添加
<profile>
<id>vendor-repos</id>
<activation>
<property>
<name>vendor-repos</name>
</property>
</activation>
<!-- Add vendor maven repositories -->
<repositories>
<!-- Cloudera -->
<repository>
<id>cloudera-releases</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<!-- Hortonworks -->
<repository>
<id>HDPReleases</id>
<name>HDP Releases</name>
<url>https://repo.hortonworks.com/content/repositories/releases/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
<repository>
<id>HortonworksJettyHadoop</id>
<name>HDP Jetty</name>
<url>https://repo.hortonworks.com/content/repositories/jetty-hadoop</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
<!-- MapR -->
<repository>
<id>mapr-releases</id>
<url>https://repository.mapr.com/maven/</url>
<snapshots><enabled>false</enabled></snapshots>
<releases><enabled>true</enabled></releases>
</repository>
</repositories>
</profile>
然后选择自己需要的CDH版本进行编译 flink-shaded-hadoop-2
mvn clean install -DskipTests -Drat.skip=true -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.15.1
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] flink-shaded 7.0 ................................... SUCCESS [ 1.619 s]
[INFO] flink-shaded-force-shading 7.0 ..................... SUCCESS [ 0.446 s]
[INFO] flink-shaded-asm-6 6.2.1-7.0 ....................... SUCCESS [ 0.563 s]
[INFO] flink-shaded-guava-18 18.0-7.0 ..................... SUCCESS [ 1.176 s]
[INFO] flink-shaded-netty-4 4.1.32.Final-7.0 .............. SUCCESS [ 2.720 s]
[INFO] flink-shaded-netty-tcnative-dynamic 2.0.25.Final-7.0 SUCCESS [ 0.625 s]
[INFO] flink-shaded-jackson-parent 2.9.8-7.0 .............. SUCCESS [ 0.014 s]
[INFO] flink-shaded-jackson-2 2.9.8-7.0 ................... SUCCESS [ 1.097 s]
[INFO] flink-shaded-jackson-module-jsonSchema-2 2.9.8-7.0 . SUCCESS [ 0.770 s]
[INFO] flink-shaded-hadoop-2 2.6.0-cdh5.15.1-7.0 .......... SUCCESS [ 10.969 s]
[INFO] flink-shaded-hadoop-2-uber 2.6.0-cdh5.15.1-7.0 ..... SUCCESS [ 21.086 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 41.375 s
[INFO] Finished at: 2020-02-02T06:45:24+08:00
[INFO] ------------------------------------------------------------------------
flink-shaded-hadoop-2编译成功之后，在切换到flink的源码目录，因之前已经在根目录编译过了，切换到flink-dist，进行flink的编译：
mvn clean install -DskipTests -Pvendor-repos -Drat.skip=true -Pinclude-hadoop -Dhadoop.version=2.6.0-cdh5.15.1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 23.090 s
[INFO] Finished at: 2020-02-02T06:45:56+08:00
[INFO] ------------------------------------------------------------------------
在..flink/flink-dist/target/flink-1.9-SNAPSHOT-bin/flink-1.9-SNAPSHOT中，找到自己需要的flink bin包
[hadoop@hadoop002 flink-1.9-SNAPSHOT-bin]$ ll
total 0
drwxrwxr-x 9 hadoop hadoop 117 Feb 2 06:45 flink-1.9-SNAPSHOT
muyingmiao
发布了134 篇原创文章 · 获赞 4 · 访问量 1万+
私信
关注
标签：INFO,SUCCESS,flink,Flink1.9,hadoop,7.0,源码,centos7.2,shaded 来源： https://blog.csdn.net/muyingmiao/article/details/104141399
本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享； 2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关； 3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关； 4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除； 5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。