ICode9

精准搜索请尝试: 精确搜索
首页 > 其他分享> 文章详细

EMR的hive包升级和gateway启动hs2

2019-12-30 17:02:39  阅读:446  来源: 互联网

标签:current 1.2 lib hive hs2 ranger usr EMR


升级:


mkdir /opt/apps/hive
cd /opt/apps/hive
wget http://emr.oss-cn-hangzhou.aliyuncs.com/ecm-repo/hive/opay/apache-hive-2.3.5-bin.tar.gz
md5sum apache-hive-2.3.5-bin.tar.gz
229e32d99d8d288bdece8c897d6cc625 apache-hive-2.3.5-bin.tar.gz
tar xvf apache-hive-2.3.5-bin.tar.gz

1. Gateway升级Hive包
a.登录Gateway机器,将新的hive包解压到一个目录,比如/opt/apps/hive
b. cp /usr/lib/hive-current/lib/emr-hive-hook*jar /opt/apps/hive/apache-hive-2.3.5-bin/lib
c. unlink /usr/lib/hive-current
d. ln -s /opt/apps/hive/apache-hive-2.3.5-bin /usr/lib/hive-current
e. 建立其它软连接

ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-impl /usr/lib/hive-current/lib/ranger-hive-plugin-impl
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-shim-1.2.0.jar /usr/lib/hive-current/lib/ranger-hive-plugin-shim-1.2.0.jar
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-plugin-classloader-1.2.0.jar /usr/lib/hive-current/lib/ranger-plugin-classloader-1.2.0.jar

ln -s /usr/lib/spark-current/jars/scala-library-2.11.12.jar /usr/lib/hive-current/lib/scala-library-2.11.12.jar
ln -s /usr/lib/spark-current/jars/spark-core_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-core_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-network-common_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-network-common_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-unsafe_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-unsafe_2.11-2.4.3.jar

f. 测试一下是否正常

 

2. HiveServer升级Hive包
a. 登录emr-header-1, 将新的hive包解压到一个目录,比如/opt/apps/hive
b. cp /usr/lib/hive-current/lib/emr-hive-hook*jar /opt/apps/hive/apache-hive-2.3.5-bin/lib
c. unlink /usr/lib/hive-current
d. ln -s /opt/apps/hive/apache-hive-2.3.5-bin /usr/lib/hive-current
e. 建立其它软连接

ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-impl /usr/lib/hive-current/lib/ranger-hive-plugin-impl
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-shim-1.2.0.jar /usr/lib/hive-current/lib/ranger-hive-plugin-shim-1.2.0.jar
ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-plugin-classloader-1.2.0.jar /usr/lib/hive-current/lib/ranger-plugin-classloader-1.2.0.jar

ln -s /usr/lib/spark-current/jars/scala-library-2.11.12.jar /usr/lib/hive-current/lib/scala-library-2.11.12.jar
ln -s /usr/lib/spark-current/jars/spark-core_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-core_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-network-common_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-network-common_2.11-2.4.3.jar
ln -s /usr/lib/spark-current/jars/spark-unsafe_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-unsafe_2.11-2.4.3.jar

f. 登录emr-header-2,重复a~d的操作
g. 页面控制台重启HiveServer
h. 使用beeline/hue测试是否正常


gateway上启动hs2: (可选)
拷贝Ranger相关配置文件
以root用户执⾏以下命令
scp root@emr-header-1:/etc/ecm/hive-conf/ranger-hive-audit.xml /etc/ecm/hive-conf
scp root@emr-header-1:/etc/ecm/hive-conf/ranger-hive-security.xml /etc/ecm/hive-conf
scp root@emr-header-1:/etc/ecm/hive-conf/ranger-policymgr-ssl.xml /etc/ecm/hive-conf
scp root@emr-header-1:/etc/ecm/hive-conf/ranger-security.xml /etc/ecm/hive-conf
chown hadoop:hadoop /etc/ecm/hive-conf/ranger-*
启动HiveServer2
以hadoop用户执行以下命令

标签:current,1.2,lib,hive,hs2,ranger,usr,EMR
来源: https://www.cnblogs.com/hongfeng2019/p/12120596.html

本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享;
2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关;
3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关;
4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除;
5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

专注分享技术,共同学习,共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有