ICode9
精准搜索请尝试： 精确搜索
首页
编程语言>
数据库
系统相关
互联网
其他分享
Python
Java
JavaScript
android
PHP
首页 > 其他分享> 文章详细

|NO.Z.00068|——————————|^^ 部署 ^^|——|Hadoop&PB级数仓.V02|---------------------------------------|PB数仓.v0

2022-04-11 19:34:27  阅读：158  来源： 互联网

标签：INFO --------------------------------------- 1.2 部署 hadoop02 PB atlas Atlas root



[BigDataHadoop：Hadoop&PB级数仓.V02]                                      [BigDataHadoop.PB级企业电商离线数仓][|章节三|Hadoop|元数据管理工具Atlas：Atlas安装配置&账户密码|]







一、安装环境准备
### --- 安装概述说明

~~~     重点讲解Atlas，不对Atlas的依赖组件做讲解，组件均采用单机模式安装。
~~~     编译才能安装。
### --- 安装依赖

~~~     Maven 3.6.3（完成）
~~~     HBase 1.1.2（不需要安装，需要软件包）
~~~     Solr 5.5.1（不需要安装，需要软件包）
~~~     atlas 1.2.0（需要编译）
~~~     官方只提供了源码，没有提供二进制的安装版本，因此Atlas需要编译。
### --- 准备需要的资源包

[root@hadoop02 ~]# ll /opt/yanqi/software/
apache-atlas-1.2.0-sources.tar.gz
apache-maven-3.6.3-bin.tar.gz
hbase-1.1.2.tar.gz
solr-5.5.1.tgz
二、部署maven依赖版本包
### --- 下载maven版本包：apache-maven-3.6.3-bin.tar.gz

[root@hadoop02 ~]# ll /opt/yanqi/software/
apache-maven-3.6.3-bin.tar.gz
### --- 安装maven版本包
~~~     安装maven版本包

[root@hadoop02 ~]# tar -zxvf /opt/yanqi/software/apache-maven-3.6.3-bin.tar.gz \
-C /opt/yanqi/servers/
~~~     # 配置maven环境变量

[root@hadoop02 ~]# vim /etc/profile
##MAVEN_HOME
export MAVEN_HOME=/opt/yanqi/servers/apache-maven-3.6.3
export PATH=$PATH:$MAVEN_HOME/bin 
~~~     # 使环境变量生效

[root@hadoop02 ~]# source /etc/profile
### --- 验证maven版本包是否部署成功
~~~     验证安装

[root@hadoop02 ~]# mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
### --- maven设置阿里镜像
~~~     重要，否则非常慢

~~~     # 添加阿里云源地址
[root@hadoop02 ~]# vim $MAVEN_HOME/conf/settings.xml
# 加在 158 行后
    <mirror>
      <id>alimaven</id>
      <name>aliyun maven</name>
      <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
      <mirrorOf>central</mirrorOf>
    </mirror>
三、安装步骤
### --- 准备软件包

[root@hadoop02 ~]# ll /opt/yanqi/software/
apache-atlas-1.2.0-sources.tar.gz
solr-5.5.1.tgz
hbase-1.1.2.tar.gz
### --- 解压缩源码，修改配置
~~~     # 解压缩

[root@hadoop02 ~]# cd /opt/yanqi/software/
[root@hadoop02 software]# tar -zxvf apache-atlas-1.2.0-sources.tar.gz
[root@hadoop02 software]# cd apache-atlas-sources-1.2.0/
~~~     # 修改配置

[root@hadoop02 apache-atlas-sources-1.2.0]# vim pom.xml
~~~ 修改配置参数
<npm-for-v2.version>3.10.8</npm-for-v2.version>     # 第645行
<hadoop.version>2.9.2</hadoop.version>              # 第652行
### --- 将HBase、Solr的包拷贝到对应的目录中
~~~     如果不拷贝这些包，就需要下载，下载 HBase 和 Solr 时速度很慢。
~~~     这里提前下载完所需的这两个组件，拷贝到对应目录中。

~~~     # 创建文件目录
[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/
[root@hadoop02 apache-atlas-sources-1.2.0]# cd distro/

[root@hadoop02 distro]# mkdir solr
[root@hadoop02 distro]# mkdir hbase
~~~     # 拷贝软件包

[root@hadoop02 distro]# cp /opt/yanqi/software/solr-5.5.1.tgz ./solr/
[root@hadoop02 distro]# cp /opt/yanqi/software/hbase-1.1.2.tar.gz ./hbase/
### --- Atlas编译
~~~     下载依赖包地址
~~~     安装之前设置maven的下载镜像地址为阿里云镜像地址

[root@hadoop02 ~]# ls /root/.m2/repository/
antlr                     bouncycastle  commons-beanutils      commons-daemon      commons-io         concurrent  it              log4j     regexp     xerces    xpp3
aopalliance               cglib         commons-chain          commons-dbcp        commons-lang       de          jakarta-regexp  logkit    ring-cors  xml-apis
asm                       ch            commons-cli            commons-digester    commons-logging    dom4j       javax           nekohtml  sslext     xmlenc
avalon-framework          classworlds   commons-codec          commons-el          commons-net        edu         jline           net       stax       xmlpull
backport-util-concurrent  colt          commons-collections    commons-fileupload  commons-pool       info        joda-time       org       tomcat     xmlunit
biz                       com           commons-configuration  commons-httpclient  commons-validator  io          junit           oro       xalan      xom
~~~     # 编译Atlas版本包

[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/
[root@hadoop02 apache-atlas-sources-1.2.0]# export MAVEN_OPTS="-Xms2g -Xmx2g"
[root@hadoop02 apache-atlas-sources-1.2.0]# mvn clean -DskipTests package -Pdist,embedded-hbase-solr
~~~ 输出参数
~~~ 编译过程中大概要下载600M左右的jar，持续的时间比较长。
[INFO] Building zip: /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/apache-atlas-1.2.0-migration-exporter.zip
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Atlas Server Build Tools 1.0 ................ SUCCESS [ 45.135 s]
[INFO] apache-atlas 1.2.0 ................................. SUCCESS [02:17 min]
[INFO] Apache Atlas Test Utility Tools 1.2.0 .............. SUCCESS [02:09 min]
[INFO] Apache Atlas Integration 1.2.0 ..................... SUCCESS [ 44.858 s]
[INFO] Apache Atlas Common 1.2.0 .......................... SUCCESS [ 10.657 s]
[INFO] Apache Atlas Client 1.2.0 .......................... SUCCESS [  2.263 s]
[INFO] atlas-client-common 1.2.0 .......................... SUCCESS [  4.763 s]
[INFO] atlas-client-v1 1.2.0 .............................. SUCCESS [  4.657 s]
[INFO] Apache Atlas Server API 1.2.0 ...................... SUCCESS [  2.996 s]
[INFO] Apache Atlas Notification 1.2.0 .................... SUCCESS [ 39.452 s]
[INFO] atlas-client-v2 1.2.0 .............................. SUCCESS [  4.233 s]
[INFO] Apache Atlas Graph Database Projects 1.2.0 ......... SUCCESS [  0.257 s]
[INFO] Apache Atlas Graph Database API 1.2.0 .............. SUCCESS [ 10.832 s]
[INFO] Graph Database Common Code 1.2.0 ................... SUCCESS [  3.870 s]
[INFO] Apache Atlas JanusGraph DB Impl 1.2.0 .............. SUCCESS [02:11 min]
[INFO] Apache Atlas Graph Database Implementation Dependencies 1.2.0 SUCCESS [  8.056 s]
[INFO] Shaded version of Apache hbase client 1.2.0 ........ SUCCESS [ 30.243 s]
[INFO] Shaded version of Apache hbase server 1.2.0 ........ SUCCESS [01:15 min]
[INFO] Apache Atlas Authorization 1.2.0 ................... SUCCESS [  6.001 s]
[INFO] Apache Atlas Repository 1.2.0 ...................... SUCCESS [ 52.256 s]
[INFO] Apache Atlas UI 1.2.0 .............................. SUCCESS [07:17 min]
[INFO] Apache Atlas Web Application 1.2.0 ................. SUCCESS [04:01 min]
[INFO] Apache Atlas Documentation 1.2.0 ................... SUCCESS [01:00 min]
[INFO] Apache Atlas FileSystem Model 1.2.0 ................ SUCCESS [ 11.244 s]
[INFO] Apache Atlas Plugin Classloader 1.2.0 .............. SUCCESS [  3.209 s]
[INFO] Apache Atlas Hive Bridge Shim 1.2.0 ................ SUCCESS [ 50.307 s]
[INFO] Apache Atlas Hive Bridge 1.2.0 ..................... SUCCESS [ 31.274 s]
[INFO] Apache Atlas Falcon Bridge Shim 1.2.0 .............. SUCCESS [ 51.045 s]
[INFO] Apache Atlas Falcon Bridge 1.2.0 ................... SUCCESS [  8.699 s]
[INFO] Apache Atlas Sqoop Bridge Shim 1.2.0 ............... SUCCESS [  6.696 s]
[INFO] Apache Atlas Sqoop Bridge 1.2.0 .................... SUCCESS [  8.834 s]
[INFO] Apache Atlas Storm Bridge Shim 1.2.0 ............... SUCCESS [ 11.894 s]
[INFO] Apache Atlas Storm Bridge 1.2.0 .................... SUCCESS [ 10.539 s]
[INFO] Apache Atlas Hbase Bridge Shim 1.2.0 ............... SUCCESS [  4.298 s]
[INFO] Apache Atlas Hbase Bridge 1.2.0 .................... SUCCESS [ 37.611 s]
[INFO] Apache Atlas Kafka Bridge 1.2.0 .................... SUCCESS [  3.626 s]
[INFO] Apache Atlas Distribution 1.2.0 .................... SUCCESS [07:46 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  37:36 min
[INFO] Finished at: 2021-10-11T14:10:31+08:00
[INFO] ------------------------------------------------------------------------
~~~     # 编译完的软件位置：
[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/

~~~     # 编译完的软件：
[root@hadoop02 ~]# ls /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/
apache-atlas-1.2.0-bin.tar.gz
### --- Atlas安装
~~~     进入编译完成后maven目录下

[root@hadoop02 ~]# cd /opt/yanqi/software/apache-atlas-sources-1.2.0/distro/target/
[root@hadoop02 target]# ls apache-atlas-1.2.0-bin.tar.gz 
apache-atlas-1.2.0-bin.tar.gz 
[root@hadoop02 target]# cp apache-atlas-1.2.0-bin.tar.gz /opt/yanqi/software/
~~~     # 解压缩并安装Atlas

[root@hadoop02 ~]# cd /opt/yanqi/software/
[root@hadoop02 software]# tar -zxvf apache-atlas-1.2.0-bin.tar.gz -C ../servers/
[root@hadoop02 software]# cd ../servers/
[root@hadoop02 servers]# mv apache-atlas-1.2.0/ atlas-1.2.0
~~~     # 配置Atlas环境变量
[root@hadoop02 ~]# vim /etc/profile
##ATLAS_HOME
export ATLAS_HOME=/opt/yanqi/servers/atlas-1.2.0
export PATH=$PATH:$ATLAS_HOME/bin
 
~~~     # 使环境变量生效
[root@hadoop02 ~]# source /etc/profile
### --- Atlas启动

~~~     # 启动服务(第一次启动服务的时间比较长)
[root@hadoop02 ~]# cd $ATLAS_HOME/bin
[root@hadoop02 bin]# ./atlas_start.py           # 启动服务
~~~ 输出参数
configured for local hbase.
hbase started.
configured for local solr.
solr started.
setting up solr collections...
starting atlas on host localhost
starting atlas on port 21000
..............................................................................................................................................................................................................................................................................................................
Apache Atlas Server started!!!
[root@hadoop02 bin]# ./atlas_stop.py            # 关闭服务
[root@hadoop02 bin]# ./solr status              # 查看服务状态
~~~     # 检查后台进程 (1个atlas、2个HBase、1个solr后台进程)

[root@hadoop02 ~]# ps -ef | grep atlas
 ~~~ 启动的进程
/opt/yanqi/servers/atlas-1.2.0/server/webapp/atlas
/opt/yanqi/servers/atlas-1.2.0/hbase/bin/hbase-daemon.sh
org.apache.hadoop.hbase.master.HMaster
/opt/yanqi/servers/atlas-1.2.0/solr/server/solr
~~~     # 检查 solr 的状态：

[root@hadoop02 ~]# cd /opt/yanqi/servers/atlas-1.2.0/solr/bin
[root@hadoop02 bin]# ./solr status
~~~ 输出参数
Found 1 Solr nodes: 

Solr process 88510 running on port 9838
{
  "solr_home":"/opt/yanqi/servers/atlas-1.2.0/solr/server/solr",
  "version":"5.5.1 c08f17bca0d9cbf516874d13d221ab100e5b7d58 - anshum - 2016-04-30 13:28:18",
  "startTime":"2021-10-11T06:31:58.698Z",
  "uptime":"0 days, 0 hours, 28 minutes, 56 seconds",
  "memory":"28.9 MB (%5.9) of 490.7 MB",
  "cloud":{
    "ZooKeeper":"localhost:2181",
    "liveNodes":"1",
    "collections":"3"}}
~~~     # 检查zookeper状态：

[root@hadoop02 ~]# echo stat| nc localhost 2181
四、Atlas.UI管理界面
### --- Web服务：http://hadoop02:21000/login.jsp

~~~     用户名 / 口令：admin / admin
~~~     后台管理UI
五、设置Atlas服务的用户名及口令
### --- 设置Atlas服务的用户名和口令

~~~     #  账号的信息存储在文件 conf/users-credentials.properties 中。
[root@hadoop02 ~]# cd /opt/yanqi/servers/atlas-1.2.0/
[root@hadoop02 atlas-1.2.0]# vim conf/users-credentials.properties
~~~ 用户名口令参数
#username=group::sha256-password
admin=ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
rangertagsync=RANGER_TAG_SYNC::e3f67240f5117d1753c940dae9eea772d36ed5fe9bd9c94a300e40413f1afb9d
~~~     # 其中 Password 通过如下方式产生sha256sum 摘要信息：

[root@hadoop02 ~]# echo -n  "admin" | sha256sum
8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918  -
[root@hadoop02 ~]# echo -n  "rangertagsync" | sha256sum
e3f67240f5117d1753c940dae9eea772d36ed5fe9bd9c94a300e40413f1afb9d  -







===============================END===============================

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart                                                                                                                                                   ——W.S.Landor



来自为知笔记(Wiz)

标签：INFO,---------------------------------------,1.2,部署,hadoop02,PB,atlas,Atlas,root
来源： https://www.cnblogs.com/yanqivip/p/16131531.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

关于我们 | 联系我们 | 留言反馈

专注分享技术，共同学习，共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有