ICode9
精准搜索请尝试： 精确搜索
首页
编程语言>
数据库
系统相关
互联网
其他分享
Python
Java
JavaScript
android
PHP
首页 > 其他分享> 文章详细

mongodump和mongorestore

2022-05-20 00:02:15  阅读：136  来源： 互联网

标签：10 simon 25T10 mongorestore mongodump 0800 2020 gzip


mongodump和mongorestore

介绍

mongodump能够在Mongodb运行时进行备份，它的工作原理是对运行的Mongodb做查询，然后将所有查到的文档写入磁盘。
但是存在的问题时使用mongodump产生的备份不一定是数据库的实时快照，如果我们在备份时对数据库进行了写入操作，
则备份出来的文件可能不完全和Mongodb实时数据相等。另外在备份时可能会对其它客户端性能产生不利的影响。

全量备份

备份可在RS副本集内任一节点执行，最好在从库

PS:mongodump备份时不需要锁库，可以通过--oplog参数将备份时段的变更记录以日志形式一起备份出来(单实例不支持oplog)

mongodump全量备份会转储admin库下的users和version，所有的用户库和备份时段内的oplog，

转储格式为bson格式的数据文件和json格式的元数据文件

增量备份（oplog日志备份）

只有副本集可以增量备份，单实例没有oplog无法增备
根据全量备份时段备份的的oplog,获取全量备份完成的时间戳

bsondump oplog.bson


可以得出时间戳为（1593309601,1），后期可以通过脚本解析json直接获取

这里说明一下：

MongoDB中的时间戳为(timestamp,i)的形式，其中timestamp为十位时间戳，精确到秒，i表示累增序数，记录着该时间戳内有多少并发操作。

# 基于时间戳的增量备份
mongodump -u admin -p 123456 -h localhost --port 27017 --authenticationDatabase admin -d local -c oplog.rs --query '{ts:{$gt:Timestamp(1593309601,1)}}' -o /var/lib/mongobak/mongodump_oplog_`date +%Y-%m-%d_%H:%M:%S`

全量恢复

mongorestore恢复时，需要服务开启且停止业务写入，防止数据恢复时写入导致数据错乱

同机（副本集）全量恢复
同副本集的恢复，需要指定主库ip,在primary主库上做恢复，通过oplog同步给从库

mongorestore -h 192.168.73.120 --port 27018 -u admin -p 123456 --authenticationDatabase admin --oplogReplay --dir /var/lib/mongobak/mongodump_2020-06-28

PS:mongorestore只针对insert做恢复，不会恢复update。对于同一个_id的文档，mongorestore会认定为重复并跳过。
可以使用--drop,在import前先删除collection

mongodump用法如下

$ mongodump --help
参数说明：
-h:指明数据库宿主机的IP
-u:指明数据库的用户名
-p:指明数据库的密码
-d:指明数据库的名字
-c:指明collection的名字
-o:指明到要导出的文件名
-q:指明导出数据的过滤条件
-j, --numParallelCollections=  number of collections to dump in parallel (4 by default)
--oplog  备份的同时备份oplog

创建备份用户

# 选择认证库
use admin

# 创建备份用户
db.createUser(
{
    user: "backup",
    pwd: "backup",
    roles: [ 
    	{ 
    	role: "backup", 	//拥有备份权限
    	db: "admin" 		//认证库
    	},
        { 
    	role: "restore", 	//拥有恢复权限
    	db: "admin" 		//认证库
    	}
    ]
}
)

# 验证用户
db.auth('backup','backup')

use admin
db.createUser(
{
    user: "backup",
    pwd: "backup",
    roles: [ 
    	        {
    	role: "read",
    	db: "local"
    	},{ 
    	role: "backup",
    	db: "admin"
    	},
        {
    	role: "restore",
    	db: "admin"
    	}
    ]
}
)



# 创建恢复用户
db.createUser(
{
    user: "restore",
    pwd: "restore",
    roles: [ { role: "restore", db: "admin" } ]
}
)
# 验证用户
db.auth('restore','restore')

mongodump和mongorestore基本使用

全库备份

mkdir -p /mongodb/backup/full
mongodump  -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup/full

[mongod@mysql-node01 mongodb]$ mongodump -uroot -pw --authenticationDatabase admin -o /mongodb/backup/full/
2020-10-25T10:18:59.040+0800	writing admin.system.users to 
2020-10-25T10:18:59.041+0800	done dumping admin.system.users (1 document)
2020-10-25T10:18:59.041+0800	writing admin.system.version to 
2020-10-25T10:18:59.041+0800	done dumping admin.system.version (2 documents)
2020-10-25T10:18:59.041+0800	writing simon.log3 to 
2020-10-25T10:18:59.041+0800	writing simon.log to 
2020-10-25T10:18:59.041+0800	writing simon.log1 to 
2020-10-25T10:18:59.041+0800	writing simon.log2 to 
2020-10-25T10:18:59.071+0800	done dumping simon.log3 (10000 documents)
2020-10-25T10:18:59.071+0800	writing simon.city to 
2020-10-25T10:18:59.072+0800	done dumping simon.log (10000 documents)
2020-10-25T10:18:59.072+0800	writing simon.city2 to 
2020-10-25T10:18:59.089+0800	done dumping simon.city (4079 documents)
2020-10-25T10:18:59.089+0800	writing simon.test to 
2020-10-25T10:18:59.092+0800	done dumping simon.log2 (10000 documents)
2020-10-25T10:18:59.092+0800	done dumping simon.log1 (10000 documents)
2020-10-25T10:18:59.093+0800	done dumping simon.test (1 document)
2020-10-25T10:18:59.095+0800	done dumping simon.city2 (4079 documents)
[mongod@mysql-node01 mongodb]$
[mongod@mysql-node01 mongodb]$ cd !$
cd /mongodb/backup/full/
[mongod@mysql-node01 full]$ ls
admin  simon
[mongod@mysql-node01 full]$ ls simon/
city2.bson           log1.bson           log3.bson           test.bson
city2.metadata.json  log1.metadata.json  log3.metadata.json  test.metadata.json
city.bson            log2.bson           log.bson
city.metadata.json   log2.metadata.json  log.metadata.json
[mongod@mysql-node01 full]$

备份simon库

mkdir -p /mongodb/backup/simon
$ mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -o /mongodb/backup/simon

[mongod@mysql-node01 full]$ mongodump -uroot -pw --authenticationDatabase admin -d simon -o /mongodb/backup/simon/
2020-10-25T10:21:22.039+0800	writing simon.log3 to 
2020-10-25T10:21:22.039+0800	writing simon.log to 
2020-10-25T10:21:22.039+0800	writing simon.log1 to 
2020-10-25T10:21:22.039+0800	writing simon.log2 to 
2020-10-25T10:21:22.059+0800	done dumping simon.log3 (10000 documents)
2020-10-25T10:21:22.059+0800	writing simon.city to 
2020-10-25T10:21:22.065+0800	done dumping simon.log (10000 documents)
2020-10-25T10:21:22.065+0800	writing simon.city2 to 
2020-10-25T10:21:22.072+0800	done dumping simon.city (4079 documents)
2020-10-25T10:21:22.072+0800	writing simon.test to 
2020-10-25T10:21:22.073+0800	done dumping simon.city2 (4079 documents)
2020-10-25T10:21:22.074+0800	done dumping simon.test (1 document)
2020-10-25T10:21:22.090+0800	done dumping simon.log1 (10000 documents)
2020-10-25T10:21:22.093+0800	done dumping simon.log2 (10000 documents)
[mongod@mysql-node01 full]$ 
[mongod@mysql-node01 full]$ 
[mongod@mysql-node01 full]$ 
[mongod@mysql-node01 full]$ cd /mongodb/backup/simon/
[mongod@mysql-node01 simon]$ ls
simon
[mongod@mysql-node01 simon]$ ls simon/
city2.bson           log1.bson           log3.bson           test.bson
city2.metadata.json  log1.metadata.json  log3.metadata.json  test.metadata.json
city.bson            log2.bson           log.bson
city.metadata.json   log2.metadata.json  log.metadata.json
[mongod@mysql-node01 simon]$

备份simon库下的log集合

$ mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -c log -o /mongodb/backup/simon

[mongod@mysql-node01 simon]$ mongodump -uroot -pw --authenticationDatabase admin -d simon -c log3 -o /mongodb/backup/simon/
2020-10-25T10:22:47.637+0800	writing simon.log3 to 
2020-10-25T10:22:47.651+0800	done dumping simon.log3 (10000 documents)
[mongod@mysql-node01 simon]$

压缩备份

$ mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -d simon -o /mongodb/backup/ --gzip
mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup/ --gzip
$ mongodump   -uroot -proot123 --port 27017 --authenticationDatabase admin -d app -c vast -o /mongodb/backup/ --gzip


[mongod@mysql-node01 simon]$ mkdir -p /mongodb/backup/full-gzip
[mongod@mysql-node01 simon]$ mongodump -uroot -pw --authenticationDatabase admin -o /mongodb/backup/full-gzip/ --gzip
2020-10-25T10:27:49.023+0800	writing admin.system.users to 
2020-10-25T10:27:49.024+0800	done dumping admin.system.users (1 document)
2020-10-25T10:27:49.024+0800	writing admin.system.version to 
2020-10-25T10:27:49.024+0800	done dumping admin.system.version (2 documents)
2020-10-25T10:27:49.024+0800	writing simon.log3 to 
2020-10-25T10:27:49.024+0800	writing simon.log to 
2020-10-25T10:27:49.024+0800	writing simon.log1 to 
2020-10-25T10:27:49.025+0800	writing simon.log2 to 
2020-10-25T10:27:49.060+0800	done dumping simon.log (10000 documents)
2020-10-25T10:27:49.060+0800	writing simon.city to 
2020-10-25T10:27:49.067+0800	done dumping simon.log3 (10000 documents)
2020-10-25T10:27:49.067+0800	writing simon.city2 to 
2020-10-25T10:27:49.086+0800	done dumping simon.city (4079 documents)
2020-10-25T10:27:49.086+0800	writing simon.test to 
2020-10-25T10:27:49.094+0800	done dumping simon.log2 (10000 documents)
2020-10-25T10:27:49.107+0800	done dumping simon.log1 (10000 documents)
2020-10-25T10:27:49.108+0800	done dumping simon.test (1 document)
2020-10-25T10:27:49.111+0800	done dumping simon.city2 (4079 documents)
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ du -sh /mongodb/backup/full
4.3M	/mongodb/backup/full
[mongod@mysql-node01 simon]$ du -sh /mongodb/backup/full-gzip/
424K	/mongodb/backup/full-gzip/
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ cd /mongodb/backup/full-gzip/
[mongod@mysql-node01 simon]$ pwd
/mongodb/backup/full-gzip
[mongod@mysql-node01 simon]$ ls	
[mongod@mysql-node01 full-gzip]$ ls simon/
city2.bson.gz           log1.metadata.json.gz  log.bson.gz
city2.metadata.json.gz  log2.bson.gz           log.metadata.json.gz
city.bson.gz            log2.metadata.json.gz  test.bson.gz
city.metadata.json.gz   log3.bson.gz           test.metadata.json.gz
log1.bson.gz            log3.metadata.json.gz
[mongod@mysql-node01 full-gzip]$

从全备中恢复指定的库

恢复simon库到 simon_gzip

$ mongorestore -uroot -pw --authenticationDatabase admin -d simon_gzip /mongodb/backup/full-gzip/simon --gzip


[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d simon_gzip /mongodb/backup/full-gzip/simon --gzip
2020-10-25T10:34:28.034+0800	the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-10-25T10:34:28.034+0800	building a list of collections to restore from /mongodb/backup/full-gzip/simon dir
2020-10-25T10:34:28.034+0800	reading metadata for simon_gzip.city from /mongodb/backup/full-gzip/simon/city.metadata.json.gz
2020-10-25T10:34:28.046+0800	reading metadata for simon_gzip.city2 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:34:28.046+0800	reading metadata for simon_gzip.log2 from /mongodb/backup/full-gzip/simon/log2.metadata.json.gz
2020-10-25T10:34:28.046+0800	reading metadata for simon_gzip.log3 from /mongodb/backup/full-gzip/simon/log3.metadata.json.gz
2020-10-25T10:34:28.046+0800	restoring simon_gzip.city from /mongodb/backup/full-gzip/simon/city.bson.gz
2020-10-25T10:34:28.064+0800	restoring simon_gzip.log2 from /mongodb/backup/full-gzip/simon/log2.bson.gz
2020-10-25T10:34:28.070+0800	restoring simon_gzip.city2 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:34:28.087+0800	restoring simon_gzip.log3 from /mongodb/backup/full-gzip/simon/log3.bson.gz
2020-10-25T10:34:28.104+0800	no indexes to restore
2020-10-25T10:34:28.104+0800	finished restoring simon_gzip.city2 (4079 documents)
2020-10-25T10:34:28.104+0800	reading metadata for simon_gzip.log1 from /mongodb/backup/full-gzip/simon/log1.metadata.json.gz
2020-10-25T10:34:28.122+0800	restoring simon_gzip.log1 from /mongodb/backup/full-gzip/simon/log1.bson.gz
2020-10-25T10:34:28.123+0800	no indexes to restore
2020-10-25T10:34:28.123+0800	finished restoring simon_gzip.city (4079 documents)
2020-10-25T10:34:28.123+0800	reading metadata for simon_gzip.log from /mongodb/backup/full-gzip/simon/log.metadata.json.gz
2020-10-25T10:34:28.135+0800	restoring simon_gzip.log from /mongodb/backup/full-gzip/simon/log.bson.gz
2020-10-25T10:34:28.164+0800	no indexes to restore
2020-10-25T10:34:28.164+0800	finished restoring simon_gzip.log2 (10000 documents)
2020-10-25T10:34:28.164+0800	reading metadata for simon_gzip.test from /mongodb/backup/full-gzip/simon/test.metadata.json.gz
2020-10-25T10:34:28.167+0800	no indexes to restore
2020-10-25T10:34:28.167+0800	finished restoring simon_gzip.log3 (10000 documents)
2020-10-25T10:34:28.184+0800	restoring simon_gzip.test from /mongodb/backup/full-gzip/simon/test.bson.gz
2020-10-25T10:34:28.201+0800	no indexes to restore
2020-10-25T10:34:28.201+0800	finished restoring simon_gzip.test (1 document)
2020-10-25T10:34:28.219+0800	no indexes to restore
2020-10-25T10:34:28.219+0800	finished restoring simon_gzip.log1 (10000 documents)
2020-10-25T10:34:28.219+0800	no indexes to restore
2020-10-25T10:34:28.219+0800	finished restoring simon_gzip.log (10000 documents)
2020-10-25T10:34:28.219+0800	done
[mongod@mysql-node01 simon]$ 
    
    
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password: 
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("013a134e-1322-4900-8466-62591614e91d") }
MongoDB server version: 3.6.20 
> 
> show dbs
admin       0.000GB
config      0.000GB
local       0.000GB
simon       0.002GB
simon_gzip  0.000GB
> use simon_gzip
switched to db simon_gzip
> show tables
city
city2
log
log1
log2
log3
test
>
> db.city2.count( { "CountryCode" : "CHN" } )
363
>

从全备中恢复指定的集合/表

恢复simon库下的city2集合

# 恢复指定的集合，需要指定具体的集合文件

mongorestore -uroot -pw --authenticationDatabase admin -d world -c t1 --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz


[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ 
[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c t1 --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz 
2020-10-25T10:44:38.259+0800	checking for collection data in /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:44:38.259+0800	reading metadata for world.t1 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:44:38.266+0800	restoring world.t1 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:44:38.330+0800	no indexes to restore
2020-10-25T10:44:38.330+0800	finished restoring world.t1 (4079 documents)
2020-10-25T10:44:38.330+0800	done
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password: 
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("873b4e0b-d53a-479c-8aba-4fac57b2d1c2") }
MongoDB server version: 3.6.20
> 
> use world
switched to db world
> db.t1.count()
4079
> 
> db.t1.count( { "CountryCode" : "CHN" } )
363
>

drop表示恢复的时候把之前的集合drop掉(危险)

# Drop 会把原来的集合删除，然后再创建新的集合

$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c city2 --drop --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz

[mongod@mysql-node01 simon]$ mongorestore -uroot -pw --authenticationDatabase admin -d world -c city2 --drop --gzip /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.074+0800	checking for collection data in /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.075+0800	reading metadata for world.city2 from /mongodb/backup/full-gzip/simon/city2.metadata.json.gz
2020-10-25T10:50:57.083+0800	restoring world.city2 from /mongodb/backup/full-gzip/simon/city2.bson.gz
2020-10-25T10:50:57.145+0800	no indexes to restore
2020-10-25T10:50:57.145+0800	finished restoring world.city2 (4079 documents)
2020-10-25T10:50:57.145+0800	done
[mongod@mysql-node01 simon]$
[mongod@mysql-node01 simon]$ mongo -uroot -p
MongoDB shell version v3.6.20
Enter password: 
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("636470f7-c0a7-4cc8-a358-933e98d08109") }
MongoDB server version: 3.6.20
> use world
switched to db world
> show tables
city2
t1
> db.city2.count( { CountryCode : 'CHN' } )
363
>

标签：10,simon,25T10,mongorestore,mongodump,0800,2020,gzip
来源： https://www.cnblogs.com/oldSimon/p/16290815.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

关于我们 | 联系我们 | 留言反馈

专注分享技术，共同学习，共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有