Docker中mongo副本集一直Restarting处理过程

环境说明

docker-composedocker-compose version 1.24.1, build 4667896b
dockerDocker version 19.03.11, build 42e35e61f3
mongov4.2.7
centosCentOS Linux release 7.8.2003 (Core)

问题说明

docker mongo副本集无法启动（如下）

➜  axinblog docker-compose ps   
Name                      Command                 State                       Ports
-------------------------------------------------------------------------------------------------------------
...                                      
mongodb_primary     bash -c chmod 400 /data/mo ...   Restarting                                       
mongodb_replica_1   bash -c chmod 400 /data/mo ...   Up           0.0.0.0:27018->27017/tcp            
mongodb_replica_2   bash -c chmod 400 /data/mo ...   Up           0.0.0.0:27019->27017/tcp            
...

步骤说明

查看日志

docker ps 先找到出问题的容器ID

➜  containers docker ps
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                           PORTS                                      NAMES
4736e67fbd2b        mongodb_primary   "bash -c 'chmod 400 …"   4 months ago        Restarting (14) 34 seconds ago                                              mongodb_primary

方式1：

cd
vim 4736e67fbd2bf513a70c115312f2bc208a1767fb4da9a23937227ddbc5bff1c0-json.log

方式2:

docker logs -f -t --since="日期" --tail=日志记录数 容器ID

docker logs -f -t --since="2020-11-13" --tail=1000 4736e67fbd2b

2020-11-13T09:01:16.191606239Z 2020-11-13T09:01:16.191+0000 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-11-13T09:01:16.196272875Z 2020-11-13T09:01:16.196+0000 W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
2020-11-13T09:01:16.287181977Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=4736e67fbd2b
2020-11-13T09:01:16.287216770Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] db version v4.2.7
2020-11-13T09:01:16.287221031Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] git version: 51d9fe12b5d19720e72dcd7db0f2f17dd9a19212
2020-11-13T09:01:16.287224322Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
2020-11-13T09:01:16.287249167Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-11-13T09:01:16.287253400Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] modules: none
2020-11-13T09:01:16.287256698Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] build environment:
2020-11-13T09:01:16.287259765Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten]     distmod: ubuntu1804
2020-11-13T09:01:16.287262814Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten]     distarch: x86_64
2020-11-13T09:01:16.287265993Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-11-13T09:01:16.287268955Z 2020-11-13T09:01:16.286+0000 I  CONTROL  [initandlisten] options: { net: { bindIp: "*" }, replication: { replSet: "mongos" }, security: { authorization: "enabled", keyFile: "/data/mongodb.key" } }
2020-11-13T09:01:16.287274257Z 2020-11-13T09:01:16.286+0000 E  NETWORK  [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
2020-11-13T09:01:16.287290311Z 2020-11-13T09:01:16.286+0000 F  -        [initandlisten] Fatal Assertion 40486 at src/mongo/transport/transport_layer_asio.cpp 684
2020-11-13T09:01:16.287293285Z 2020-11-13T09:01:16.286+0000 F  -        [initandlisten] 
2020-11-13T09:01:16.287295907Z 
2020-11-13T09:01:16.287298537Z ***aborting after fassert() failure

分析问题

Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted 这问题看起来是没有权限的问题谷歌一下

解决方式

方法一：映射tmp 使mongodb-27017文件拥有相应权限

修改docker-compose.yml，增加 ./db/mongodb/primary/tmp:/tmp卷映射

$ vim docker-compose.yml
mongodb_primary:
 container_name: mongodb_primary
 build:
   context: ./db/mongodb/docker/
 ports:
   - "27017:27017"
 restart: always
 environment:
   MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
   MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
   MONGO_DATA_DIR: ${MONGO_DATA_DIR}
   MONGO_LOG_DIR: ${MONGO_LOG_DIR}
 volumes:
   - ./db/mongodb/primary/data:/data/db
   - ./db/mongodb/primary/example_db:/data/example_db
   - ./db/mongodb/primary/logs:/data/logs
   - ./db/mongodb/mongodb.key:/data/mongodb.key
   - ./db/mongodb/primary/tmp:/tmp

重新build启动容器

$ docker-compose stop
// 注：停止正在运行的容器，可以通过docker-compose start 再次启动
$ docker-compose down --volumes
// 注：–volumes，删除已经在compose文件中定义的和匿名的附在容器上的数据卷
docker-compose build --no-cache
// 注：–no-cache 构建镜像过程中不使用缓存

到你的tmp目录中查看是否有mongodb-27017.sock，有表示成功
➜  tmp ll
total 0
srwx------ 1 polkitd ssh_keys 0 Nov 13 17:43 mongodb-27017.sock

如果发现重新build没有目录生成一般都是权限不足

首先进入容器查看mongo用的权限是什么

$ docker-compose exec mongodb_primary bash
root@f52e93626d9f:/# cat /etc/pa
···
mongodb:x:999:999::/home/mongodb:/bin/sh
···

设置你本地目录权限与mongo一致就行了

chown 999:999 tmp

方法二：删除mongodb-27017.sock，重启mongo服务

sudo rm /tmp/mongodb-27017.sock

因为我这边是进不了容器的，所以无法使用

参考文献

docker 下的mongodb 日志查看