Docker存储驱动direct-lvm的配置

devicemapper是Linux发行版RHEL下Docker Engine的默认存储驱动，它有两种配置模式: loop-lvm和direct-lvm

• loop-lvm是默认模式，基于loop设备文件创建thin pool来存储docker镜像及容器数据，官方不推荐在生产环境中使用此模式
• direct-lvm是直接使用块设备创建thin pool，在中等负载和高密度环境下会有更好的性能优势，也是官方推荐在生产环境中使用的模式

基础环境准备

#

操作系统: CentOS7.2

Docker Engine: docker-engine-1.13.1-1.el7.centos.x86_64

硬盘: /dev/sda,/dev/sdb

$ lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  200G  0 disk 
├─sda1            8:1    0  500M  0 part /boot
└─sda2            8:2    0  190G  0 part 
  ├─centos-root 253:0    0   90G  0 lvm  /
  └─centos-data 253:1    0  100G  0 lvm  /data
sdb               8:16   0  100G  0 disk
sr0              11:0    1  603M  0 rom  

划分逻辑卷

#

• 基于/dev/sdb制作逻辑卷组appvg, 并划分逻辑卷appvg-docker，用于挂载docker graph目录(/docker)

$ pvcreate /dev/sdb
  Physical volume "/dev/sdb" successfully created
$ vgcreate appvg /dev/sdb
  Volume group "appvg" successfully created

#规划20G用于逻辑卷/dev/appvg/docker
$ lvcreate -n docker -L 20G appvg
  Logical volume "docker" created.
  
$ lvs appvg
  LV     VG    Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  docker appvg -wi-a----- 20.00g                                                                 

$ mkfs.xfs /dev/appvg/docker 
meta-data=/dev/appvg/docker      isize=256    agcount=4, agsize=1310720 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=5242880, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

$ mkdir /docker
$ mount /dev/appvg/docker /docker

#开机自动挂载
$ tail -n1 /etc/fstab 
/dev/mapper/appvg-docker /docker                xfs     defaults         0 0

$ lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  200G  0 disk 
├─sda1            8:1    0  500M  0 part /boot
└─sda2            8:2    0  190G  0 part 
  ├─centos-root 253:0    0   90G  0 lvm  /
  └─centos-data 253:1    0  100G  0 lvm  /data
sdb               8:16   0  100G  0 disk 
└─appvg-docker  253:2    0   20G  0 lvm  /docker
sr0              11:0    1  603M  0 rom  

• 基于逻辑卷组appvg, 创建thinpool和thinpoolmeta

$ lvcreate --wipesignatures y -n thinpool -L 40G appvg 
  Logical volume "thinpool" created.
$ lvcreate --wipesignatures y -n thinpoolmeta -L 800M appvg
  Logical volume "thinpoolmeta" created.
  
#将逻辑卷thinpool和thinpoolmeta转换为thin pool
$ lvconvert -y --zero n -c512K --thinpool appvg/thinpool --poolmetadata appvg/thinpoolmeta
  WARNING: Converting logical volume appvg/thinpool and appvg/thinpoolmeta to pool's data and metadata volumes.
  THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
  Converted appvg/thinpool to thin pool.

#创建thinpool自动扩容配置文件
$vi /etc/lvm/profile/appvg-thinpool.profile
activation {
        thin_pool_autoextend_threshold=60
        thin_pool_autoextend_percent=20
}

#应用thinpool配置文件
$ lvchange --metadataprofile appvg-thinpool appvg/thinpool
  Logical volume "thinpool" changed.

#查看thinpool处于监控状态
$ lvs -o+seg_monitor appvg
  LV       VG    Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Monitor  
  docker   appvg -wi-ao---- 20.00g                                                              
  thinpool appvg twi-a-t--- 40.00g             0.00   0.01                             monitored

启动docker服务

#

• 创建docker配置文件/etc/docker/daemon.json

$ cat /etc/docker/daemon.json
{
        "insecure-registries": ["0.0.0.0/0"],
        "hosts": ["unix:///var/run/docker.sock", "tcp://0.0.0.0:3666"],
        "graph": "/docker",
        "storage-driver": "devicemapper",
        "storage-opts": [
                "dm.thinpooldev=/dev/mapper/appvg-thinpool",
                "dm.use_deferred_deletion=true",
                "dm.use_deferred_removal=true"
        ]
}

• 启动docker服务并执行docker info

$ systemctl start docker

$ docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.13.1
Storage Driver: devicemapper
 Pool Name: appvg-thinpool                                         --> 显示为创建的逻辑卷thinpool
 Pool Blocksize: 524.3 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: 
 Metadata file: 
 Data Space Used: 20.45 MB
 Data Space Total: 42.95 GB
 Data Space Available: 42.93 GB
 Metadata Space Used: 110.6 kB
 Metadata Space Total: 838.9 MB
 Metadata Space Available: 838.8 MB
 Thin Pool Minimum Free Space: 4.295 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
...中间省略...
Docker Root Dir: /docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Experimental: false
Insecure Registries:
 0.0.0.0/0
 127.0.0.0/8
Live Restore Enabled: false